Komparasi Naive Bayes dan Support Vector Machine dalam Mendeteksi Sarkasme Teks Twitter (X)

Abstract

The use of non-standard language on social media has become one of the challenges in detecting sarcasm because the meaning conveyed is not always recognizable based solely on the words present in the text. This study aims to compare the performance of Naive Bayes and Support Vector Machine (SVM) in detecting sarcasm in non-standard Indonesian Twitter text (X). The Wilcoxon Signed-Rank Test is used to test the significance of the performance difference between the two algorithms. The audit results in 15,072 unique tweets, consisting of 12,057 training data and 3,015 test data. The test results show that Naive Bayes achieved an accuracy of 69.45%, precision of 67.66%, recall of 64.19%, and F1-Score of 65.88%, while SVM achieved an accuracy of 68.39%, precision of 66.51%, recall of 62.82%, and F1-Score of 64.61%. In the 5-fold cross-validation, Naive Bayes produced an average accuracy of 68.74% with a standard deviation of ±0.33%, while SVM achieved 68.17% with a standard deviation of ±1.14%. The Wilcoxon Signed-Rank Test results in a p-value of 0.4375 (>0.05), indicating that the performance difference between the two algorithms is not statistically significant. Thus, Naive Bayes shows slightly higher evaluation results numerically, but it cannot yet be stated that it has significantly superior performance compared to SVM on this research dataset.