Ekstraksi Relasi Antar Karakter Pada Teks Satua Bali Menggunakan Support Vector Machine dengan Pendekatan Fitur Leksikon

I Kadek Dwi Adnyana; Ngurah Agus Sanjaya ER; I Ketut Gede Suhartana

doi:10.24252/instek.v11i2.73157

Authors

I Kadek Dwi Adnyana Universitas Udayana
Ngurah Agus Sanjaya ER Universitas Udayana
I Ketut Gede Suhartana Universitas Udayana

DOI:

https://doi.org/10.24252/instek.v11i2.73157

Keywords:

ekstraksi relasi, natural language processing, satua bali, support vector machine, TF-IDF

Abstract

Pemetaan relasi sosial antar tokoh pada Satua Bali penting untuk mendukung preservasi konteks budaya dan pengembangan knowledge graph sastra Nusantara. Penelitian ini mengklasifikasikan relasi Keluarga, Teman, dan Musuh menggunakan Support Vector Machine (SVM) dengan arsitektur Dual-Stream Pipeline yang menggabungkan TF-IDF N-gram pada teks dengan entitas tersamarkan dan fitur leksikon pada teks asli. Teks mentah dikumpulkan dari situs web publik yang memuat cerita Satua Bali, sedangkan kontribusi data penelitian terletak pada penyusunan skema dan anotasi manual tiga kelas relasi. Dari 637 segmen teranotasi, deduplikasi menghasilkan 600 segmen unik yang dievaluasi menggunakan Stratified 5-Fold Cross Validation. Model Dual-Stream mencapai akurasi 75,0% dan Macro F1 75,2%, lebih tinggi dibandingkan baseline TF-IDF pada teks dengan entitas tersamarkan dengan akurasi dan Macro F1 sebesar 71,5%. Sementara itu, TF-IDF pada teks asli mencapai akurasi 83,7%, menunjukkan bahwa nama tokoh dan leksikon asli sangat prediktif, tetapi berpotensi membawa bias identitas. Hasil ini menunjukkan bahwa Dual-Stream mampu memulihkan sebagian informasi relasional akibat masking dan berfungsi sebagai strategi mitigasi terhadap trade-off entity masking.

Downloads

Download data is not yet available.

References

[1] Komang Ria Ariestiyani, I Made Suweta, I Putu Suardipa, and I Ketut Ngurah Ardiawan, “Development Of A Flipbook Of Satua Bali I Lutung Teken I Kakua With Mathematical Numeracy Literacy In Grade Iii Elementary School,” IJPSE Indones. J. Prim. Sci. Educ., vol. 6, no. 1 SE-Articles, pp. 62–70, Nov. 2025, doi: 10.33752/ijpse.v6i1.9222.

[2] N. Made, A. Arsita, N. T. Astawa, U. H. Negeri, G. Bagus, and S. Denpasar, “Suaraku Bahasaku: Dinamika Kesulitan Berbahasa Bali dalam Dunia Siswa SMA Negeri 2 Mengwi,” J. Onoma Pendidikan, Bahasa, dan Sastra, vol. 12, no. 2 SE-Articles, pp. 1525–1547, 2026, doi: 10.30605/khw7f008.

[3] X. Liu, C. Hu, R. Zhang, J. Chen, and B. Xu, “Improving Data Annotation for Low-Resource Relation Extraction with Logical Rule-Augmented Collaborative Language Models,” in Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), L. Chiruzzo, A. Ritter, and L. Wang, Eds., Albuquerque, New Mexico: Association for Computational Linguistics, Apr. 2025, pp. 1497–1510. doi: 10.18653/v1/2025.naacl-long.70.

[4] P. Agustina and R. A. Putri, “Combination of Lexical Resources and Support Vector Machine for Film Sentiment Analysis,” Sink. J. dan Penelit. Tek. Inform., vol. 8, no. 3 SE-, pp. 1526–1538, Jul. 2024, doi: 10.33395/sinkron.v8i3.13733.

[5] A. R. Noor and D. Normawati, “Perbandingan support vector machine dan k-nearest neighbor pada analisis sentimen pelaksanaan pemilihan presiden tahun 2024,” J. Ilm. Penelit. dan Pembelajaran Inform., vol. 11, no. 2, pp. 1021–1030, 2026.

[6] R. Difandana, I. Imaduddin, and I. Indra, “Comparative Analysis of Naive Bayes and Support Vector Machine for Hate Speech Classification,” MALCOM Indones. J. Mach. Learn. Comput. Sci., vol. 6, no. 1, pp. 414–422, 2026, doi: 10.57152/malcom.v6i1.2571.

[7] M. N. Pratama and D. P. Ardianto, “Klasifikasi Pesan SMS Menggunakan Metode TF-IDF dan Support Vector Machine,” Semin. Nas. Teknol. Sains, vol. 5, no. 1 SE-Articles, pp. 151–160, 2026, doi: 10.29407/9y2vn411.

[8] V. Nurcahyawati and Z. Mustaffa, “Vader Lexicon and Support Vector Machine Algorithm to Detect Customer Sentiment Orientation,” J. Inf. Syst. Eng. Bus. Intell., vol. 9, no. 1 SE-Articles, pp. 108–118, Apr. 2023, doi: 10.20473/jisebi.9.1.108-118.

[9] R. H. Muhammadi, T. G. Laksana, and A. B. Arifa, “Combination of Support Vector Machine and Lexicon- Based Algorithm in Twitter Sentiment Analysis,” Khazanah Inform. J. Ilmu Komput. dan Inform., pp. 59–71, 2022.

[10] A. A. Nabil, F. R. Wildantama, D. Satrianto, M. G. Bakara, I. Budiawan, and D. Mulyati, “Klasifikasi Hoax Menggunakan Metode TF-IDF + SVM: Penelitian,” J. Pengabdi. Masy. dan Ris. Pendidik., vol. 4, no. 3 SE-Articles, pp. 17653–17660, Jan. 2025, doi: 10.31004/jerkin.v4i3.5078.

[11] F. Koto, A. Rahimi, J. H. Lau, and T. Baldwin, “IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP,” in Proceedings of the 28th International Conference on Computational Linguistics, D. Scott, N. Bel, and C. Zong, Eds., Barcelona, Spain (Online): International Committee on Computational Linguistics, Dec. 2020, pp. 757–770. doi: 10.18653/v1/2020.coling-main.66.

[12] W. Wongso, D. S. Setiawan, S. Limcorn, and A. Joyoadikusumo, “NusaBERT: Teaching IndoBERT to be Multilingual and Multicultural,” in Proceedings of the Second Workshop in South {E}ast {A}sian Language Processing, D. Wijaya, A. F. Aji, C. Vania, G. I. Winata, and A. Purwarianti, Eds., Online: Association for Computational Linguistics, Jan. 2025, pp. 10–26. [Online]. Available: https://aclanthology.org/2025.sealp-1.2/

[13] N. Fajriyah, N. T. Lapatta, D. W. Nugraha, and R. Laila, “Implementasi svm dan smote pada analisis sentimen media sosial x terhadap pelantikan agus harimurti yudhoyono,” J. Ilm. Penelit. dan Pembelajaran Inform., vol. 10, no. 2, pp. 1359–1370, 2025.

[14] C. Alt, A. Gabryszak, and L. Hennig, “Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault, Eds., Online: Association for Computational Linguistics, Jul. 2020, pp. 1534–1545. doi: 10.18653/v1/2020.acl-main.140.

[15] E. Pirade, I. K. Gede, D. Putra, D. Purnami, and S. Putri, “Automated Generation of Folklore Short Stories Using T5 Transformer Model,” J. Appl. Informatics Comput., vol. 9, no. 5, pp. 2338–2346, 2025.

[16] K. Lee, D. Eck, D. Ippolito, A. Nystrom, C. Callison-burch, and N. Carlini, “Deduplicating Training Data Makes Language Models Better,” Proc. 60th Annu. Meet. Assoc. Comput. Linguist., 2021.

[17] S. Martínez, D. Sánchez, A. Valls, and M. Batet, “Privacy protection of textual attributes through a semantic-based masking method,” Inf. Fusion, vol. 13, no. 4, pp. 304–314, 2012, doi: https://doi.org/10.1016/j.inffus.2011.03.004.

[18] G. Park, J. Rayz, and L. Pouchard, “Figure Descriptive Text Extraction using Ontological Representation,” 2009.

[19] G. M. Emelyanov, D. V Mikhailov, and A. P. Kozlov, “The TF-IDF measure and analysis of links between words within N-grams in the formation of knowledge units for open tests,” Pattern Recognit. Image Anal., vol. 27, no. 4, pp. 825–831, 2017, doi: 10.1134/S1054661817040058.

[20] J. Hou, X. Li, R. Zhu, C. Zhu, Z. Wei, and C. Zhang, “A Neural Relation Extraction Model for Distant Supervision in Counter-Terrorism Scenario,” IEEE Access, vol. 8, pp. 225088–225096, 2020, doi: 10.1109/ACCESS.2020.3042672.

[21] Z. Yang, Z. Li, Z. Xu, Z. Gan, and W. Cao, “A Joint Relation Extraction Model Based on Domain N-Gram Adapter and Axial Attention for Military Domain,” in Web Information Systems and Applications, L. Yuan, S. Yang, R. Li, E. Kanoulas, and X. Zhao, Eds., Singapore: Springer Nature Singapore, 2023, pp. 233–245.

[22] I. M. Wahyu, G. Negara, N. Agus, and S. Er, “Basic Word Extraction Algorithm Based on Morphological Rules for Balinese Texts,” vol. 8, no. 4, pp. 401–410, 2020.

[23] A. Nurkasanah and M. Hayaty, “Feature Extraction using Lexicon on the Emotion Recognition Dataset of Indonesian Text,” Ultim. J. Tek. Inform., vol. 14, no. 1, pp. 20–27, 2022, doi: 10.31937/ti.v14i1.2540.

[24] M. Taboada, J. Brooke, M. Tofiloski, K. Voll, and M. Stede, “Lexicon-basedmethods for sentiment analysis,” Comput. Linguist., vol. 37, no. 2, pp. 267–307, 2011, doi: 10.1162/COLI_a_00049.

[25] A. Siddiqua, V. Bindumathi, G. Raghu, and Y. S. Vamsi Bhargav, “Aspect-based Sentiment Analysis (ABSA) using Machine Learning Algorithms,” in 2024 Third International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE), 2024, pp. 1–6. doi: 10.1109/ICDCECE60827.2024.10549140.

[26] E. A. Kyei et al., “A Hybrid Sentiment Analysis Model to Detect Racist Tweets Using Lexicon-Based Sentiment Analysis and a Support Vector Machine Algorithm,” in AI Revolution: Research, Ethics and Society, H. R. Arabnia, L. Deligiannidis, S. Amirian, F. Ghareh Mohammadi, and F. Shenavarmasouleh, Eds., Cham: Springer Nature Switzerland, 2026, pp. 365–389.

[27] M. Mittal, H. M. Al-Jawahry, N. Varshney, S. P. Kumar, J. Michaelson, and R. A. Reddy, “Improving Support Vector Machine Performance with Advanced Kernel Methods,” in 2024 7th International Conference on Contemporary Computing and Informatics (IC3I), 2024, pp. 1749–1754. doi: 10.1109/IC3I61595.2024.10828664.

[28] M. Li, Y. Liu, and J. Wang, “A new parameter optimization algorithm of SVM,” in 2011 International Conference on Advanced Intelligence and Awareness Internet (AIAI 2011), 2011, pp. 174–178. doi: 10.1049/cp.2011.1451.

[29] J. Rusman, B. Z. Haryati, and A. Michael, “Optimisasi Hiperparameter Tuning pada Metode Support Vector Machine untuk Klasifikasi Tingkat Kematangan Buah Kopi,” J. Komput. dan Inform., vol. 11, no. 2, pp. 195–202, 2023, doi: 10.35508/jicon.v11i2.12571.

[30] Siti Kania Nur Alya Putri, Indah Jumiatin, Indri Sulistia, Novan Alkaf Bahraini Saputra, and Nuruddin Wiranda, “Implementation of Hyperparameter Tuning for Classification Models in Heart Disease Risk Prediction,” vol. 5, no. October, pp. 1181–1189, 2025.

[31] B. Astario, T. Tukino, A. Hananto, F. Nurapriani, and E. Novalia, “Text Data Classification Using the SVM Model on the LMDB Minecraft Dataset,” J. Inf. dan Teknol., pp. 21–26, May 2025, doi: 10.60083/jidt.vi0.620.

[32] I. G. M. N. A. Bhawanaputra, I. G. I. Sudipa, N. P. S. Meinarni, I. G. A. A. M. Aristamy, and I. Pratistha, “SVM-Based Pediatric Disease Classification Model from the Balinese Lontar Usada Rare Manuscript,” Sink. J. dan Penelit. Tek. Inform., vol. 10, no. 1, pp. 481–494, 2026, doi: 10.33395/sinkron.v10i1.15508.

[33] V. W. Lumumba, D. Kiprotich, M. L. Mpaine, N. G. Makena, and M. D. Kavita, “Comparative Analysis of Cross-Validation Techniques: LOOCV, K-folds Cross-Validation, and Repeated K-folds Cross-Validation in Machine Learning Models,” Am. J. Theor. Appl. Stat., vol. 13, no. 5, pp. 127–137, 2024, doi: 10.11648/j.ajtas.20241305.13.

[34] I. Markoulidakis and G. Markoulidakis, “Probabilistic Confusion Matrix: A Novel Method for Machine Learning Algorithm Generalized Performance Analysis,” Technologies, vol. 12, no. 7, 2024, doi: 10.3390/technologies12070113.

[35] M. Gong, “A Novel Performance Measure for Machine,” vol. 13, no. 1, pp. 1–19, 2021, doi: 10.5121/ijmit.2021.13101.

[36] G. Naidu, T. Zuva, and E. M. Sibanda, “A Review of Evaluation Metrics in Machine Learning Algorithms,” in Artificial Intelligence Application in Networks and Systems, R. Silhavy and P. Silhavy, Eds., Cham: Springer International Publishing, 2023, pp. 15–25.

[37] A. Tilevik, “Classification and Performance Metrics,” in Multivariate Statistics and Machine Learning in R For Beginners: With Applications in Biology and Medicine, Cham: Springer Nature Switzerland, 2025, pp. 147–170. doi: 10.1007/978-3-032-01851-9_10.