Context-Aware Transformer-Based Model for Aspect-Based Sentiment Analysis: A Systematic Literature Review
Model Berbasis Transformer yang Sadar Konteks untuk Analisis Sentimen Berbasis Aspek: Tinjauan Literatur Sistematis
DOI:
https://doi.org/10.21070/joincs.v9i1.1712Keywords:
Aspect-Based Sentiment Analysis, BERT, Transformer, Context-Aware, Multi-Task Learning, Opinion Mining, Scopus, PRISMAAbstract
Aspect-Based Sentiment Analysis (ABSA) is a critical natural language processing task aimed at identifying specific aspects within text and determining the sentiment polarity toward each aspect. Transformer-based models, particularly BERT and its variants, have demonstrated significant advances in ABSA through powerful contextual representations. However, challenges in capturing target-specific context and managing inter-subtask dependencies remain.
This Systematic Literature Review (SLR) identifies, evaluates, and synthesizes current research on context-aware transformer models for ABSA, with emphasis on context-aware mechanisms, multi-task learning approaches, and BERT-family models. Following the PRISMA 2020 protocol, a structured search was conducted on the Scopus database using three Boolean queries, yielding 851 initial records. After deduplication (n=70), title/abstract screening (n=554 excluded), retrieval (n=147 not retrieved), and full-text eligibility assessment (n=48 excluded), 32 studies were included for synthesis.
Three primary model categories were identified: (1) BERT baselines establishing strong end-to-end ABSA performance; (2) context-aware variants employing context-guided attention (CG-BERT, QACG-BERT, LCF-ATEPC, cascade models); and (3) multi-task transformers (BERT-MTL, RoBERTa-MTL, MTL-AraBERT, SABKG, MLEGCN) handling ABSA subtasks jointly. Reported F1-scores ranged from 50–89% across SemEval-2014/2015/2016 and domain-specific datasets. ntext-aware and multi-task transformer models represent the state of the art in ABSA. Open challenges include implicit aspect handling, cross-domain generalization, model efficiency, and evaluation of large generative language models (LLMs) for fine-grained sentiment tasks.
References
[1] Liao, W., Zeng, B., Yin, X., et al. (2021). An improved aspect-category sentiment analysis model for text sentiment analysis based on RoBERTa. *Applied Intelligence*, 51, 3522–3533. https://doi.org/10.1007/S10489-020-01964-1
[2] Zhu, Q., Wang, X., Liu, X., et al. (2023). Multi-task learning for aspect level semantic classification combining complex aspect target semantic enhancement and adaptive local focus. *Mathematical Biosciences and Engineering*, 20(10), 17833–17854. https://doi.org/10.3934/mbe.2023824
[3] Abulnaja, O. A. (2023). Multi-Task Learning Model with Data Augmentation for Arabic Aspect-Based Sentiment Analysis. *CMC-Computers, Materials & Continua*, 74(1), 1–18. https://doi.org/10.32604/cmc.2023.037112
[4] Pei, Y., Wang, Y., Wang, W., et al. (2022). Aspect-Based Sentiment Analysis with Multi-Task Learning. In *Proceedings of ICCBD 2022* (pp. 1–6). IEEE. https://doi.org/10.1109/ICCBD56965.2022.10080017
[5] Fan, X., & Zhang, Z. (2024). A fine-grained sentiment analysis model based on multi-task learning. In *Proceedings of ISCTIS 2024* (pp. 1–6). IEEE. https://doi.org/10.1109/isctis63324.2024.10698954
[6] Aziz, K., Li, F., Chakrabarti, P., et al. (2024). Unifying aspect-based sentiment analysis BERT and multi-layered graph convolutional networks for comprehensive sentiment dissection. *Scientific Reports*, 14, 12345. https://doi.org/10.1038/s41598-024-61886-7
[7] Patel, M., & Ezeife, C. I. (2021). BERT-Based Multi-Task Learning for Aspect-Based Opinion Mining. In *Proceedings of DEXA 2021*, LNCS 12923 (pp. 190–205). Springer. https://doi.org/10.1007/978-3-030-86472-9_18
[8] Pei, Y., Wang, Y., & Wang, W. (2022). A multi-task learning network using shared BERT models for aspect-based sentiment analysis. In *Proceedings of DEIM 2021* (pp. D13-2). https://proceedings-of-deim.github.io/DEIM2021/papers/D13-2.pdf
[9] Ding, H.-Y., Huang, S., Jin, W., et al. (2022). A Novel Cascade Model for End-to-End Aspect-Based Social Comment Sentiment Analysis. *Electronics*, 11(12), 1810. https://doi.org/10.3390/electronics11121810
[10] Karni, S., Avhad, P., Tyagi, R., et al. (2025). Multitask & Meta Learning for Language Models: Enhancing Aspect Based Sentiment Analysis. *Preprint*. https://doi.org/10.21203/rs.3.rs-7654632/v1
[11] He, Z., Chen, Y., & Wang, X. (2023). Multi-task learning model based on BERT and knowledge graph for aspect-based sentiment analysis. *Electronics*, 12(3), 737. https://doi.org/10.3390/electronics12030737
[12] Fadel, A. S., Saleh, M., Salama, R., et al. (2024). MTL-AraBERT: An Enhanced Multi-Task Learning Model for Arabic Aspect-Based Sentiment Analysis. *Computers*, 13(4), 98. https://doi.org/10.3390/computers13040098
[13] Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In *Proceedings of NAACL-HLT 2019* (pp. 4171–4186). ACL. https://doi.org/10.18653/v1/N19-1423
[14] Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention is all you need. In *Advances in Neural Information Processing Systems* (Vol. 30, pp. 5998–6008). NeurIPS.
[15] Liu, Y., Ott, M., Goyal, N., et al. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. *arXiv preprint arXiv:1907.11692*.
[16] Yang, Z., Dai, Z., Yang, Y., et al. (2019). XLNet: Generalized Autoregressive Pretraining for Language Understanding. In *Advances in Neural Information Processing Systems* (Vol. 32). NeurIPS.
[17] Pontiki, M., Galanis, D., Papageorgiou, H., et al. (2014). SemEval-2014 Task 4: Aspect Based Sentiment Analysis. In *Proceedings of SemEval-2014* (pp. 27–35). ACL. https://doi.org/10.3115/v1/S14-2004
[18] Hakim, R., Santoso, B., & Wijaya, A. (2024). Multi-Task Learning Aspect Based Sentiment Analysis with BERT. In *Proceedings of ICITEE 2024* (pp. 1–6). IEEE. https://doi.org/10.1109/icitee62483.2024.10808387
[19] Chauhan, P., Sharma, R., & Gupta, V. (2025). Multi-task learning for unified aspect identification in text reviews. *Expert Systems with Applications*, 267, 128855. https://doi.org/10.1016/j.eswa.2025.128855
[20] Han, J., Kim, S., & Lee, H. (2024). Design Knowledge as Attention Emphasizer in LLM-based Sentiment Analysis. *Journal of Computing and Information Science in Engineering*, 24(6), 061001. https://doi.org/10.1115/1.4067212
[21] He, P., Liu, X., Gao, J., & Chen, W. (2021). DeBERTa: Decoding-enhanced BERT with Disentangled Attention. In *Proceedings of ICLR 2021*. https://arxiv.org/abs/2006.03654
[22] Pontiki, M., Galanis, D., Papageorgiou, H., et al. (2015). SemEval-2015 Task 12: Aspect Based Sentiment Analysis. In *Proceedings of SemEval-2015* (pp. 486–495). ACL.
[23] Li, X., Bing, L., Li, P., & Lam, W. (2019). Exploiting BERT for End-to-End Aspect-based Sentiment Analysis. In *Proceedings of W-NUT 2019* (pp. 34–41). ACL. https://doi.org/10.18653/V1/D19-5505
[24] He, Z., Chen, Y., & Wang, X. (2023). SABKG: Multi-task BERT and knowledge graph for ABSA. *Electronics*, 12(3), 737. https://doi.org/10.3390/electronics12030737
[25] Page, M. J., McKenzie, J. E., Bossuyt, P. M., et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. *BMJ*, 372, n71. https://doi.org/10.1136/bmj.n71
[26] Biswas, S., Roy, A., & Chakraborty, S. (2024). Transformers and Attention: Decoding and Understanding of Aspect-based Opinions in User-Generated Contents. *IEEE Access*, 12, 167890–167905. https://doi.org/10.1109/access.2024.3498440
[27] Wu, Z., & Ong, D. C. (2021). Context-Guided BERT for Targeted Aspect-Based Sentiment Analysis. In *Proceedings of AAAI 2021* (pp. 14094–14102). AAAI. https://doi.org/10.1609/aaai.v35i16.17659
[28] Karimi, A., Rossi, L., & Prati, A. (2020). Improving BERT Performance for Aspect-Based Sentiment Analysis. *arXiv preprint arXiv:2010.11731*. https://arxiv.org/abs/2010.11731
[29] Verma, P., Shukla, N., & Tiwari, A. (2023). IAN-BERT: combining post-trained BERT with interactive attention network for ABSA. *SN Computer Science*, 4(6), 789. https://doi.org/10.1007/s42979-023-02229-7
[30] Xu, L., Chen, M., & Wang, J. (2024). Improving Cross-Domain ABSA using BERT-BiLSTM Model and Dual Attention Mechanism. *Advances in AI and Machine Learning*, 4(3), 1234–1245. https://doi.org/10.54364/aaiml.2024.43144
Published
How to Cite
License
Copyright (c) 2026 Moch. Fauzan, Ika Safitri Windiarti
This work is licensed under a Creative Commons Attribution 4.0 International License.
