Abstract
Aspect-based sentiment analysis (ABSA) in natural language processing enables organizations to understand customer opinions on specific product aspects. While deep learning models are widely used for English ABSA, their application in Arabic is limited due to the scarcity of labeled data. Researchers have attempted to tackle this issue by using pre-trained contextualized language models such as BERT. However, these models are often based on fact-based data, which can introduce bias in domain-specific tasks like ABSA. To our knowledge, no studies have applied adaptive pre-training with Arabic contextualized models for ABSA. This research proposes a novel approach using domain-adaptive pre-training for aspect-sentiment classification (ASC) and opinion target expression (OTE) extraction. We examine fine-tuning strategies - feature extraction, full fine-tuning, and adapter-based methods - to enhance performance and efficiency, utilizing multiple adaptation corpora and contextualized models. Our results show that in-domain adaptive pre-training yields modest improvements. Adapter-based fine-tuning is a computationally efficient method that achieves competitive results. However, error analyses reveal issues with model predictions and dataset labeling. In ASC, common problems include incorrect sentiment labeling, misinterpretation of contrastive markers, positivity bias for early terms, and challenges with conflicting opinions and subword tokenization. For OTE, issues involve mislabeling targets, confusion over syntactic roles, difficulty with multi-word expressions, and reliance on shallow heuristics. These findings underscore the need for syntax- and semantics-aware models, such as graph convolutional networks, to more effectively capture long-distance relations and complex aspect-based opinion alignments.
Abstract (translated)
在自然语言处理领域，基于方面的情感分析（ABSA）使组织能够理解客户对产品特定方面的意见。虽然深度学习模型广泛用于英语的ABSA任务，但由于缺乏标注数据，在阿拉伯语中的应用却受到限制。研究人员尝试通过使用像BERT这样的预训练上下文化语言模型来解决这一问题。然而，这些基于事实的数据的模型在领域特定任务如ABSA中可能引入偏差。据我们所知，尚未有研究将自适应预训练与阿拉伯语上下文化模型应用于ABSA。 这项研究提出了一种新颖的方法，使用领域自适应预训练来进行方面情感分类（ASC）和意见目标表达（OTE）提取。我们探讨了三种微调策略：特征抽取、全量微调以及基于适配器的方法，以提高性能和效率，并利用多种适应语料库和上下文化模型进行测试。我们的结果显示，在领域内自适应预训练可获得适度改进。基于适配器的微调是一种计算效率高的方法，能够取得竞争性的结果。然而，错误分析揭示了与模型预测及数据集标注有关的问题。 在ASC中，常见问题包括情感标签不正确、对比标志误读、早期术语积极偏见以及对立意见和子词分词的挑战。对于OTE而言，则涉及目标标记不正确、句法角色混淆、多词表达困难以及对浅层启发式方法依赖等问题。这些发现强调了使用语法和语义感知模型（如图卷积网络）以更有效地捕捉长距离关系及复杂的基于方面意见对齐的重要性。
URL
https://arxiv.org/abs/2509.16788
