Text_Classification
-
AstroConcepts: A Large-Scale Multi-Label Classification Corpus for Astrophysics
2026-04-02 15:27:00Atilla Kaan Alkan, Felix Grezes, Sergi Blanco-Cuaresma, Jennifer Lynn Bartlett, Daniel Chivvis, Anna Kelbert, Kelly Lockhart, Alberto AccomazziAbstract
Scientific multi-label text classification suffers from extreme class imbalance, where specialized terminology exhibits severe power-law distributions that challenge standard classification approaches. Existing scientific corpora lack comprehensive controlled vocabularies, focusing instead on broad categories and limiting systematic study of extreme imbalance. We introduce AstroConcepts, a corpus of English abstracts from 21,702 published astrophysics papers, labeled with 2,367 concepts from the Unified Astronomy Thesaurus. The corpus exhibits severe label imbalance, with 76% of concepts having fewer than 50 training examples. By releasing this resource, we enable systematic study of extreme class imbalance in scientific domains and establish strong baselines across traditional, neural, and vocabulary-constrained LLM methods. Our evaluation reveals three key patterns that provide new insights into scientific text classification. First, vocabulary-constrained LLMs achieve competitive performance relative to domain-adapted models in astrophysics classification, suggesting a potential for parameter-efficient approaches. Second, domain adaptation yields relatively larger improvements for rare, specialized terminology, although absolute performance remains limited across all methods. Third, we propose frequency-stratified evaluation to reveal performance patterns that are hidden by aggregate scores, thereby making robustness assessment central to scientific multi-label evaluation. These results offer actionable insights for scientific NLP and establish benchmarks for research on extreme imbalance.
Abstract (translated)
科学多标签文本分类面临严重的类别不平衡问题，其中专业术语呈现极端的幂律分布，这对标准分类方法构成挑战。现有的科学语料库缺乏全面的受控词汇表，转而聚焦于宽泛类别，限制了针对极端不平衡的系统性研究。我们推出了AstroConcepts语料库，该库包含21,702篇已发表天体物理学论文的英文摘要，并采用《统一天文学术语表》中的2,367个概念进行标注。该语料库表现出严重的标签不平衡，76%的概念仅有不到50个训练样本。通过发布这一资源，我们能够对科学领域的极端类别不平衡进行系统性研究，并为传统方法、神经网络方法以及词汇约束大语言模型方法建立强基准。我们的评估揭示了三个关键模式，为科学文本分类提供了新的见解。首先，在天体物理学分类任务中，词汇约束大语言模型相对于领域适应模型能取得具有竞争力的性能，这表明参数高效方法具有潜力。其次，领域适应对罕见专业术语的提升相对更大，但所有方法的绝对性能仍有限。第三，我们提出频率分层评估方法，以揭示被聚合分数掩盖的性能模式，从而将鲁棒性评估置于科学多标签评估的核心。这些结果为科学自然语言处理提供了可操作的见解，并为极端不平衡研究建立了基准。
URL
https://arxiv.org/abs/2604.02156
PDF
-
From Baselines to Preferences: A Comparative Study of LoRA/QLoRA and Preference Optimization for Mental Health Text Classification
Abstract
Mental health text classification has rapidly adopted modern adaptation methods, yet practical guidance on which optimization strategy to use, when, and why remains limited. This paper presents a systematic comparative study of optimization pathways for a joint mental-health classification task, moving from strong vanilla baselines to progressively more specialized techniques. We first establish classical and encoder references, then examine parameter-efficient supervised fine-tuning with LoRA/QLoRA under multiple objective and optimization settings, and finally evaluate preference-based optimization with DPO, ORPO, and KTO, including class-rebalanced training. Rather than emphasizing a single headline score, we focus on methodological insight: how performance changes with objective formulation, adapter choice, optimizer behavior, context windowing, and class-balance intervention. The results show that optimization effects are highly method-dependent: some approaches deliver stable, transferable gains, while others are sensitive to configuration and data balance. Preference optimization, in particular, exhibits large variation across objectives, indicating that method selection is more consequential than simply adding a preference-training stage. The central contribution is a clear optimization narrative for mental health NLP: start from transparent baselines, apply controlled tuning, and use preference optimization selectively where its gains are demonstrable. This provides a reproducible and practically grounded framework for choosing effective training strategies beyond architecture choice alone.
Abstract (translated)
URL
https://arxiv.org/abs/2604.00773
PDF
-
Can LLMs Infer Conversational Agent Users' Personality Traits from Chat History?
2026-03-31 07:29:32Derya C\"ogendez, Verena Zimmermann, No\'e ZuffereyAbstract
Sensitive information, such as knowledge about an individual's personality, can be can be misused to influence behavior (e.g., via personalized messaging). To assess to what extent an individual's personality can be inferred from user interactions with LLM-based conversational agents (CAs), we analyze and quantify related privacy risks of using CAs. We collected actual ChatGPT logs from N=668 participants, containing 62,090 individual chats, and report statistics about the different types of shared data and use cases. We fine-tuned RoBERTa-base text classification models to infer personality traits from CA interactions. The findings show that these models achieve trait inference with accuracy (ternary classification) better than random in multiple cases. For example, for extraversion, accuracy improves by +44% relative to the baseline on interactions for relationships and personal reflection. This research highlights how interactions with CAs pose privacy risks and provides fine-grained insights into the level of risk associated with different types of interactions.
Abstract (translated)
URL
https://arxiv.org/abs/2604.19785
PDF
-
Meta-Harness: End-to-End Optimization of Model Harnesses
2026-03-30 05:33:50Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, Chelsea FinnAbstract
The performance of large language model (LLM) systems depends not only on model weights, but also on their harness: the code that determines what information to store, retrieve, and present to the model. Yet harnesses are still designed largely by hand, and existing text optimizers are poorly matched to this setting because they compress feedback too aggressively. We introduce Meta-Harness, an outer-loop system that searches over harness code for LLM applications. It uses an agentic proposer that accesses the source code, scores, and execution traces of all prior candidates through a filesystem. On online text classification, Meta-Harness improves over a state-of-the-art context management system by 7.7 points while using 4x fewer context tokens. On retrieval-augmented math reasoning, a single discovered harness improves accuracy on 200 IMO-level problems by 4.7 points on average across five held-out models. On agentic coding, discovered harnesses surpass the best hand-engineered baselines on TerminalBench-2. Together, these results show that richer access to prior experience can enable automated harness engineering.
Abstract (translated)
大型语言模型（LLM）系统的性能不仅取决于模型权重，也取决于其工具链：即决定向模型存储、检索和呈现何种信息的代码。然而，工具链的设计目前仍主要依赖人工完成，而现有的文本优化器因过度压缩反馈信息，与这一场景适配不佳。我们提出Meta-Harness，一种用于搜索LLM应用工具链的外循环系统。该系统采用一个可访问源代码、历史候选方案评分及执行轨迹的智能体式提议器，通过文件系统进行迭代优化。在在线文本分类任务中，Meta-Harness相比最先进的上下文管理系统提升了7.7分，同时仅使用四分之一的上下文令牌。在检索增强型数学推理任务中，单次发现的工具链在200道国际数学奥林匹克难度问题上，平均使五个未参与训练的模型准确率提升4.7分。在智能体编程任务中，发现的工具链在TerminalBench-2基准上超越了最佳人工设计的基线方案。这些结果共同表明，更丰富的历史经验访问能力可实现工具链的自动化工程优化。
URL
https://arxiv.org/abs/2603.28052
PDF
-
Navigating the Prompt Space: Improving LLM Classification of Social Science Texts Through Prompt Engineering
2026-03-26 13:14:48Erkan Gunes, Christoffer Florczak, Tevfik Murat YildirimAbstract
Recent developments in text classification using Large Language Models (LLMs) in the social sciences suggest that costs can be cut significantly, while performance can sometimes rival existing computational methods. However, with a wide variance in performance in current tests, we move to the question of how to maximize performance. In this paper, we focus on prompt context as a possible avenue for increasing accuracy by systematically varying three aspects of prompt engineering: label descriptions, instructional nudges, and few shot examples. Across two different examples, our tests illustrate that a minimal increase in prompt context yields the highest increase in performance, while further increases in context only tend to yield marginal performance increases thereafter. Alarmingly, increasing prompt context sometimes decreases accuracy. Furthermore, our tests suggest substantial heterogeneity across models, tasks, and batch size, underlining the need for individual validation of each LLM coding task rather than reliance on general rules.
Abstract (translated)
社会科学领域利用大语言模型进行文本分类的最新进展表明，该方法可大幅降低成本，且其性能有时能媲美现有计算方法。然而，由于当前测试中性能表现差异较大，我们转而探讨如何最大化性能。本文聚焦于提示上下文，通过系统性地调整提示工程的三个方面——标签描述、指令引导和少样本示例——来提升准确率。针对两个不同案例的测试显示，提示上下文的微小增加能带来最大的性能提升，而进一步增加上下文仅能带来边际效益。令人担忧的是，增加提示上下文有时反而会降低准确率。此外，我们的测试表明不同模型、任务和批处理规模之间存在显著异质性，这凸显了针对每个大语言模型编码任务进行单独验证的必要性，而非依赖通用规则。
URL
https://arxiv.org/abs/2603.25422
PDF
-
Automated Detection of Dosing Errors in Clinical Trial Narratives: A Multi-Modal Feature Engineering Approach with LightGBM
2026-03-25 14:56:34Mohammad AL-SmadiAbstract
Clinical trials require strict adherence to medication protocols, yet dosing errors remain a persistent challenge affecting patient safety and trial integrity. We present an automated system for detecting dosing errors in unstructured clinical trial narratives using gradient boosting with comprehensive multi-modal feature engineering. Our approach combines 3,451 features spanning traditional NLP (TF-IDF, character n-grams), dense semantic embeddings (all-MiniLM-L6v2), domain-specific medical patterns, and transformer-based scores (BiomedBERT, DeBERTa-v3), used to train a LightGBM model. Features are extracted from nine complementary text fields (median 5,400 characters per sample) ensuring complete coverage across all 42,112 clinical trial narratives. On the CT-DEB benchmark dataset with severe class imbalance (4.9% positive rate), we achieve 0.8725 test ROC-AUC through 5-fold ensemble averaging (cross-validation: 0.8833 + 0.0091 AUC). Systematic ablation studies reveal that removing sentence embeddings causes the largest performance degradation (2.39%), demonstrating their critical role despite contributing only 37.07% of total feature importance. Feature efficiency analysis demonstrates that selecting the top 500-1000 features yields optimal performance (0.886-0.887 AUC), outperforming the full 3,451-feature set (0.879 AUC) through effective noise reduction. Our findings highlight the importance of feature selection as a regularization technique and demonstrate that sparse lexical features remain complementary to dense representations for specialized clinical text classification under severe class imbalance.
Abstract (translated)
URL
https://arxiv.org/abs/2604.19759
PDF
-
LGSE: Lexically Grounded Subword Embedding Initialization for Low-Resource Language Adaptation
2026-03-23 23:07:16Hailay Teklehaymanot, Dren Fazlija, Wolfgang NejdlAbstract
Adapting pretrained language models to low-resource, morphologically rich languages remains a significant challenge. Existing vocabulary expansion methods typically rely on arbitrarily segmented subword units, resulting in fragmented lexical representations and loss of critical morphological information. To address this limitation, we propose the Lexically Grounded Subword Embedding Initialization (LGSE) framework, which introduces morphologically informed segmentation for initializing embeddings of novel tokens. Instead of using random vectors or arbitrary subwords, LGSE decomposes words into their constituent morphemes and constructs semantically coherent embeddings by averaging pretrained subword or FastText-based morpheme representations. When a token cannot be segmented into meaningful morphemes, its embedding is constructed using character n-gram representations to capture structural information. During Language-Adaptive Pretraining, we apply a regularization term that penalizes large deviations of newly introduced embeddings from their initialized values, preserving alignment with the original pretrained embedding space while enabling adaptation to the target language. To isolate the effect of initialization, we retain the original pre-trained model vocabulary and tokenizer and update only the new embeddings during adaptation. We evaluate LGSE on three NLP tasks: Question Answering, Named Entity Recognition, and Text Classification, in two morphologically rich, low-resource languages: Amharic and Tigrinya, where morphological segmentation resources are available. Experimental results show that LGSE consistently outperforms baseline methods across all tasks, demonstrating the effectiveness of morphologically grounded embedding initialization for improving representation quality in underrepresented languages. Project resources are available in the GitHub link.
Abstract (translated)
将预训练语言模型适配到低资源、形态丰富的语言仍是一项重大挑战。现有词汇扩展方法通常依赖任意分割的子词单元，导致词汇表示碎片化并丢失关键的形态学信息。为应对这一局限，我们提出了词汇锚定子词嵌入初始化（LGSE）框架，该框架引入基于形态学的分割来初始化新词元的嵌入。LGSE不使用随机向量或任意子词，而是将单词分解为构成语素，并通过平均预训练的子词或基于FastText的语素表示来构建语义连贯的嵌入。当词元无法分割为有意义的语素时，则使用字符n-gram表示来捕获结构信息以构建其嵌入。在语言自适应预训练过程中，我们应用一个正则化项来惩罚新引入嵌入与初始化值之间的较大偏差，从而在保持与原始预训练嵌入空间对齐的同时实现目标语言适配。为隔离初始化效果，我们保留原始预训练模型的词汇表和分词器，并在适配过程中仅更新新嵌入。我们在两种形态丰富且资源稀缺的语言（阿姆哈拉语和提格里尼亚语，其形态分割资源可用）上，于三个NLP任务（问答、命名实体识别和文本分类）中评估了LGSE。实验结果表明，LGSE在所有任务上均持续优于基线方法，证明了基于形态学的嵌入初始化对提升低资源语言表示质量的有效性。项目资源可在GitHub链接获取。
URL
https://arxiv.org/abs/2603.22629
PDF
-
Exploring Data Augmentation and Resampling Strategies for Transformer-Based Models to Address Class Imbalance in AI Scoring of Scientific Explanations in NGSS Classroom
2026-03-21 07:12:10Prudence Djagba, Kevin Haudek, Clare G. C. Franovic, Leonora KaldarasAbstract
Automated scoring of students' scientific explanations offers the potential for immediate, accurate feedback, yet class imbalance in rubric categories particularly those capturing advanced reasoning remains a challenge. This study investigates augmentation strategies to improve transformer-based text classification of student responses to a physical science assessment based on an NGSS-aligned learning progression. The dataset consists of 1,466 high school responses scored on 11 binary-coded analytic categories. This rubric identifies six important components including scientific ideas needed for a complete explanation along with five common incomplete or inaccurate ideas. Using SciBERT as a baseline, we applied fine-tuning and test these augmentation strategies: (1) GPT-4--generated synthetic responses, (2) EASE, a word-level extraction and filtering approach, and (3) ALP (Augmentation using Lexicalized Probabilistic context-free grammar) phrase-level extraction. While fine-tuning SciBERT improved recall over baseline, augmentation substantially enhanced performance, with GPT data boosting both precision and recall, and ALP achieving perfect precision, recall, and F1 scores across most severe imbalanced categories (5,6,7 and 9). Across all rubric categories EASE augmentation substantially increased alignment with human scoring for both scientific ideas (Categories 1--6) and inaccurate ideas (Categories 7--11). We compared different augmentation strategies to a traditional oversampling method (SMOTE) in an effort to avoid overfitting and retain novice-level data critical for learning progression alignment. Findings demonstrate that targeted augmentation can address severe imbalance while preserving conceptual coverage, offering a scalable solution for automated learning progression-aligned scoring in science education.
Abstract (translated)
URL
https://arxiv.org/abs/2604.19754
PDF
-
Enhancing Hyperspace Analogue to Language Representations via Attention-Based Pooling for Text Classification
2026-03-20 17:25:09Ali Sakour, Zoalfekar SakourAbstract
The Hyperspace Analogue to Language (HAL) model relies on global word co-occurrence matrices to construct distributional semantic representations. While these representations capture lexical relationships effectively, aggregating them into sentence-level embeddings via standard mean pooling often results in information loss. Mean pooling assigns equal weight to all tokens, thereby diluting the impact of contextually salient words with uninformative structural tokens. In this paper, we address this limitation by integrating a learnable, temperature-scaled additive attention mechanism into the HAL representation pipeline. To mitigate the sparsity and high dimensionality of the raw co-occurrence matrices, we apply Truncated Singular Value Decomposition (SVD) to project the vectors into a dense latent space prior to the attention layer. We evaluate the proposed architecture on the IMDB sentiment analysis dataset. Empirical results demonstrate that the attention-based pooling approach achieves a test accuracy of 82.38%, yielding an absolute improvement of 6.74 percentage points over the traditional mean pooling baseline (75.64%). Furthermore, qualitative analysis of the attention weights indicates that the mechanism successfully suppresses stop-words and selectively attends to sentiment-bearing tokens, improving both classification performance and model interpretability.
Abstract (translated)
超空间语言类比（HAL）模型依赖全局词共现矩阵构建分布语义表示。尽管这些表示能有效捕捉词汇关系，但通过标准均值池化聚合为句子级嵌入时往往导致信息损失。均值池化对所有词元赋予相同权重，从而稀释了语境显著词汇的影响，同时放大了无信息量的结构词元的作用。本文针对这一局限，在HAL表示流程中集成了可学习的温度缩放可加注意力机制。为缓解原始共现矩阵的稀疏性与高维问题，我们在注意力层前采用截断奇异值分解（SVD）将向量投影至稠密潜在空间。所提架构在IMDB情感分析数据集上进行了评估。实证结果表明，基于注意力池化的方法达到82.38%的测试准确率，相较于传统均值池化基线（75.64%）实现了6.74个百分点的绝对提升。此外，对注意力权重的定性分析显示，该机制成功抑制了停用词，并选择性地聚焦于携带情感信息的词元，从而提升了分类性能与模型可解释性。
URL
https://arxiv.org/abs/2603.20149
PDF
-
TextReasoningBench: Does Reasoning Really Improve Text Classification in Large Language Models?
2026-03-20 01:53:12Xinyu Guo, Yazhou Zhang, Jing QinAbstract
Eliciting explicit, step-by-step reasoning traces from large language models (LLMs) has emerged as a dominant paradigm for enhancing model capabilities. Although such reasoning strategies were originally designed for problems requiring explicit multi-step reasoning, they have increasingly been applied to a broad range of NLP tasks. This expansion implicitly assumes that deliberative reasoning uniformly benefits heterogeneous tasks. However, whether such reasoning mechanisms truly benefit classification tasks remains largely underexplored, especially considering their substantial token and time costs. To fill this gap, we introduce TextReasoningBench, a systematic benchmark designed to evaluate the effectiveness and efficiency of reasoning strategies for text classification with LLMs. We compare seven reasoning strategies, namely IO, CoT, SC-CoT, ToT, GoT, BoC, and long-CoT across ten LLMs on five text classification datasets. Beyond traditional metrics such as accuracy and macro-F1, we introduce two cost-aware evaluation metrics that quantify the performance gain per reasoning token and the efficiency of performance improvement relative to token cost growth. Experimental results reveal three notable findings: (1) Reasoning does not universally improve classification performance: while moderate strategies such as CoT and SC-CoT yield consistent but limited gains (typically +1% to +3% on big models), more complex methods (e.g., ToT and GoT) often fail to outperform simpler baselines and can even degrade performance, especially on small models; (2) Reasoning is often inefficient: many reasoning strategies increase token consumption by 10$\times$ to 100$\times$ (e.g., SC-CoT and ToT) while providing only marginal performance improvements.
Abstract (translated)
从大语言模型（LLMs）中提取显式的逐步推理痕迹，已成为提升模型能力的主流范式。尽管此类推理策略最初专为需要显式多步推理的问题设计，但现已日益广泛应用于各类自然语言处理任务。这种扩展隐含地假设审慎推理能普遍惠及异构任务。然而，此类推理机制是否真正有益于分类任务，尤其是考虑到其高昂的令牌与时间成本，仍 largely 未被充分探索。为填补这一空白，我们推出 TextReasoningBench——一个系统性基准测试，旨在评估 LLM 文本分类中推理策略的有效性与效率。我们在五个文本分类数据集上，对比了七种推理策略（IO、CoT、SC-CoT、ToT、GoT、BoC 及 long-CoT）在十种 LLM 上的表现。除准确率与宏 F1 等传统指标外，我们引入两项成本感知评估指标：量化每推理令牌的性能增益，以及性能提升效率相对于令牌成本增长的比率。实验结果显示三项关键发现：（1）推理并非普遍提升分类性能：适度策略（如 CoT 与 SC-CoT）在大模型上仅带来稳定但有限的增益（通常为 +1% 至 +3%），而复杂方法（如 ToT 与 GoT）常无法超越简单基线，甚至可能损害性能，在小模型上尤为明显；（2）推理往往效率低下：多数策略使令牌消耗增加 10 倍至 100 倍（如 SC-CoT 与 ToT），却仅换取微小的性能提升。
URL
https://arxiv.org/abs/2603.19558
PDF
-
AlignMamba-2: Enhancing Multimodal Fusion and Sentiment Analysis with Modality-Aware Mamba
2026-03-19 03:47:21Yan Li, Yifei Xing, Xiangyuan Lan, Xin Li, Haifeng Chen, Dongmei JiangAbstract
In the era of large-scale pre-trained models, effectively adapting general knowledge to specific affective computing tasks remains a challenge, particularly regarding computational efficiency and multimodal heterogeneity. While Transformer-based methods have excelled at modeling inter-modal dependencies, their quadratic computational complexity limits their use with long-sequence data. Mamba-based models have emerged as a computationally efficient alternative; however, their inherent sequential scanning mechanism struggles to capture the global, non-sequential relationships that are crucial for effective cross-modal alignment. To address these limitations, we propose \textbf{AlignMamba-2}, an effective and efficient framework for multimodal fusion and sentiment analysis. Our approach introduces a dual alignment strategy that regularizes the model using both Optimal Transport distance and Maximum Mean Discrepancy, promoting geometric and statistical consistency between modalities without incurring any inference-time overhead. More importantly, we design a Modality-Aware Mamba layer, which employs a Mixture-of-Experts architecture with modality-specific and modality-shared experts to explicitly handle data heterogeneity during the fusion process. Extensive experiments on four challenging benchmarks, including dynamic time-series (on the CMU-MOSI and CMU-MOSEI datasets) and static image-related tasks (on the NYU-Depth V2 and MVSA-Single datasets), demonstrate that AlignMamba-2 establishes a new state-of-the-art in both effectiveness and efficiency across diverse pattern recognition tasks, ranging from dynamic time-series analysis to static image-text classification.
Abstract (translated)
URL
https://arxiv.org/abs/2603.18462
PDF
-
Uncertainty Estimation for the Open-Set Text Classification systems
2026-03-17 12:25:34Leonid Erlygin, Alexey ZaytsevAbstract
Accurate uncertainty estimation is essential for building robust and trustworthy recognition systems. In this paper, we consider the open-set text classification (OSTC) task - and uncertainty estimation for it. For OSTC a text sample should be classified as one of the existing classes or rejected as unknown. To account for the different uncertainty types encountered in OSTC, we adapt the Holistic Uncertainty Estimation (HolUE) method for the text domain. Our approach addresses two major causes of prediction errors in text recognition systems: text uncertainty that stems from ill formulated queries and gallery uncertainty that is related the ambiguity of data distribution. By capturing these sources, it becomes possible to predict when the system will make a recognition error. We propose a new OSTC benchmark and conduct extensive experiments on a wide range of data, utilizing the authorship attribution, intent and topic classification datasets. HolUE achieves 40-365% improvement in Prediction Rejection Ratio (PRR) over the quality-based SCF baseline across datasets: 365% on Yahoo Answers (0.79 vs 0.17 at FPIR 0.1), 347% on DBPedia (0.85 vs 0.19), 240% on PAN authorship attribution (0.51 vs 0.15 at FPIR 0.5), and 40% on CLINC150 intent classification (0.73 vs~0.52). We make public our code and protocols this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2604.08560
PDF
-
Mastering the Minority: An Uncertainty-guided Multi-Expert Framework for Challenging-tailed Sequence Learning
2026-03-16 13:34:56Ye Wang, Zixuan Wu, Lifeng Shen, Jiang Xie, Xiaoling Wang, Hong Yu, Guoyin WangAbstract
Imbalanced data distribution remains a critical challenge in sequential learning, leading models to easily recognize frequent categories while failing to detect minority classes adequately. The Mixture-of-Experts model offers a scalable solution, yet its application is often hindered by parameter inefficiency, poor expert specialization, and difficulty in resolving prediction conflicts. To Master the Minority classes effectively, we propose the Uncertainty-based Multi-Expert fusion network (UME) framework. UME is designed with three core innovations: First, we employ Ensemble LoRA for parameter-efficient modeling, significantly reducing the trainable parameter count. Second, we introduce Sequential Specialization guided by Dempster-Shafer Theory (DST), which ensures effective specialization on the challenging-tailed classes. Finally, an Uncertainty-Guided Fusion mechanism uses DST's certainty measures to dynamically weigh expert opinions, resolving conflicts by prioritizing the most confident expert for reliable final predictions. Extensive experiments across four public hierarchical text classification datasets demonstrate that UME achieves state-of-the-art performance. We achieve a performance gain of up to 17.97\% over the best baseline on individual categories, while reducing trainable parameters by up to 10.32\%. The findings highlight that uncertainty-guided expert coordination is a principled strategy for addressing challenging-tailed sequence learning. Our code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2603.15708
PDF
-
Towards Privacy-Preserving Machine Translation at the Inference Stage: A New Task and Benchmark
2026-03-16 02:41:06Wei Shao, Lemao Liu, Yinqiao Li, Guoping Huang, Shuming Shi, Linqi SongAbstract
Current online translation services require sending user text to cloud servers, posing a risk of privacy leakage when the text contains sensitive information. This risk hinders the application of online translation services in privacy-sensitive scenarios. One way to mitigate this risk for online translation services is introducing privacy protection mechanisms targeting the inference stage of translation models. However, compared to subfields of NLP like text classification and summarization, the machine translation research community has limited exploration of privacy protection during the inference stage. There is no clearly defined privacy protection task for the inference stage, dedicated evaluation datasets and metrics, and reference benchmark methods. The absence of these elements has seriously constrained researchers' in-depth exploration of this direction. To bridge this gap, this paper proposes a novel "Privacy-Preserving Machine Translation" (PPMT) task, aiming to protect the private information in text during the model inference stage. For this task, we constructed three benchmark test datasets, designed corresponding evaluation metrics, and proposed a series of benchmark methods as a starting point for this task. The definition of privacy is complex and diverse. Considering that named entities often contain a large amount of personal privacy and commercial secrets, we have focused our research on protecting only the named entity's privacy in the text. We expect this research work will provide a new perspective and a solid foundation for the privacy protection problem in machine translation.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14756
PDF
-
Selective Fine-Tuning of GPT Architectures for Parameter-Efficient Clinical Text Classification
2026-03-15 02:16:06Fariba Afrin Irany, Sampson AkwafuoAbstract
The rapid expansion of electronic health record (EHR) systems has generated large volumes of unstructured clinical narratives that contain valuable information for disease identification, patient cohort discovery, and clinical decision support. Extracting structured knowledge from these free-text documents remains challenging because clinical language is highly specialized, labeled datasets are limited, and full fine-tuning of large pretrained language models can require substantial computational resources. Efficient adaptation strategies are therefore essential for practical clinical natural language processing applications. This study proposes a parameter-efficient selective fine-tuning framework for adapting GPT-2 to clinical text classification tasks. Instead of updating the entire pretrained model, the majority of network parameters are frozen, and only the final Transformer block, the final layer normalization module, and a lightweight classification head are updated during training. This design substantially reduces the number of trainable parameters while preserving the contextual representation capabilities learned during pretraining. The proposed approach is evaluated using radiology reports from the MIMIC-IV-Note dataset with automatically derived CheXpert-style labels. Experiments on 50,000 radiology reports demonstrate that selective fine-tuning achieves approximately 91% classification accuracy while updating fewer than 6% of the model parameters. Comparative experiments with head-only training and full-model fine-tuning show that the proposed method provides a favorable balance between predictive performance and computational efficiency. These results indicate that selective fine-tuning offers an efficient and scalable framework for clinical text classification.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14183
PDF
-
CMHL: Contrastive Multi-Head Learning for Emotionally Consistent Text Classification
2026-03-14 18:47:44Menna Elgabry, Ali Hamdi, Khaled ShabanAbstract
Textual Emotion Classification (TEC) is one of the most difficult NLP tasks. State of the art approaches rely on Large language models (LLMs) and multi-model ensembles. In this study, we challenge the assumption that larger scale or more complex models are necessary for improved performance. In order to improve logical consistency, We introduce CMHL, a novel single-model architecture that explicitly models the logical structure of emotions through three key innovations: (1) multi-task learning that jointly predicts primary emotions, valence, and intensity, (2) psychologically-grounded auxiliary supervision derived from Russell's circumplex model, and (3) a novel contrastive contradiction loss that enforces emotional consistency by penalizing mutually incompatible predictions (e.g., simultaneous high confidence in joy and anger). With just 125M parameters, our model outperforms 56x larger LLMs and sLM ensembles with a new state-of-the-art F1 score of 93.75\% compared to (86.13\%-93.2\%) on the dair-ai Emotion dataset. We further show cross domain generalization on the Reddit Suicide Watch and Mental Health Collection dataset (SWMH), outperforming domain-specific models like MentalBERT and MentalRoBERTa with an F1 score of 72.50\% compared to (68.16\%-72.16\%) + a 73.30\% recall compared to (67.05\%-70.89\%) that translates to enhanced sensitivity for detecting mental health distress. Our work establishes that architectural intelligence (not parameter count) drives progress in TEC. By embedding psychological priors and explicit consistency constraints, a well-designed single model can outperform both massive LLMs and complex ensembles, offering a efficient, interpretable, and clinically-relevant paradigm for affective computing.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14078
PDF
-
98$times$ Faster LLM Routing Without a Dedicated GPU: Flash Attention, Prompt Compression, and Near-Streaming for the vLLM Semantic Router
2026-03-13 04:33:53Xunzhuo Liu, Bowei He, Xue Liu, Andy Luo, Haichen Zhang, Huamin ChenAbstract
System-level routers that intercept LLM requests for safety classification, domain routing, and PII detection must be both fast and operationally lightweight: they should add minimal latency to every request, yet not require a dedicated GPU -- an expensive resource better used for LLM inference itself. When the router co-locates on the same GPU as vLLM serving instances, standard attention's $O(n^2)$ memory makes long-context classification (8K--32K tokens) impossible: at 8K tokens, three concurrent classifiers need ${\sim}$4.5\,GB for attention masks alone, far exceeding the memory left by vLLM. We present three staged optimizations for the vLLM Semantic Router, benchmarked on AMD Instinct MI300X, that solve both the latency and the memory problem. \emph{Stage~1}: a custom CK Flash Attention operator for ONNX Runtime on ROCm reduces attention memory from $O(n^2)$ to $O(n)$ and end-to-end (E2E) latency from 4{,}918\,ms to 127\,ms (\textbf{38.7$\times$}), enabling 8K--32K tokens where SDPA OOMs. \emph{Stage~2}: classical NLP prompt compression (TextRank, position weighting, TF-IDF, and novelty scoring) reduces all inputs to ${\sim}$512 tokens without neural inference, capping both latency and GPU memory at a constant regardless of original prompt length (E2E 127$\to$62\,ms, \textbf{2.0$\times$}). \emph{Stage~3}: near-streaming body processing with adaptive chunking and zero-copy JSON eliminates serialization overhead (E2E 62$\to$50\,ms, \textbf{1.2$\times$}). Cumulatively: \textbf{98$\times$} improvement (4{,}918\,ms to 50\,ms), 16K-token routing in 108\,ms, and a total router GPU footprint under 800\,MB -- small enough to share a GPU with LLM serving and removing the need for a dedicated accelerator. Stage~1 targets AMD ROCm (NVIDIA GPUs already have FlashAttention via cuDNN); Stages~2 and~3 are hardware-agnostic.
Abstract (translated)
URL
https://arxiv.org/abs/2603.12646
PDF
-
BTZSC: A Benchmark for Zero-Shot Text Classification Across Cross-Encoders, Embedding Models, Rerankers and LLMs
2026-03-12 14:43:20Ilias AarabAbstract
Zero-shot text classification (ZSC) offers the promise of eliminating costly task-specific annotation by matching texts directly to human-readable label descriptions. While early approaches have predominantly relied on cross-encoder models fine-tuned for natural language inference (NLI), recent advances in text-embedding models, rerankers, and instruction-tuned large language models (LLMs) have challenged the dominance of NLI-based architectures. Yet, systematically comparing these diverse approaches remains difficult. Existing evaluations, such as MTEB, often incorporate labeled examples through supervised probes or fine-tuning, leaving genuine zero-shot capabilities underexplored. To address this, we introduce BTZSC, a comprehensive benchmark of 22 public datasets spanning sentiment, topic, intent, and emotion classification, capturing diverse domains, class cardinalities, and document lengths. Leveraging BTZSC, we conduct a systematic comparison across four major model families, NLI cross-encoders, embedding models, rerankers and instruction-tuned LLMs, encompassing 38 public and custom checkpoints. Our results show that: (i) modern rerankers, exemplified by Qwen3-Reranker-8B, set a new state-of-the-art with macro F1 = 0.72; (ii) strong embedding models such as GTE-large-en-v1.5 substantially close the accuracy gap while offering the best trade-off between accuracy and latency; (iii) instruction-tuned LLMs at 4--12B parameters achieve competitive performance (macro F1 up to 0.67), excelling particularly on topic classification but trailing specialized rerankers; (iv) NLI cross-encoders plateau even as backbone size increases; and (v) scaling primarily benefits rerankers and LLMs over embedding models. BTZSC and accompanying evaluation code are publicly released to support fair and reproducible progress in zero-shot text understanding.
Abstract (translated)
零样本文本分类（ZSC）提供了一种希望，即通过将文本直接匹配到可读的标签描述来消除耗资的任务特定标注。虽然早期的方法主要依赖于为自然语言推理（NLI）微调的交叉编码器模型，但最近在文本嵌入模型、重排序器和指令调整的大规模语言模型（LLM）方面的进步挑战了基于NLI架构的主导地位。然而，系统地比较这些多样化的方法仍然困难重重。现有的评估方法，如MTEB，通常通过监督探针或微调来整合带标签的例子，这使得真正意义上的零样本能力被忽视了。为了解决这一问题，我们介绍了BTZSC，这是一个全面的基准测试集合，包括22个公共数据集，涵盖了情绪、主题、意图和情感分类，覆盖了多样化的领域、类别数量以及文档长度。利用BTZSC，我们在四种主要模型家族之间进行了系统比较：NLI交叉编码器、嵌入模型、重排序器以及指令调整的LLM，包括38个公共和自定义检查点。 我们的结果表明： (i) 现代重排序器，如Qwen3-Reranker-8B，在宏观F1分数上达到了0.72的新最高水平； (ii) 强大的嵌入模型，例如GTE-large-en-v1.5，显著缩小了准确性的差距，并且在精度和延迟之间提供了最佳的权衡； (iii) 参数量为4至12B的指令调整LLM取得了竞争性性能（宏观F1分数最高达0.67），尤其擅长主题分类，但不及专门化的重排序器的表现； (iv) NLI交叉编码器即使随着骨干模型规模的增长也表现停滞； (v) 规模化主要有利于重排序器和LLM超过嵌入模型。 BTZSC及其伴随的评估代码已公开发布，以支持零样本文本理解中的公正且可复现的进步。
URL
https://arxiv.org/abs/2603.11991
PDF
-
An Automatic Text Classification Method Based on Hierarchical Taxonomies, Neural Networks and Document Embedding: The NETHIC Tool
2026-03-12 10:24:14Luigi Lomasto, Rosario Di Florio, Andrea Ciapetti, Giuseppe Miscione, Giulia Ruggiero, Daniele TotiAbstract
This work describes an automatic text classification method implemented in a software tool called NETHIC, which takes advantage of the inner capabilities of highly-scalable neural networks combined with the expressiveness of hierarchical taxonomies. As such, NETHIC succeeds in bringing about a mechanism for text classification that proves to be significantly effective as well as efficient. The tool had undergone an experimentation process against both a generic and a domain-specific corpus, outputting promising results. On the basis of this experimentation, NETHIC has been now further refined and extended by adding a document embedding mechanism, which has shown improvements in terms of performance on the individual networks and on the whole hierarchical model.
Abstract (translated)
这项工作描述了一种自动文本分类方法，该方法被实现在名为NETHIC的软件工具中。NETHIC利用了高度可扩展神经网络的内在能力，并结合了分层分类法的表现力，从而成功地创建了一个既有效又高效的文本分类机制。该工具针对通用语料库和特定领域语料库进行了实验测试，取得了令人鼓舞的结果。基于这些实验结果，NETHIC现已进一步改进并扩充，添加了一种文档嵌入机制，在各个单独网络以及整个分层模型上都显示出性能的提升。
URL
https://arxiv.org/abs/2603.11770
PDF
-
Temporal Text Classification with Large Language Models
2026-03-11 20:49:20Nishat Raihan, Marcos ZampieriAbstract
Languages change over time. Computational models can be trained to recognize such changes enabling them to estimate the publication date of texts. Despite recent advancements in Large Language Models (LLMs), their performance on automatic dating of texts, also known as Temporal Text Classification (TTC), has not been explored. This study provides the first systematic evaluation of leading proprietary (Claude 3.5, GPT-4o, Gemini 1.5) and open-source (LLaMA 3.2, Gemma 2, Mistral, Nemotron 4) LLMs on TTC using three historical corpora, two in English and one in Portuguese. We test zero-shot and few-shot prompting, and fine-tuning settings. Our results indicate that proprietary models perform well, especially with few-shot prompting. They also indicate that fine-tuning substantially improves open-source models but that they still fail to match the performance delivered by proprietary LLMs.
Abstract (translated)
语言随着时间的推移而发生变化。计算模型可以被训练来识别这些变化，从而能够估计文本的出版日期。尽管近年来大型语言模型（LLM）取得了显著进展，但它们在自动确定文本年代上的表现——即时间文本分类（TTC）方面尚未得到充分研究。本研究首次系统地评估了领先的专有模型（Claude 3.5、GPT-4o、Gemini 1.5）和开源模型（LLaMA 3.2、Gemma 2、Mistral、Nemotron 4）在TTC上的表现，使用三个历史语料库进行评估，其中包括两个英语语料库和一个葡萄牙语文料库。我们测试了零样本和少量样本提示以及微调设置的表现。我们的结果显示，专有模型表现良好，特别是在少量样本提示下。此外，结果还表明开源模型在经过微调后性能显著提升，但仍然无法达到专有LLM的水平。
URL
https://arxiv.org/abs/2603.11295
PDF
