Abstract
In the rapidly evolving landscape of enterprise natural language processing (NLP), the demand for efficient, lightweight models capable of handling multi-domain text automation tasks has intensified. This study conducts a comparative analysis of three prominent lightweight Transformer models - DistilBERT, MiniLM, and ALBERT - across three distinct domains: customer sentiment classification, news topic classification, and toxicity and hate speech detection. Utilizing datasets from IMDB, AG News, and the Measuring Hate Speech corpus, we evaluated performance using accuracy-based metrics including accuracy, precision, recall, and F1-score, as well as efficiency metrics such as model size, inference time, throughput, and memory usage. Key findings reveal that no single model dominates all performance dimensions. ALBERT achieves the highest task-specific accuracy in multiple domains, MiniLM excels in inference speed and throughput, and DistilBERT demonstrates the most consistent accuracy across tasks while maintaining competitive efficiency. All results reflect controlled fine-tuning under fixed enterprise-oriented constraints rather than exhaustive hyperparameter optimization. These results highlight trade-offs between accuracy and efficiency, recommending MiniLM for latency-sensitive enterprise applications, DistilBERT for balanced performance, and ALBERT for resource-constrained environments.
Abstract (translated)
在企业自然语言处理（NLP）迅速发展的背景下，对能够高效处理多领域文本自动化任务的轻量级模型的需求日益增加。本研究对三种突出的轻量级Transformer模型——DistilBERT、MiniLM和ALBERT，在客户情感分类、新闻主题分类以及毒性与仇恨言论检测三个不同领域进行了比较分析。我们使用IMDb、AG News以及衡量仇恨言论语料库的数据集，通过包括准确率、精确度、召回率和F1值在内的基于准确性的指标，以及模型大小、推理时间、吞吐量和内存使用等效率指标进行性能评估。 研究的关键发现表明，没有单一的模型在所有性能维度上占据主导地位。ALBERT在多个领域的特定任务准确性方面表现出最高水平，MiniLM在推理速度和吞吐量方面表现优异，而DistilBERT则展现了其跨任务的一致准确性和相对高效的特性。所有结果均基于受控的微调，在企业导向的固定约束下进行，而非经过详尽的超参数优化。 这些结果突出了准确性与效率之间的权衡，并建议对于延迟敏感的企业应用选择MiniLM，追求平衡性能时使用DistilBERT，而针对资源受限环境则推荐ALBERT。
URL
https://arxiv.org/abs/2601.00444
