Abstract
Large language models (LLMs) are increasingly being used for text classification across the social sciences, yet researchers overwhelmingly classify one text per variable per prompt. Coding 100,000 texts on four variables requires 400,000 API calls. Batching 25 items and stacking all variables into a single prompt reduces this to 4,000 calls, cutting token costs by over 80%. Whether this degrades coding quality is unknown. We tested eight production LLMs from four providers on 3,962 expert-coded tweets across four tasks, varying batch size from 1 to 1,000 items and stacking up to 25 coding dimensions per prompt. Six of eight models maintained accuracy within 2 pp of the single-item baseline through batch sizes of 100. Variable stacking with up to 10 dimensions produced results comparable to single-variable coding, with degradation driven by task complexity rather than prompt length. Within this safe operating range, the measurement error from batching and stacking is smaller than typical inter-coder disagreement in the ground-truth data.
Abstract (translated)
大型语言模型（LLMs）正日益被用于社会科学领域的文本分类，但研究者们绝大多数情况下每个提示仅对单个文本的单个变量进行编码。若对10万条文本的四个变量进行编码，需调用40万次API。而通过批量处理25个项目并将所有变量整合到单个提示中，可将调用次数降至4,000次，使token成本降低超过80%。然而，这种方法是否会降低编码质量尚不明确。我们在四个任务中测试了来自四家提供商的八个生产级LLM，使用3,962条专家编码的推文作为数据，将批量大小从1到1,000不等进行测试，并在单个提示中堆叠最多25个编码维度。结果显示，八个模型中有六个在批量达到100时，准确率仍保持在单条处理基准的2个百分点以内。当堆叠变量维度不超过10个时，其效果与单变量编码相当，性能下降主要由任务复杂度而非提示长度导致。在此安全操作范围内，批量处理和变量堆叠带来的测量误差小于真实数据中常见的编码者间分歧。
URL
https://arxiv.org/abs/2604.03684
