Abstract
Handwritten STEM exams capture open-ended reasoning and diagrams, but manual grading is slow and difficult to scale. We present an end-to-end workflow for grading scanned handwritten engineering quizzes with multimodal large language models (LLMs) that preserves the standard exam process (A4 paper, unconstrained student handwriting). The lecturer provides only a handwritten reference solution (100%) and a short set of grading rules; the reference is converted into a text-only summary that conditions grading without exposing the reference scan. Reliability is achieved through a multi-stage design with a format/presence check to prevent grading blank answers, an ensemble of independent graders, supervisor aggregation, and rigid templates with deterministic validation to produce auditable, machine-parseable reports. We evaluate the frozen pipeline in a clean-room protocol on a held-out real course quiz in Slovenian, including hand-drawn circuit schematics. With state-of-the-art backends (GPT-5.2 and Gemini-3 Pro), the full pipeline achieves $\approx$8-point mean absolute difference to lecturer grades with low bias and an estimated manual-review trigger rate of $\approx$17% at $D_{\max}=40$. Ablations show that trivial prompting and removing the reference solution substantially degrade accuracy and introduce systematic over-grading, confirming that structured prompting and reference grounding are essential.
Abstract (translated)
手写STEM考试能够捕捉到开放性推理和图表，但人工评分既缓慢又难以扩展。我们提出了一种端到端的工作流程，用于使用多模态大型语言模型（LLM）对扫描的手写工程小测验进行评分，该工作流保持了标准的考试过程（A4纸张、不受限制的学生手写）。讲师只需提供一份手写的参考答案（100%正确率）和一组简短的评分规则；参考答案被转换成仅包含文本的摘要，以条件化评分而不暴露参考扫描。可靠性通过多阶段设计实现，包括格式/存在检查以防止空白答案被评分、独立评分级联、监督聚合以及使用确定性验证的固定模板来生成可审计和机器解析报告。 我们在一个清洁房协议中对一项保留的真实课程测验进行了冻结管道的评估，该测验包含手绘电路图，并且是用斯洛文尼亚语进行的。在最先进的后台（GPT-5.2 和 Gemini-3 Pro）支持下，整个工作流程与讲师评分相比达到了约8分的平均绝对差异，偏差较低，并且估计的手动审查触发率为约17%（当最大分数D为40时）。消融研究表明，简单的提示和移除参考答案会显著降低准确性并引入系统性的过度打分，确认结构化提示和参考依据是必不可少的。
URL
https://arxiv.org/abs/2601.00730
