Abstract
This study proposes the dual technological innovation framework, including a cross-modal differ entiated quantization framework for vision-language models (VLMs) and a scene-aware vectorized memory multi-agent system for visually impaired assistance. The modular framework was developed implementing differentiated processing strategies, effectively reducing memory requirements from 38GB to 16GB while maintaining model performance. The multi-agent architecture combines scene classification, vectorized memory, and multimodal interaction, enabling persistent storage and efficient retrieval of scene memories. Through perception-memory-reasoning workflows, the system provides environmental information beyond the current view using historical memories. Experiments show the quantized 19B-parameter model only experiences a 2.05% performance drop on MMBench and maintains 63.7 accuracy on OCR-VQA (original: 64.9), outperforming smaller models with equivalent memory requirements like the Molmo-7B series. The system maintains response latency between 2.83-3.52 seconds from scene analysis to initial speech output, substantially faster than non-streaming methods. This research advances computational efficiency and assistive technology, offering visually impaired users comprehensive real-time assistance in scene perception, text recognition, and navigation.
Abstract (translated)
本研究提出了双技术创新框架，包括视觉-语言模型（VLMs）的跨模态差异化量化框架以及面向视障人士辅助的场景感知向量内存多智能体系统。该模块化框架通过实施差异化的处理策略开发而成，在保持模型性能的同时有效将内存需求从38GB降至16GB。多智能体架构结合了场景分类、向量记忆和跨模态交互，能够持久存储并高效检索场景记忆。通过感知-记忆-推理工作流程，系统利用历史记忆提供超出当前视图范围的环境信息。实验表明，量化后的190亿参数模型在MMBench上仅性能下降2.05%，而在OCR-VQA上的准确率保持为63.7%（原始准确率为64.9%），优于具有同等内存需求的小型模型如Molmo-7B系列。从场景分析到初始语音输出，系统响应延迟在2.83至3.52秒之间，远快于非流式处理方法。这项研究推进了计算效率和辅助技术的发展，为视障用户提供全面的实时场景感知、文本识别和导航帮助。
URL
https://arxiv.org/abs/2508.18177
