Recognition
-
Replay Pruning and Few-Bit Integer Inference for Hardware-Efficient Speed-Sign Recognition
2026-10-16 01:41:06Se Un Park, Kyungkeon Chung, Huigeol Kim, Junyoung ParkAbstract
Speed-limit sign recognition on an automotive camera system-on-chip (SoC) must fit a detector and a classifier (the reader) into the memory, arithmetic, and energy budgets of the device. In our two-stage recognizer, the detector, a YOLO-family network with ReLU and hard-sigmoid operators, is compressed by channel replay: an iterative pruning cascade identifies the channels to remove, the recorded deletions are replayed on the original task-trained network, the parent, and one continuous knowledge-distillation recovery follows. On a YOLOv5n, replay removes 60.3% of the parameters at the accuracy of the parent and, in validation mAP50-95 over five matched seeds, outperforms iterative pruning and training from scratch, and its per-step variant (per-step replay) is the most accurate pruning method. On a YOLOv8n, replay and per-step replay outperform every other method and the unpruned parent. The reader, a five-layer CNN with 18k parameters, and the post-processing of the YOLOv5n and YOLOv8n heads use integer arithmetic only. With 4-bit weights and 8-bit activations (W4A8), the reader matches FP32 accuracy in 12.1 KB of on-chip memory, and an executable streaming model reproduces the INT8 class predictions of its runtime on all 9,850 test crops. On a Jetson AGX Orin, INT8 is faster than FP16 only in the implicit formulation of TensorRT, and the replay detector runs fastest on the deep-learning accelerator (DLA). Trained on public data only, the combined system of 0.72 M parameters recognizes 95.2% of the signs of a held-out automotive camera at 90% precision and 94 frames per second.
Abstract (translated)
URL
https://arxiv.org/abs/2610.22811
PDF
-
Improving OCR Accuracy Through Multi-Classifier Verification of Visually Similar Characters
2026-10-15 22:56:34Nuzhat Khan, Ab Al-Hadi Ab Rahman, Mohamad Anuar Kamaruddin, Muhammad Paend Bakht, Sana Fatima, Farhad Khodadadi, Ibrahim Yousef AlshareefAbstract
Distinguishing visually similar characters is a persistent challenge for optical character recognition (OCR) systems, causing errors in critical applications of text digitization. While preprocessing techniques improve image quality and word-level post-processing corrects spelling errors, neither addresses character-level visual ambiguity, an intrinsic property of certain character pairs that limits even state-of-the-art OCR models. We present the Character Verification Model (CVM), a character-level post-processing framework that verifies and corrects ambiguous characters in OCR outputs using specialized binary classifiers trained on visually confusable pairs such as 'O'/'0', 'Z'/'2', and 'S'/'5'. CVM operates independently of dictionary validation, making it suitable for non-lexical data including serial numbers, alphanumeric codes, and license plates. Experiments on three OCR engines (Tesseract, EasyOCR, TrOCR) and ten fonts with low to high complexity demonstrate accuracy improvements of 3.95-14.81 percentage points, with an 87.3% correction rate for visually similar character errors and a false-correction rate of only 2.3%. The modular architecture integrates with existing OCR pipelines without modification to underlying recognition models.
Abstract (translated)
URL
https://arxiv.org/abs/2610.22726
PDF
-
DINO Eats Geometry: Self-Supervised Sign Representations that Absorb Articulated Hand Geometry
2026-10-15 20:29:41Junyi Hu, Zhewen He, Haomian Huang, Zhenhua Li, Zhifei Li, Shuaihang Yuan, Yi FangAbstract
Handshape and orientation are geometric components of sign language, yet many sign encoders learn from appearance alone. We introduce SignDEG (DINO Eats Geometry), a temporal sign encoder that uses articulated hand geometry as a pre-training target and requires only video at inference. Frozen DINOv3 features of tracked hands and faces feed per-stream temporal Transformers. Geometric latent prediction (GLP) supervises an intermediate block with WiLoR latents, MANO articulation, and 3D joints, including at masked frames. Geometric relation transfer (GRT) matches cross-video frame-similarity distributions at the final block. Both objectives are learned jointly with temporal self-distillation; WiLoR and the geometry heads are then discarded. Under the SHuBERT public-data evaluation protocol, SignDEG reaches 17.9/24.5 BLEU-4 on How2Sign/OpenASL, compared with 16.2/23.2 for SHuBERT. Rank-1 adaptation improves ASL Citizen and Sem-Lex recognition by 0.02 Recall@1 each, and phonological classification improves across 16 attributes. Removing GRT while retaining GLP reduces How2Sign BLEU-4 by 1.4 and ASL Citizen Recall@1 by 0.034. The results support transferring articulated structure into an appearance encoder without running a hand reconstruction model at deployment.
Abstract (translated)
URL
https://arxiv.org/abs/2609.06296
PDF
-
Three LiDARs, Twenty-Five Datasets: Cross-Source Generalization in Tree-Species Recognition
Abstract
FOR-species20K is the largest public LiDAR benchmark for tree-species classification, pooling 25 independently collected terrestrial, mobile, and airborne laser-scanning datasets across 33 species. We introduce a multi-resolution, multi-view ConvNeXtV2 model that improves official hidden-test accuracy from 79.5% to 85.0%, ranking first at the time of submission. We then examine what this score measures. Because the hidden test is species-balanced, its overall accuracy is more comparable to a macro metric than to accuracy under the public set's long-tailed distribution. Moreover, the official random split places every source dataset in both training and evaluation. A logistic probe recovers source identity from out-of-fold model probabilities with 69% accuracy, while geometric features identify the source above 90% within two widely shared species; 18 of the 33 species occur in only one source. Under a source-species-cell holdout, evaluated on 5,566 trees from eight eligible species that retain the same LiDAR-type training support in other sources, the unweighted species mean falls from 82% to 36%, and tree-weighted accuracy falls from 95% to 60%. Count-matched retraining and experiments with two alternative backbones indicate that this gap is not explained by reduced training support alone, while an external PureForest experiment shows a smaller but consistent grouped-split effect. These results establish the 85% leaderboard score as strong in-distribution performance, but not as an estimate of source-disjoint generalization, motivating balanced metrics and source-aware evaluation. The code is available at: this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2610.22606
PDF
-
At the Limits of Computability: Steps Towards Autonomous AI Discovery on the Busy Beaver Frontier
2026-10-15 19:56:48Marco Dos Santos, Vivek Veeriah, Alex Havrilla, Andrew Bolt, Goran Zuzic, Satinder Singh, Tom ZahavyAbstract
Recent breakthroughs in AI for mathematics---ranging from Olympiad medal performances to resolving long-standing open conjectures---have rapidly expanded what automated systems can achieve. Yet these successes primarily concern predefined, human-formulated problems, leaving open how AI systems can autonomously uncover mathematical structure and formulate new questions. To push beyond static challenges, we explore the Busy Beaver (BB) domain, an uncomputable frontier that is fundamentally open-ended. Because no algorithm can compute the Busy Beaver number $\mathrm{BB}(n)$ for every $n$, this domain provides a natural setting for sustained exploration through mathematical conjecture and proof. In this paper, we introduce a Gemini-based agentic pipeline that automatically discovers accelerations of Turing machines, expressed as compact mathematical rules summarizing many execution steps. Our system integrates pattern recognition, automated conjecturing, informal mathematical reasoning, and formal theorem proving in Lean. With this framework, our system autonomously discovered and formally verified novel Collatz-like accelerations for eleven machines in $\mathrm{BB}(6)$. Subject to community acceptance, we provisionally classify these machines as Cryptids, namely machines whose blank-tape executions admit exact, compact mathematical descriptions but whose halting behavior remains unresolved. These results advance the analysis of $\mathrm{BB}(6)$ holdouts on the Busy Beaver frontier and demonstrate the potential of autonomous AI systems for open-ended mathematical discovery in fundamental computer science.
Abstract (translated)
URL
https://arxiv.org/abs/2610.22603
PDF
-
RTISSLAM-3D: Full 6-DoF SLAM With an In-Air 3D Imaging Sonar
2026-10-15 18:47:38Jan Steckel, Vincent Katter, Wouter JansenAbstract
Echolocating bats navigate dense forests using sonar alone, and in-air 3D imaging sonars now capture the entire frontal hemisphere with every pulse. Sonar SLAM, however, has so far been planar: current systems estimate a 2D trajectory and discard the elevation information in the sonar images, an assumption that does not hold on rough and sloped terrain such as a forest floor. In this paper, we introduce RTISSLAM-3D, a novel 6-DoF SLAM system for a wheeled robot that carries an in-air 3D imaging sonar, an IMU and wheel encoders. A CFAR detector on the sphere reduces every sonar image to a sparse set of 3D detections, which we use for a 3D acoustic odometry that models the shape of the sonar beam, for place recognition with polar submap descriptors, and for the verification of loop closures by 3D registration, all combined in a 6-DoF pose graph with attitude factors. We evaluated RTISSLAM-3D on a 545m drive through a forest, with a 3D lidar SLAM as reference. The loop closures reduced the horizontal error of the acoustic odometry from 1.74m to 0.28m, with a height error of 0.12m, and 94.6% of the resulting 3D sonar map lies within 0.3 m of the lidar map.
Abstract (translated)
URL
https://arxiv.org/abs/2610.22552
PDF
-
Grounded Human-Attributed Description and Activity Recognition in Videos
2026-10-15 17:44:55Jay N. Paranjape, Jurijs Nazarovs, Rohan Sarkar, Yue Guo, Lipeng Ke, Sankar Venkataraman, Vishal M. Patel, Nataraj JammalamadakaAbstract
Understanding characteristics and activities of individuals in complex multi-person environments is crucial for real-world applications. However, existing datasets often simplify this problem to grounded group activity recognition, single-person activity recognition, or close-set activity recognition, thus limiting the generalizability of models trained on these datasets. In this work, we introduce the task of Grounded Human-Attributed Description and Activity Recognition (GHADAR), which involves describing the characteristics and activity descriptions of \textbf{every} person in a video, provided the location of the person in the video, which is more practical for real-world applications. To facilitate this, we introduce a new dataset derived from AVA-Actions by generating open-set captions for the person description and activity. In addition, we propose a novel method to effectively utilize the information contained in grounding during training by constraining the cross-attention masks during training in VLMs to improve performance for this task. Our experiments show that our method outperforms SOTA VLMs on this task. Finally, we demonstrate the limitations of existing evaluation metrics, which are overly reliant on human-annotations and exact text-text matching. As an added video-based evaluation, we propose a holistic VLM-based evaluation schema that compares concepts {\em directly} between the video and the generated predictions. Thus, in this work, we develop a complete framework for GHADAR, including a dataset, a novel method and an evaluation schema, thereby establishing a strong foundation for future research in this domain.
Abstract (translated)
URL
https://arxiv.org/abs/2610.22295
PDF
-
Pathology foundation models for training-free case retrieval in salivary gland tumors
2026-10-15 15:40:48Maximilian J. Roesel, Niklas Kiermeyer, Philipp Keyl, Frederick KlauschenAbstract
In pathology, pattern recognition relies on a frame of reference built through extensive training and experience, which may still fall short when encountering rare cancer subtypes. Archives of previously diagnosed tumors often contain relevant precedents, yet identifying the most relevant cases remains challenging. Here we explore whether content-based retrieval with current pathology foundation models facilitates rare cancer diagnostics without any task-specific training. Pathologists can query a digitized archive using a region of interest (ROI) from a whole slide image (WSI) and retrieve morphologically similar cases, along with their ICD-O codes and corresponding WSIs. The ROI can be selected manually or automatically using a gated attention-based multiple instance learning model. We evaluate our approach by leave-one-case-out on an imbalanced dataset of 137 salivary gland tumor cases spanning 12 ICD-O morphology categories, comparing four pathology foundation models (Atlas2, Atlas2-B, Atlas2-S, and UNI2h). Top-1/3/5 retrieval was 62.5%/76.7%/81.3% for Atlas2, 50.2/66.6/72.9% for UNI2h, 52.3%/71.5%/77.6% for Atlas2-B and 54.4/65.2/73.7% for Atlas2-S. Macro-averaged F1 ranged from 69.0% for the smallest model Atlas2-S to 76.6% for the largest model Atlas2 across the five classes with at least ten cases. Our modified Chamfer similarity separated correct from incorrect matches among the Top-5 more reliably for Atlas2 (AUC 0.754) than for Atlas2-B (0.679, p = 0.020) and Atlas2-S (0.653, p = 0.003), but not UNI2h (0.697, p = 0.196). Requiring no additional training or manual annotation, this approach makes departmental archives searchable with pathology foundation models, providing a low-barrier route to case-based differential diagnosis and extending the individual frame of reference to the collective experience of the department.
Abstract (translated)
URL
https://arxiv.org/abs/2610.22087
PDF
-
From Signals to Evidence: Interpretable Underwater Acoustic Target Sensing with Large Language Models - Part I: Framework and Recognition
2026-10-15 15:27:49Yuge Zhang, Jie Chen, Zhengqiao Zhao, Xiling Yao, Lingji Xu, Jingdong Chen, Yixin YangAbstract
Underwater acoustic target detection and recognition are essential to marine surveillance and intelligent ocean sensing. Despite substantial advances in deep learning, existing systems typically produce binary detection decisions, categorical labels, or scalar confidence scores, offering limited insight into the physical evidence underlying their outputs. Large language models (LLMs) offer an opportunity to move underwater acoustic sensing toward interpretable, evidence-grounded understanding; however, generic audio encoders struggle to capture the distinctive physical characteristics of underwater acoustics, and physically annotated data are scarce. In this paper, the first of a two-part series, we propose a from-signals-to-evidence framework that integrates physics-aware acoustic encoding with a pretrained LLM, organized around a unified feature-evidence-decision chain with a shared reasoning core. An open input layer accommodates heterogeneous information-raw waveforms, time-frequency representations, environmental and geographic data, and textual physical priors-unifying detection and recognition within a common formulation. In Part I, we instantiate this framework for target recognition through an Underwater Acoustic Physics-aware Transformer (UPT), which tokenizes complementary acoustic representations and learns task-disentangled physical representations through four physics-oriented learning tasks. We develop a physics-parameterized data engine that provides traceable supervision at two levels of granularity and introduce an attribute-level evaluation protocol to assess the physical fidelity of generated descriptions. Experiments on simulated recordings of four vessels demonstrate accurate vessel recognition and reliable estimation of shaft frequency, blade-passing frequency, and machinery-related attributes.
Abstract (translated)
URL
https://arxiv.org/abs/2610.22064
PDF
-
Momentum, Commentary Tone, and Earnings Surprise in LLM Financial Forecasts
Abstract
Which inputs drive a language model's financial forecast? We construct synthetic earnings announcements that independently reverse 20-day momentum, non-informational commentary tone, and whether fixed actual EPS beats a manipulated consensus. Six instruction/chat checkpoint conversions from four families (0.49-7.61B base parameters) receive 2,016 prompts each under a direct-answer Q8_0 protocol. The design includes 24 numerical blocks, six commentary pairs, profits and losses, rising and falling annual EPS, and two surprise sizes. Both A/B answer assignments and U/D cover all 384 primary scenarios per conversion. We estimate matched changes in conditional answer-prefix scores, not calibrated market probabilities. Across these checkpoints, the primary EPS contrast ranges from -0.19 to 86.80 percentage points. The magnitude of the primary signed tone contrast exceeds that of EPS in 5 of six models, and momentum exceeds it in 5. Natural-word numerical diagnostic accuracy ranges from 51.0 to 95.8 percent. Answer-encoding, wording, context, and conversion controls qualify these results. The experiment measures observable input reliance and numerical recognition; it establishes neither an internal reasoning mechanism nor realized return predictability. Public benchmark prompts, recorded scores, and an interactive recorded-results explorer accompany the study.
Abstract (translated)
URL
https://arxiv.org/abs/2610.22043
PDF
-
When Models Misread Ink: Auditing VLMs for Arabic Calligraphy Recognition
2026-10-15 14:58:11Niaz Mohaiman Abtahi, Evan Maksud, Reyana ShahidAbstract
Arabic calligraphy is a high-stakes cultural visual domain for Muslim communities, yet VLM evaluations rarely test its fine-grained script traditions. We audit 12 VLMs on a cleaned 1,195-image six-way benchmark using numeric, visual-cue, and Diwani-first prompts. In this closed-set setting, the best numeric-prompt accuracy is 60.9% for closed/API models and 47.2% for open-weight models. The main failure is default-label collapse: every numeric run is dominated by a single label taking 33-63% of predictions, against an 18.8% majority-class baseline. For 8 of 12 models the default is Thuluth; GPT-4o labels 52.9% of all images as Thuluth although it is only 18.1% of the set. Prompting helps unevenly: visual cues move accuracy by more than 10 points for some models but hurt others, and foregrounding Diwani often relocates the default rather than removing it. Arabic calligraphy recognition is therefore a compact, culturally grounded stress test for VLM robustness and for measuring which visual traditions become legible to automated systems.
Abstract (translated)
URL
https://arxiv.org/abs/2610.22018
PDF
-
Learning Speech Attribute Representations via Multi-Teacher Distillation
2026-10-15 14:15:32Maryem Bouziane, Salima Mdhaffar, Yannick Est\`eveAbstract
Recent post-training approaches have shown that self-supervised speech encoders can be adapted to learn a single utterance-level speech attribute through knowledge distillation. In this work, we investigate whether a single speech encoder can simultaneously preserve multiple complementary utterance-level speech representations. We propose a multi-attribute multi-teacher distillation framework that learns semantic, speaker, language, and emotion representations from four frozen teacher models using a shared SSL backbone and attribute-specific branches. Experiments on multilingual semantic retrieval, speaker verification, language identification, and speech emotion recognition show that the proposed model achieves performance close to dedicated single-task students while requiring only a single shared encoder. Furthermore, the learned layer aggregation strategies reveal that different speech attributes naturally emerge at different depths of the SSL encoder, providing new insights into the organization of speech representations.
Abstract (translated)
URL
https://arxiv.org/abs/2610.21941
PDF
-
Flow-CLIP: Image-Conditioned Residual Scoring for CompositionalImage-Text Matching
2026-10-15 14:12:30Minghong Wei, Enguang Zhu, Pu Cao, Lu Yang, Qing SongAbstract
Vision--language pre-training has enabled substantial progress in image--text retrieval and zero-shot recognition by learning transferable cross-modal representations. However, sentence-level alignment remains insufficient for compositional matching, where captions can describe the same objects and attributes while differing in how these elements are bound. To address this limitation, we propose Flow-CLIP, which refines global image-text alignment through image-conditioned changes in token--patch evidence. Global Compatibility Calibration (GCC) learns a sentence-level similarity metric, while Image-Conditioned Evidence Refinement (ICER) incorporates visual patch context into token queries and uses the resulting change in local matching evidence to adjust the global score. We further introduce Cross-Image Margin Ranking (CIMR), which compares the same positive--negative caption pair across matched and mismatched images, encouraging a larger ranking margin when the image supports the correct description. Experiments on compositional matching and attribute--object binding benchmarks show that Flow-CLIP improves binding accuracy over pretrained CLIP and learned global alignment through local evidence refinement, while cross-image ranking supervision mitigates reliance on language priors. Code will be released at link.
Abstract (translated)
URL
https://arxiv.org/abs/2610.21937
PDF
-
BridgeAfford: Bridging Query-Target Gap for Generalizable 3D Affordance Grounding
2026-10-15 14:03:16Yijie Su, Yakun Zhu, Yi Bin, Pengpeng Zeng, Liang Chen, Guoqing Wang, Jingkuan Song, Heng Tao ShenAbstract
Language-guided 3D affordance grounding aims to localize functional regions on 3D objects according to interaction intents. However, existing methods exhibit poor generalization to unseen objects. One challenge is the query-target gap: a semantic discrepancy between query features encoding interaction intents and target-region features, leading to inaccurate grounding. End-to-end training favors query representations tailored to seen objects, limiting their compatibility with shifted target-region features on unseen objects. To address this issue, we propose BridgeAfford, a framework that reduces the query-target semantic gap from two complementary perspectives. First, Interaction Region Alignment (IRA) promotes cross-object alignment among geometrically similar interaction regions during training. This enables structures on unseen objects that resemble previously observed ones to produce representations similar to those learned during training, facilitating target-region recognition. Second, Visual Evidence Aggregation (VEA) leverages a 2D affordance model to refine the attention distribution and guide the query toward the correct region, yielding more target-aware queries. Extensive experiments on the LASO and PIAD datasets demonstrate that BridgeAfford outperforms existing methods, particularly on unseen objects, highlighting its robust generalization ability.
Abstract (translated)
URL
https://arxiv.org/abs/2610.21918
PDF
-
Anvil: Scaling Embodied Spatial Intelligence through Knowledge Recirculation
2026-10-15 13:47:08Weilin Ruan, Bohan Zhou, Jinghao Li, Yangkai Wei, Wenqian Li, Peiran Dong, Yingjie Xu, Leyi Wu, Haojian Huang, Binghui Xie, Han Yang, Kaiwen Zhou, Yinchuan Li, Ying-Cong Chen, James ChengAbstract
Embodied spatial intelligence requires more than visual recognition: models must localize entities, reason about their relations, and express this knowledge in action-relevant forms. Yet joint training can conflate the knowledge needed at different stages because supervision is fragmented across incompatible geometric and semantic outputs. We introduce Anvil, a unified framework for data generation, training, and benchmarking for embodied spatial intelligence. (1) Anvil-Gen is a heterogeneous data flywheel that combines form-specific generation, cross-stage consistency verification, and failure-driven recollection across 11 Ground--Relation--Act forms. (2) Anvil-Ben provides scene-aligned supervision for learning diverse embodied spatial capabilities and scene-linked assessments that distinguish stage competence from error propagation. (3) Anvil-MOPD is a 9B student that consolidates heterogeneous spatial knowledge through distillation from specialist teachers. Semantic-first-fault routing targets teacher supervision to the student's earliest decisive failure, while stage normalization controls the loss scale across native outputs of different lengths. We construct 5.8M source-grounded records spanning 160K images, reserving 580K spatial records for held-out evaluation and using the remaining 5.22M records for training. On average across seven public benchmarks, Anvil-MOPD outperforms Qwen3.5-9B and RynnBrain-8B by 9.29 and 11.03 percentage points, respectively, while reaching 80.67% on VABench-Point. Controlled studies support targeted coverage, selective transfer, and feedback-guided repair as complementary framework components.
Abstract (translated)
URL
https://arxiv.org/abs/2610.21882
PDF
-
CF-NODE: Counterfactual Spatial Node Intervention for Test-Time Adaptation Under Domain Shift, Evaluated Across Five Diabetic Retinopathy Datasets
2026-10-15 12:44:10Sabarish G, Deepak ParasharAbstract
Automated fundus-based diabetic retinopathy (DR) grading can substantially expand screening capacity, but source-trained models degrade sharply across different cameras, clinics, and patient populations. Existing test-time adaptation methods depend on entropy minimization or target information accumulated from image batches and streams. These mechanisms can reinforce confidently incorrect predictions and break down when clinical images arrive independently. We introduce CF-NODE, a novel counterfactual test-time adaptation framework for cross- domain DR grading. Instead of trusting prediction confidence or estimating a target distribution, CF-NODE asks how each prediction responds under a controlled intervention on the evidence supporting it. CF-NODE ranks internal spatial nodes using model-derived attention, feature magnitude, and node-set similarity, progressively attenuates the highest-ranked nodes, and uses the stability of the resulting expected-grade response to apply a bounded calibration of the predicted grade distribution. The framework is fully frozen and adapts every image independently using only geometric views and a fixed source-derived prior, requiring no target-derived information or model updates. We evaluate CF-NODE on five external DR datasets under repeated-seed evaluation against TENT, EATA, SAR, and T3A. CF-NODE outperforms all four methods with statistically significant improvements over the second-best method, improves AUC on every target domain, and attains the highest average AUC and macro-F1, delivering stronger grade discrimination together with more balanced recognition across DR severity levels. These results demonstrate that counterfactual intervention sensitivity provides consistent and deployment-ready domain robustness for single-image clinical inference.
Abstract (translated)
URL
https://arxiv.org/abs/2610.22437
PDF
-
Fine-grained Human Motion Understanding with Language Models
2026-10-15 10:10:55Thomas Markhorst, Zhi-Yi Lin, Jouh Yeong Chew, Jan van Gemert, Xucong ZhangAbstract
Fine-grained motion understanding requires knowing not only which poses occur, but also when they occur. Existing motion-language models either compress motion clips or concatenate pose features without retaining the timing of each pose, making action duration and temporal scale ambiguous under temporal subsampling. We introduce FiGMo, an LLM-based model that represents motion as timestamp-grounded skeletal poses, supporting reasoning over action order and duration. A unified pose encoder applies this representation to both 2D and 3D skeletons, including poses detected from real videos, while RGB can optionally provide contextual information. To reason over timestamped pose sequences, the model must understand both individual body configurations and how they change over time. As existing data provides limited pose-level grounding, we construct the required pose-to-motion supervision. Controlled experiments show that timestamps improve temporal localization and preserve physical timing under subsampling, while the constructed supervision drives the main performance gains. FiGMo achieves state-of-the-art results across motion question answering, dense motion captioning, and action recognition with detected poses. Our code is available at: this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2606.20888
PDF
-
SITAR: A Situational Distortion Metric for Automatic Speech Recognition Evaluation
2026-10-15 07:48:03Mingzhe Lu, Jinhao Liu, Yunpeng Li, Hang Su, Lichun Fan, Jian LuanAbstract
In automatic speech recognition (ASR), transcription errors with similar lexical or semantic discrepancies can have markedly different consequences for the situation conveyed by speech. Capturing these differences requires not merely contextual assessment, but an understanding of what the spoken discourse describes, establishes, or seeks to bring about. However, relying on LLM judges introduces computational cost and scoring variability. We introduce SITAR, a situational distortion metric for ASR, realized through a comprehensive end-to-end construction framework. Specifically, this framework aggregates situational severity and cumulative extent into a unified score via a bounded recursive formulation. Using targets derived from this formulation with limited offline LLM support, we train SITAR as a lightweight BERT-family regressor that scores reference-hypothesis pairs in a single forward pass, entirely free of any prohibitive runtime costs. Recognizing that introducing a new metric demands rigorous justification, we formalize a principled validation protocol spanning both extrinsic validity, assessed at macro and micro levels, and intrinsic scorer reliability. The former, evaluated across diverse datasets and models, demonstrates that SITAR reliably tracks broad capability scaling while providing orthogonal diagnostic utility, whereas the latter, examined on a purpose-built controlled benchmark, confirms the operational soundness of the scoring instrument itself. Through systematic validation, we demonstrate that SITAR holds strong potential as a practical, lightweight complement to existing ASR metrics. We will release the full recipes (code, model weights, dataset, etc.) after the internal review to support community research.
Abstract (translated)
URL
https://arxiv.org/abs/2610.21440
PDF
-
Where Is the 'Self' of a Large Language Model? -- From Self-Index to Self-Reference
2026-10-15 06:38:57Yuchen Li, Xintong Li, Zongqi Fan, Ken-tye Yong, Nguyen H. TranAbstract
Does a large language model have a self, and if so, of what kind? We separate two functional levels familiar from biology: a self-index, which distinguishes one's own material from external material and shapes processing, as corollary discharge does in nearly all animals, and self-reference, the higher ability to recognize what is one's own, as in mirror self-recognition. We probe both in Qwen3.8-27B by placing the same text in an Assistant, a User, or a System message rendered by the model's own chat template, and report three levels of evidence. Persistence: as predicted in advance, the representation of the text separates the model's own message from both external sources at all 64 layers and for every test text, along a shared direction, even when the answer does not depend on who supplied the text. Coupling: asked who wrote the preceding message, the model claims its own messages (0.77) and assigns User and System messages alike to the user, and how strongly it claims each message is predicted by the size of its self-index (Spearman correlation up to 0.68). Intervention: exchanging the internal states that follow a message, rather than those of its text, reverses the self-attribution between layers 15 and 31; removing the self-index direction from them at layer 39 lowers the claim more than a random direction; and in arithmetic the model adopts a wrong step more often when it wrote the step itself (0.92 versus 0.75), an effect carried by the step's representation at layers 31 to 39. Reasoning first in thinking mode raises the probability of the correct answer (0.96 instead of 0.77), but recognition is present without it. We propose that self-reference is a higher-level ability built on a self-index, a content-independent displacement superimposed on the whole representation of the model's own messages.
Abstract (translated)
URL
https://arxiv.org/abs/2610.16717
PDF
-
Making every stage of medical device document assessment with large language models deterministic
2026-10-15 05:45:20Ryosuke SuzukiAbstract
To show that a medical device complies with regulation, experts judge whether documents and records meet each requirement and record the judgment for others to verify. For each assessment item that experts define, large language models could prepare a preliminary assessment: a label such as Satisfied or Not satisfied, a reason and quoted evidence. We sent fourteen commercial deployments of such models (seven models from three providers, through six routes) the same request repeatedly, with the deterministic sampling settings each accepted; every deployment returned differing assessments for at least one request, and in twelve the label changed. Reducing reasoning where the interface allows it did not reduce the variation, and prompts of retrieved passages removed it only for the smallest models. The same open-weight model was reproducible or not depending only on how it was served. A pipeline we built to be deterministic at every stage -- text extraction and optical character recognition (OCR), chunking, embedding, indexing, retrieval and inference -- returned byte-identical output at each stage across reruns, interruptions, edits, machines of one graphics processing unit (GPU) model and days, for blood glucose meter and infusion pump documents. OCR, embedding and approximate retrieval varied when their own serving was not made invariant, and outputs changed with the GPU model and with the inference engine's scheduling settings, which therefore belong in the recorded configuration. End-to-end determinism is achievable and gives a testable basis for audit and change control.
Abstract (translated)
URL
https://arxiv.org/abs/2610.21300
PDF
