Activity
-
A Node Embedding Framework for Model-Centric Conformance Checking in Process Mining
2026-10-13 04:34:39Mostafa Abbasi, Homayoun NajjaranAbstract
Evaluating the quality of process models discovered from event logs is a fundamental challenge in process mining. Traditional evaluation relies on conformance metrics such as fitness and precision, which are highly sensitive to noise and deviations in event logs and therefore often fail to give a consistent assessment of discovery algorithms. We introduce a model-centric evaluation framework that combines node embeddings with Procrustes alignment to enable cross-graph comparability, since embeddings derived from different graphs reside in distinct vector spaces. The proposed PetriNetGraph method transforms Petri net process models into graph representations compatible with embedding algorithms. Comparing the node embeddings of ground-truth and discovered models through a parameterized cosine distance yields a unified and bounded metric for evaluating process discovery across datasets and methods. We evaluate the framework under static and context-aware scenarios to examine how prior knowledge of activity existence influences the assessment of structural similarity. The framework also integrates a compliance checking mechanism that localizes deviations at the activity level and an entropy-regularized node distance aggregation function that quantifies global deviations across the model. Experiments on nine publicly available event logs show that the approach provides a reliable embedding-based conformance indicator that reflects the behavior of fitness and precision. Experiments on synthetic logs with controlled noise levels confirm that it is substantially more robust to log deviations than traditional conformance metrics, PetriNet2Vec, and its dynamic-window and Procrustes-aligned variants, while remaining sensitive to process model degradation. All implementation details and datasets are publicly available.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17702
PDF
-
HelpMeOut: A Neuropsychology-Grounded Benchmark for Error-Aware Monitoring of Household Task Execution
2026-10-13 04:20:13Haoran Wang, Jingwei Huang, Anqing Duan, Ho Seok Ahn, Bruce MacdonaldAbstract
We introduce HelpMeOut, an embodied AI benchmark for error-aware monitoring of household task execution. Existing embodied household benchmarks typically assume that humans execute tasks correctly, but this assumption does not hold for cognitively impaired persons or older adults whose everyday activities produce structured execution errors. To address this gap, HelpMeOut operationalizes research on everyday action errors into 20000 paired activity episodes, covering nine neuropsychologically motivated error categories. In each episode, agents observe a streaming human activity trajectory and must monitor whether the execution is unfolding correctly, diagnosing any structured errors as early as possible. We evaluate learned-monitor, symbolic, change-point, and frontier LLM prompting baselines. Results show that these methods perform poorly as error monitors, often missing when execution goes wrong. These results suggest a gap left by standard goal-inference and task-completion benchmarks, making HelpMeOut a testbed for error-aware assistive agents.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17694
PDF
-
Hardware-Assisted Multi-Stage Ransomware Detection with Adaptive Machine Learning
2026-10-11 22:17:25Salma Aboelmagd, Md. Rakibul Ahasan, Zhixin Pan, Abdulrahman Takiddin, Ziyu ShuAbstract
Ransomware is a fast-escalating privacy threat causing significant financial and data losses worldwide. As attacks grow more sophisticated, traditional static signatures and dynamic behavior analysis struggle to defend. In this paper, we propose an novel framework that couples software-level scanning with hardware-level microprocessor activity monitoring to achieve efficient ransomware detection and mitigation. The contributions are threefold: (1) we employ adversarial training to harden static analysis against obfuscation and leverage hardware-assisted monitoring for real-time detection; (2) we integrate neural architecture search (NAS) to automatically tailor models for stronger generalizability; and (3) We validate the proposed approach through extensive experiments, demonstrating substantially improved detection accuracy and reduced latency compared with existing methods. Moreover, it provides trace-level interpretability that explains the underlying decision process.
Abstract (translated)
URL
https://arxiv.org/abs/2610.15951
PDF
-
Vision: 'What Don't I Know About User's Preferences?' How Uncertainty Estimation Improves Proactive Assistant Alignment
2026-10-11 22:13:37Zhiyu An, Ziyi Xuan, Yu Yang, Wan DuAbstract
Proactive Smart Assistants (PSAs) increasingly use general knowledge, learned patterns from group-level data, patterns from individual user's data, and recent sensor observations to anticipate which action a user may prefer. Existing systems typically select actions from predicted user preference likelihoods, but these likelihoods alone do not reveal how reliable the prediction is, which evidence supports it, or why ambiguity remains. In this vision paper, we argue that uncertainty estimation can provide this missing information and substantially change appropriate downstream behavior. Using the example of a home assistant, we introduce the questions that uncertainty estimation aims to answer, i.e. how sure is the assistant about the predicted preference likelihood, whether the confidence come from group-level assumptions or individual's data, and whether likelihood ambiguity reflect the user's intrinsic preference variability or insufficient data for the particular situation. We further introduce uncertainty estimation applications in neighboring domains, such as human trajectory prediction, personalized activity recognition, information-collection, and model-based control. Finally, we draw on insights from neighboring domains to envision how future uncertainty-aware proactive assistants may address key challenges in proactive interaction.
Abstract (translated)
URL
https://arxiv.org/abs/2610.15948
PDF
-
Learning Multimodal Sensor Fusion with JEPA Representations for Breathing Activity and Pose Recognition
2026-10-11 20:13:12Yunxiang Luo, Constantino \'Alvarez Casado, Xiaoting Wu, Nhi Nguyen, Le Nguyen, Manuel Lage Ca\~nellas, Miguel Bordallo L\'opezAbstract
Predictive video representations do not by themselves specify how a multimodal system should handle missing sensors. We transfer frozen V-JEPA2 features to a set of heterogeneous sensors comprised by RGB video, depth video, and radar time series. A shared fusion model uses modality queries, fixed sensor slots, and explicit availability to classify breathing activity and pose from all seven non-empty sensor subsets. The contribution is a controlled evaluation of subset supervision, pooling capacity, and representation transfer on 39 OMuSense-23 subjects with five subject-group folds and three seeds. With matched downstream parameter counts, the final model reaches recording-level activity macro-F1 of $75.23%\pm0.60$ averaged across seven subsets. Supervising all subsets in each batch improves the matched query-pooling control by 1.923 percentage points (conditional 95% subject-bootstrap interval [0.647, 3.209]), while query pooling shows no clear advantage over matched mean pooling under balanced supervision. Radar probes support transfer beyond pooled wavelet pixels, 64-token inference approximately preserves mean activity performance, and the tested Consensus-JEPA loss adds no measured benefit against its exact control.
Abstract (translated)
URL
https://arxiv.org/abs/2610.15891
PDF
-
Siam-ALS: An Explainable Siamese Deep Learning Framework for Early ALS Detection Using EEG and Eye-Tracking Signals
2026-10-11 05:14:29Abdur Rahman, A. A. M. Muzahid, Istiak Ahmed, Dipto Biswas, Hua Han, Yujin Zhang, Ferdous SohelAbstract
Amyotrophic lateral sclerosis (ALS) diagnosis remains challenging due to heterogeneous symptoms and reliance on clinical assessment, underscoring the need for objective, data-driven approaches that leverage patient-generated physiological signals. We propose Siam-ALS, an explainable Siamese deep learning framework that jointly models synchronized electroencephalography (EEG) and eye-tracking (ET) signals. EEG captures ALS-related changes in brain activity, while ET provides information about oculomotor control, allowing the model to learn complementary patterns associated with ALS. The framework employs two modality-specific encoders: a convolutional EEG encoder that captures spatiotemporal neural patterns and an MLP-based ET encoder that models oculomotor dynamics. These modality-specific features are processed through weight-sharing Siamese branches and fused into a compact 64-dimensional joint embedding. A lightweight classifier is used to predict ALS, while pairwise metric learning helps the model learn more effective patterns from the small, diverse ALS datasets. Performance was evaluated on the public EEGET-ALS dataset using strict subject-level cross-validation, where Siam-ALS achieved an accuracy of 0.988 and an AUROC of 0.998, demonstrating strong performance compared with classical machine learning, DL, and non-Siamese fusion approaches. Cross-dataset evaluation on the independent BigP3BCI dataset achieved an AUROC of approximately 0.98, supporting the generalization capability of Siam-ALS across different BCI paradigms. To enhance interpretability, six XAI methods were evaluated in the learned embedding space, with SHAP achieving the highest, though statistically comparable, fidelity in identifying important EEG and ET patterns that contribute to ALS predictions. Overall, Siam-ALS provides an accurate, generalizable, and explainable framework for multimodal ALS detection.
Abstract (translated)
URL
https://arxiv.org/abs/2610.15351
PDF
-
StimFlow: Synthesizing Brain Activity from Images via Stimulus-conditioned Flow Matching
2026-10-11 04:45:03Bui Van Hung, Dang Thanh Vu, Hoang-Son Vo, Mai Duc Minh Huy, Mai Tien Dung, Soo-Hyung KimAbstract
The inherent variability of fMRI responses across trials encourages a generative view of the stimulus-to-response mapping. Prior generative encoders often operate in compressed latent spaces and condition predominantly on semantic features, which provide limited low-level cues for early visual cortex. We propose StimFlow, which synthesizes fMRI directly in native voxel space using conditional flow matching with a linear path. This design aligns training and inference distributions, enabling competitive performance with a single sample and reducing reliance on averaging. To match the visual hierarchy, StimFlow combines structural DINOv2 features with semantic CLIP features, which prove complementary along the cortical processing stream. With a single sample, StimFlow achieves a per-voxel Pearson correlation of 0.371, matching baselines that average five draws. Generation in voxel space allows direct ROI analysis and concept localization without a decoder. Furthermore, it recovers over half of full-data performance from just one hour of scans, supporting efficient cross-subject adaptation.
Abstract (translated)
URL
https://arxiv.org/abs/2610.15341
PDF
-
CHEAR: Confusion-Hierarchy Evidence-Aware Reasoning for Cost-Efficient Human Activity Recognition
2026-10-11 03:06:40Xiangqing Lu, Tao Zhu, Mingxing Nie, Liming ChenAbstract
Large language models (LLMs) offer a new approach to wearable-sensor-based human activity recognition. Retrieval-augmented generation (RAG) further introduces similar motion examples and discriminative sensor knowledge, enabling decisions grounded in observations relevant to the current query. More evidence, however, does not necessarily mean better-organized evidence. When online prompts and LLM agents primarily orchestrate inference, decisions about which activities to compare and when to invoke discriminative knowledge remain concentrated at inference time. As the activity vocabulary grows, the model must repeatedly process more candidates and associated knowledge, whereas pruning candidates too early risks excluding the true activity. We propose confusion-hierarchy evidence-aware reasoning (CHEAR), which moves part of this knowledge organization offline. CHEAR constructs and validates an activity hierarchy from recurrent cross-subject retrieval errors and builds corresponding activity-pair knowledge. At inference time, the hierarchy orders candidate comparisons; the LLM retains candidates progressively, and local discriminative knowledge is accessed only for the remaining activities. Across eight datasets, CHEAR achieves 86.79\% mean cross-subject accuracy, a 75.91\% candidate reduction rate, and 96.44\% true-activity retention. Per-sample token use is 85.15\% lower on average than in a variant without the hierarchical structure. Mean accuracy across eight cross-dataset transfer directions is 88.81\%. These results show that organizing candidates and discriminative knowledge around observed activity confusions reduces online LLM input while maintaining recognition performance.
Abstract (translated)
URL
https://arxiv.org/abs/2610.15298
PDF
-
OntoAR-HAR: Ontology-Guided Agentic RAG with Personalized Memory for Human Activity Recognition
2026-10-11 02:57:12Xiangqing Lu, Tao Zhu, Liang Yu, Shuangjian Li, Lulu Ban, Furong Duan, Mingxing Nie, Liming Chen, Yaping WanAbstract
LLMs increasingly support sensor-based human activity recognition (HAR) by linking motion signals with activity semantics and physical knowledge. Retrieval-augmented generation (RAG) grounds decisions in sensor examples and domain features, enabling training-free target-user inference with less dependence on raw numerical arrays. However, existing LLM/RAG-based HAR typically searches single-level activity spaces using static population knowledge. Consequently, they compare many unrelated candidates using the same static population evidence and overlook personalized motion patterns. We propose OntoAR-HAR, an ontology-guided agentic RAG framework combining static population and personalized knowledge. Its tree-structured activity ontology decomposes classification into local decisions among semantically related activities. The static store supplies node-constrained examples, while offline-discovered executable signal features provide numerical evidence for local distinctions. During inference, the personalized store collects selected user observations, which the LLM integrates with static knowledge and signal features while traversing the ontology to an activity label. Cross-subject and heterogeneous-sensing experiments show complementary gains from hierarchical retrieval and personalized memory across multiple LLM backbones, achieving state-of-the-art performance. By explicitly modeling hierarchical relations across activity granularities, OntoAR-HAR also addresses the long-overlooked semantic inclusion problem and avoids treating related labels as independent categories.
Abstract (translated)
URL
https://arxiv.org/abs/2610.15292
PDF
-
Window Size Matters: Benchmarking Voice Activity Detectors in Noise
2026-10-10 05:51:27Alok Kumar, Mohit Srivastava, Anupam PurwarAbstract
Voice activity detection (VAD) sits in front of almost every streaming speech recognizer, yet the choice of detector is often made by feel rather than by measurement. We report a frame-level offline benchmark of three detectors on the test-clean split of LibriVAD-small: the neural TEN VAD, the neural Silero VAD and a simple RMS energy baseline. The evaluation covers 2,214 noisy utterances (about 7.1 hours) built from LibriSpeech, nine noise families and six SNR levels from -5 to 20 dB. At a 16 ms hop, TEN VAD reaches a ROC AUC of 0.917, an average precision of 0.968 and an F1 of 0.913. A sweep over 20, 30, 40 and 50 ms windows shows TEN to be almost insensitive to frame length (AUC 0.916 to 0.919), while RMS stays near 0.75 to 0.77. Silero matches TEN only near its native 32 ms input size and drops to 0.63 at 20 ms. We hypothesize that this is likely caused by our fixed-frame protocol rather than by the model itself. Zero padding, sub-chunk averaging and unreset model state are the likely factors, but we have not ablated them. In a per-condition breakdown of TEN, babble noise at low SNR stays close to chance. We also show that F1 at a fixed 0.5 threshold is a weak metric on this data, because a trivial all-speech detector already scores 0.854. All settings needed to reproduce the results are given in the paper.
Abstract (translated)
URL
https://arxiv.org/abs/2610.14607
PDF
-
Revisiting the Task of Building Damage Assessment by Satellite Imaging
2026-10-09 17:07:03Anthony Cioppa, Virgile Blua, S\'ebastien Pi\'erard, Pietro Ceccato, Marc Van DroogenbroeckAbstract
Natural disasters such as earthquakes, tropical storms, and floods cause extensive damage to buildings, and rapid, reliable damage maps are essential to guide emergency response. Automating this task, referred to as building damage assessment (BDA), from satellite imagery, has become an active research topic at the intersection of remote sensing, computer vision, and machine learning, particularly since supervised deep learning methods were shown to outperform earlier handcrafted approaches. Yet, despite a decade of intense activity and the release of several large-scale datasets, the field lacks a shared foundation. This chapter revisits the task of BDA from first principles, organized around three pillars: data, evaluation, and methods. We first clarify the candidate task for- mulations and argue for a sensor-agnostic, instance-based definition that tolerates missing pre-event imagery. We then review existing datasets along the axes of sensing modality, temporal configuration, and geographic coverage. On the evaluation side, we show that prevailing practices, largely based on a single aggregate score, are theoretically fragile, and we lay the foundations of a more informative, ranking-based, multi-score protocol. We further review methods ranging from task-specific deep networks to recent Earth observation foundation models, noticing that generalization to unseen events and regions remains an open difficulty. Finally, we discuss how these elements can inform the organization of a BDA challenge, including cross-validation, challenge, and private data-sharing setups designed to build trust between developers and users.
Abstract (translated)
URL
https://arxiv.org/abs/2610.13857
PDF
-
Neurons++: A Unified Visual-Stream Framework for Image and Video Reconstruction from fMRI with Neural-to-Visual Interpretability
2026-10-09 15:49:07Haonan Wang, Yuchen Wang, Jingyu Lu, Qixiang Zhang, Lehan Wang, Xuanqi Huang, Xiaomeng LiAbstract
Decoding visual experience from fMRI is important for understanding human perception and developing brain-computer interfaces. Recent generative decoding methods can reconstruct plausible images and videos, but visual plausibility alone does not ensure that the output is faithfully grounded in brain activity. In this work, we propose Neurons++, a visual-stream fMRI reconstruction framework for both static image and dynamic video reconstruction. Compared with our conference version Neurons, Neurons++ extends the paradigm from video-only reconstruction to visual-stream reconstruction for both images and videos, and replaces parallel task conditioning with serial cortex-inspired decoding. The model first uses image-fMRI pretraining to obtain a stable spatial-semantic initialization, and then performs subject-specific fine-tuning for image and video reconstruction separately. The serial design organizes fMRI-derived representations into explicit spatial, semantic, language-level, and temporal cues. Extensive experiments show that Neurons++ achieves state-of-the-art video reconstruction and competitive image reconstruction across standard benchmarks. Beyond reconstruction fidelity, neural-to-visual analyses indicate that Neurons++ exhibits qualitative directional controllability under neural perturbation, that its fMRI encoder read-out is sensitive to stimulus-relevant cortical features, and that it supports ROI-guided semantic modulation. Code and model weights are available at: this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2610.13763
PDF
-
QC-MI-OVSEL: Benchmarking Query-Conditioned Multi-Instance Open-Vocabulary Sound Event Localization
2026-10-09 08:19:02Yongqiang Chen, Peiwei Chang, Yuelan Cheng, Xiatian Zhu, Wenwu WangAbstract
We study query-conditioned multi-instance open-vocabulary sound event localization (QC-MI-OVSEL), where a system must recover all active physical sources matching a natural-language category query, including when the queried category is absent. We introduce the QC-MI-OVSEL benchmark, a 40,000-scene first-order Ambisonics benchmark with 80,000 balanced present/absent query views over 83 classes, including 17 novel classes excluded from training and development. Each frame is annotated with the activity and 3-D direction of every query-matching source, with up to three simultaneous targets. We further propose Q-RSet, which uses query-conditioned source slots and responsibility-based evidence aggregation for source-set prediction. Experiments show improved source recovery, localization recall, cardinality estimation, exact-set recovery, and absent-query suppression over complementary baselines. Analysis indicates that novel categories, paraphrased queries, and higher source multiplicity mainly impair source discovery and set completion, while directional accuracy remains comparatively stable for successfully recovered sources.
Abstract (translated)
URL
https://arxiv.org/abs/2610.13198
PDF
-
From Benchmark to Bench: Can Agents Survive Real-World Drug Discovery?
2026-10-09 08:05:52Pierre Llompart, Levent Guner, Helen Lai, Alessandro Tibo, Yijie XuAbstract
Agentic systems increasingly coordinate molecular-design tools, but it is unclear which layer of the stack limits outcomes on real projects. We developed MAGI, an open modular agent that authors objectives, launches and monitors optimization, interprets structure--activity relationships, and revises its strategy accordingly. MAGI generates molecules either directly through the LLM or by delegating to REINVENT 4, with scoring services interchangeable behind a common contract. We tested it across nine retrospective lead-optimization campaigns from three pharmaceutical companies, replayed under fixed temporal cutoffs. Both routes produced valid structures: LLM proposals stayed closer to local chemistry and reached comparable or higher primary activity in fewer operations, whereas REINVENT explored broader chemical space. Whether a campaign met its objective depended on the predictive models, not on the generation route: attainment followed model accuracy on the chemistry proposed, dropping once that chemistry moved outside the model's applicability domain. Separately, a blinded evaluation asked whether the MAGI's output could pass as expert work: chemists were not able to discriminate agentic proposals from held-out compounds, and judged the SAR reasoning broadly plausible yet incomplete. Together, these results position MAGI as a coordination layer pluggable into existing computational chemistry workflows. The ceiling on real projects, however, remains currently set by scorer applicability rather than by tool orchestration.
Abstract (translated)
URL
https://arxiv.org/abs/2610.06411
PDF
-
From 2.1M Hours of Raw Broadcast Audio to Finnish Speech SSL: Confidence Filtering and Architectural Analysis at Scale
2026-10-09 06:57:05Yaroslav Getman, Tam\'as Gr\'osz, Tommi Lehtonen, Mikko KurimoAbstract
Broadcast archives offer scale and diversity for self-supervised learning (SSL) but contain substantial noisy, non-speech, and multilingual content. Prior work typically relies on voice activity detection or language identification (LID) to extract speech, ensuring presence rather than quality. Whether and how to filter unlabeled speech by quality for SSL, particularly from noisy broadcasts, remains underexplored. We present a large-scale Finnish SSL study starting from 2.1M hours of unconstrained broadcast audio. We propose automatic speech recognition (ASR) confidence as a proxy for pretraining data quality and show 3-7% relative WER improvements over LID-based filtering despite a smaller training set. We further address the lack of controlled architectural comparisons in speech SSL by evaluating Transformer, Macaron-Net, Conformer, and E-Branchformer at matched parameter budgets. Our models improve upon prior Finnish SSL baselines, particularly at Base scale and out-of-domain.
Abstract (translated)
URL
https://arxiv.org/abs/2610.13102
PDF
-
AV-FlowEdit: Training-Free Prompt-Driven Audio-Visual Editing with Event-Aware Spectral Trajectory Preservation
2026-10-09 06:31:08Shaohua Pan, Xiao LiuAbstract
Joint audio-video diffusion models generate synchronized content, but editing existing clips requires target changes without disrupting non-target structure or cross-modal correspondence. We present AV-FlowEdit, a training-free and inversion-free framework for prompt-driven audio-video editing. Our key insight is that source--target flow differences provide both cross-modal editing directions and preservation cues. Under shared perturbations, they jointly update the video and audio branches, supporting visual-only, audio-only, and joint edits while retaining pretrained cross-modal interactions. To suppress drift accumulated during sampling, Cross-Modal Event-Aware Spectral Trajectory Preservation combines flow saliency, aligned audio activity, multiscale low-frequency displacement, and diffusion-stage scheduling. This adaptively preserves unsupported regions while allowing intended audio-visual events. AV-FlowEdit requires no model training, external masks, or optimization-based inversion. Experiments show strong prompt alignment and temporal consistency, together with competitive audio quality and cross-modal coherence.
Abstract (translated)
URL
https://arxiv.org/abs/2610.13072
PDF
-
What Egocentric Video Embeddings Encode and How to Combine Them with Inertial Sensors for Human Activity Recognition
2026-10-09 03:15:39Kansuke HabanoAbstract
Egocentric video and body-worn inertial sensors are increasingly recorded together, and the video is usually encoded by pretrained models. What these embeddings represent when the camera sits on the head, and how they should be combined with inertial sensing, has received little attention. Across VideoMAEv2, CLIP, DINOv2 and the egocentric VC-1, the leading principal directions of the embedding are dominated by where the camera looks. Measured head orientation explains them on Ego-Exo4D for egocentric (R^2 0.49-0.70) but not exocentric cameras, and the scene layout explains them on WEAR. The part that the limbs' inertial signals predict separates activities across people. In a set-level probe, the rest separates exercise sets within a participant but not activities across people. This is consistent with separate roles: the video finds the clips of the same set, and the inertial classifier labels them. On WEAR, with temporal order removed and one random limb per one-second clip, we propagate inertial predictions over a within-participant video graph whose leading components are removed. This raises subject-wise cross-validated macro-F1 from 0.648 to 0.852. In our comparisons, propagation helps only over the video graph, and fusing the modalities in one classifier, or adding inertial edges to the graph, performs worse.
Abstract (translated)
URL
https://arxiv.org/abs/2610.12926
PDF
-
Teaching neural decoders with language
2026-10-09 03:12:42Xuan Wen, Yonghao Song, Yuhong Zhang, Guoliang Li, Tzyy-Ping Jung, Xiaorong GaoAbstract
Brain-computer interfaces decode neural activity, yet class-label supervision (CLS) leaves relationships among classes implicit. We propose Language as Supervision (LaS), which aligns neural features with natural-language class descriptions through learned projections and predicts classes using candidate text prototypes. We evaluate seven encoder architectures with independently trained models for visual, motor, and speech decoding from scalp and intracranial recordings. LaS improves speech-decoding accuracy by up to 17.15~pp over matched CLS baselines. Its representations support candidate-set reuse and label-supervised subject and session adaptation, while LaS enables conditional unseen-frequency decoding in steady-state visual evoked potential (SSVEP) responses. Correct class-description correspondence supports this generalization, and dual-sided attribution visualization shows neural and textual contributions to the same class-matching score. These findings suggest that text descriptions can serve as general-purpose references for learning and interpretation in neural decoding.
Abstract (translated)
URL
https://arxiv.org/abs/2610.12921
PDF
-
Neural networks are more modular than single neurons suggest
2026-10-09 00:58:07Shreya Saha, Benjamin Bergen, Meenakshi KhoslaAbstract
Neural networks routinely develop representations in which individual neurons contribute to multiple distinct functions, leading to the widespread view that such networks lack modular organization. We challenge this conclusion by introducing a formally nested hierarchy of modularity: unit-level, which requires distinct computations to occupy disjoint neurons; orthogonal, which asks whether a rotation reveals functionally independent subspaces; and linear, which asks whether a bounded linear map can separate task-relevant computations into independent subspaces. We develop a unified framework for learning decompositions at each level and evaluating them causally through selective ablation. Across vision models, language models and multi-task Recurrent Neural Networks (RNNs), we find that networks which lack unit-level modularity nonetheless often exhibit clear functional modularity at the subspace level. In vision and language models, content versus style and syntax versus semantics respectively dissociate under subspace ablation despite sharing overlapping populations of neurons. These learned subspaces also align differentially with brain data, with content and style subspaces predicting higher and lower visual cortex respectively, and semantic subspaces outperforming syntactic ones in predicting language cortex activity. In RNNs trained on multiple cognitive tasks and previously shown to lack modular structure, our decomposition recovers sharp double dissociations between tasks. The absence of unit-level modularity frequently coexists with clear subspace-level modularity across domains, enabling targeted intervention on specific computations even in networks where no individual neuron is specialized, with broad implications for mechanistic interpretability and for understanding functional organization in both biological and artificial neural systems.
Abstract (translated)
URL
https://arxiv.org/abs/2610.12842
PDF
-
EgoVoice: Proactive Spoken Assistance from Egocentric Multimodal Streams
2026-10-08 16:24:12Heeseung KimAbstract
Wearable augmented reality (AR) assistants are moving toward continuous real-world interaction, where they perceive the user's activity through first-person video and audio and provide timely spoken guidance without being explicitly asked. While proactive video assistants, spoken dialog systems, and egocentric task understanding have each advanced rapidly, existing systems do not address the joint problem of deciding when to speak and what to say from continuous first-person streams. We introduce EgoVoice, a framework for training and evaluating proactive egocentric spoken assistants. From HoloAssist video recordings of real human instructors, we construct clean audio streams through source separation and speech resynthesis, and convert each video session into a format where the model must decide at each moment whether to remain silent or provide spoken guidance. We fine-tune an omni-modal LLM with our data, and further improve its proactive intervention behavior with direct preference optimization. Experiments across closed and open-source models show that existing systems rarely produce well-timed, meaningful proactive interventions, while EgoVoice yields clear improvements in intervention timing, content relevance, and human preference over the zero-shot backbone.
Abstract (translated)
URL
https://arxiv.org/abs/2610.12248
PDF
