Optical_Flow
-
HiSync: Spatio-Temporally Aligning Hand Motion from Wearable IMU and On-Robot Camera for Command Source Identification in Long-Range HRI
2026-03-12 11:10:54Chengwen Zhang, Chun Yu, Borong Zhuang, Haopeng Jin, Qingyang Wan, Zhuojun Li, Zhe He, Zhoutong Ye, Yu Mei, Chang Liu, Weinan Shi, Yuanchun ShiAbstract
Long-range Human-Robot Interaction (HRI) remains underexplored. Within it, Command Source Identification (CSI) - determining who issued a command - is especially challenging due to multi-user and distance-induced sensor ambiguity. We introduce HiSync, an optical-inertial fusion framework that treats hand motion as binding cues by aligning robot-mounted camera optical flow with hand-worn IMU signals. We first elicit a user-defined (N=12) gesture set and collect a multimodal command gesture dataset (N=38) in long-range multi-user HRI scenarios. Next, HiSync extracts frequency-domain hand motion features from both camera and IMU data, and a learned CSINet denoises IMU readings, temporally aligns modalities, and performs distance-aware multi-window fusion to compute cross-modal similarity of subtle, natural gestures, enabling robust CSI. In three-person scenes up to 34m, HiSync achieves 92.32% CSI accuracy, outperforming the prior SOTA by 48.44%. HiSync is also validated on real-robot deployment. By making CSI reliable and natural, HiSync provides a practical primitive and design guidance for public-space HRI.
Abstract (translated)
长距离人机交互（HRI）领域尚待探索。其中，命令来源识别（CSI）——即确定哪位用户发出的指令——由于多用户环境和因距离导致的传感器模糊性而尤为具有挑战性。我们提出了一种光学-惯性融合框架HiSync，该框架通过将机器人安装摄像头捕捉到的手部运动光流与佩戴在手上的IMU信号对齐来处理手势绑定线索。 首先，我们从一组12名用户中定义并征集了一系列自定义手势，并收集了长距离多用户HRI场景下的跨模态命令手势数据集（N=38）。接下来，HiSync框架会分别从摄像头和IMU数据中提取频域下手部运动特征；通过CSINet算法对IMU读数进行去噪处理并完成不同传感器的时间同步工作，并采用具有距离感知能力的多窗口融合技术计算细微自然手势间的跨模态相似度，从而实现可靠的CSI。 在三人场景（最远34米）中，HiSync框架实现了92.32%的CSI准确性，比之前的最佳解决方案高出48.44%。此外，HiSync还在实际机器人部署中进行了验证。通过确保CSI既可靠又自然，HiSync为公共场所的人机交互提供了一种实用的基础技术，并对未来的系统设计提出了指导性建议。
URL
https://arxiv.org/abs/2603.11809
PDF
-
Edge-Assisted Multi-Robot Visual-Inertial SLAM with Efficient Communication
2026-03-11 03:35:28Xin Liu, Shuhuan Wen, Jing Zhao, Tony Z. Qiu, Hong ZhangAbstract
The integration of cloud computing and edge computing is an effective way to achieve global consistent and real-time multi-robot Simultaneous Localization and Mapping (SLAM). Cloud computing effectively solves the problem of limited computing, communication and storage capacity of terminal equipment. However, limited bandwidth and extremely long communication links between terminal devices and the cloud result in serious performance degradation of multi-robot SLAM systems. To reduce the computational cost of feature tracking and improve the real-time performance of the robot, a lightweight SLAM method of optical flow tracking based on pyramid IMU prediction is proposed. On this basis, a centralized multi-robot SLAM system based on a robot-edge-cloud layered architecture is proposed to realize real-time collaborative SLAM. It avoids the problems of limited on-board computing resources and low execution efficiency of single robot. In this framework, only the feature points and keyframe descriptors are transmitted and lossless encoding and compression are carried out to realize real-time remote information transmission with limited bandwidth resources. This design reduces the actual bandwidth occupied in the process of data transmission, and does not cause the loss of SLAM accuracy caused by data compression. Through experimental verification on the EuRoC dataset, compared with the current most advanced local feature compression method, our method can achieve lower data volume feature transmission, and compared with the current advanced centralized multi-robot SLAM scheme, it can achieve the same or better positioning accuracy under low computational load.
Abstract (translated)
云计算与边缘计算的融合是实现全球一致性和实时多机器人同时定位与地图构建（SLAM）的有效途径。云计算有效地解决了终端设备在计算、通信和存储能力方面的局限性问题。然而，终端设备与云端之间的有限带宽以及极长的通信链路导致了多机器人SLAM系统性能严重下降。为了降低特征跟踪的计算成本并提高机器人的实时性能，提出了一种基于金字塔惯性测量单元（IMU）预测的光流跟踪轻量级SLAM方法。在此基础上，提出了一个基于机器人-边缘-云分层架构的集中式多机器人SLAM系统，以实现实时协作SLAM。这种方法避免了车载计算资源有限和单个机器人执行效率低的问题。在该框架中，仅传输特征点和关键帧描述符，并进行无损编码压缩，从而实现在有限带宽资源下的实时远程信息传输。这一设计减少了数据传输过程中的实际带宽占用，并且不会因数据压缩而导致SLAM精度的损失。通过在EuRoC数据集上的实验验证表明，与目前最先进的局部特征压缩方法相比，我们的方法可以实现更小的数据量特征传输；与当前先进的集中式多机器人SLAM方案相比，在低计算负载下可以达到相同或更好的定位精度。
URL
https://arxiv.org/abs/2603.11085
PDF
-
Robotic Ultrasound Makes CBCT Alive
2026-03-10 20:37:24Feng Li, Ziyuan Li, Zhongliang Jiang, Nassir Navab, Yuan BiAbstract
Intraoperative Cone Beam Computed Tomography (CBCT) provides a reliable 3D anatomical context essential for interventional planning. However, its static nature fails to provide continuous monitoring of soft-tissue deformations induced by respiration, probe pressure, and surgical manipulation, leading to navigation discrepancies. We propose a deformation-aware CBCT updating framework that leverages robotic ultrasound as a dynamic proxy to infer tissue motion and update static CBCT slices in real time. Starting from calibration-initialized alignment with linear correlation of linear combination (LC2)-based rigid refinement, our method establishes accurate multimodal correspondence. To capture intraoperative dynamics, we introduce the ultrasound correlation UNet (USCorUNet), a lightweight network trained with optical flow-guided supervision to learn deformation-aware correlation representations, enabling accurate, real-time dense deformation field estimation from ultrasound streams. The inferred deformation is spatially regularized and transferred to the CBCT reference to produce deformation-consistent visualizations without repeated radiation exposure. We validate the proposed approach through deformation estimation and ultrasound-guided CBCT updating experiments. Results demonstrate real-time end-to-end CBCT slice updating and physically plausible deformation estimation, enabling dynamic refinement of static CBCT guidance during robotic ultrasound-assisted interventions. The source code is publicly available at this https URL.
Abstract (translated)
术中锥形束计算机断层摄影（CBCT）提供了一种可靠的三维解剖学背景，这对于介入规划至关重要。然而，由于其静态特性，无法持续监测由呼吸、探针压力和手术操作引起的软组织变形，导致导航误差。我们提出了一种基于机器人超声的变形感知CBCT更新框架，利用动态代理来推断组织运动，并实时更新静态CBCT切片。从线性组合（LC2）基础刚体细化的校准初始化对齐开始，我们的方法建立了准确的多模式对应关系。为了捕捉术中动力学变化，我们引入了超声相关UNet (USCorUNet)，这是一个轻量级网络，在光流引导监督下训练，以学习感知变形的相关表示，能够从超声流中实现精确、实时密集变形场估计。推断出的变形经过空间正则化后被转移到CBCT参考上，从而生成不需重复辐射暴露的变形一致可视化图。我们通过变形估计和超声引导CBCT更新实验验证了该方法的有效性。结果表明，实现了端到端的实时CBCT切片更新，并且可以进行物理上合理的变形估算，使静态CBCT指导在机器人超声辅助介入过程中能够动态精炼化。源代码可在以下链接获取：[此URL]
URL
https://arxiv.org/abs/2603.10220
PDF
-
Streaming Autoregressive Video Generation via Diagonal Distillation
2026-03-10 10:45:24Jinxiu Liu, Xuanming Liu, Kangfu Mei, Yandong Wen, Ming-HsuanYang, Weiyang LiuAbstract
Large pretrained diffusion models have significantly enhanced the quality of generated videos, and yet their use in real-time streaming remains limited. Autoregressive models offer a natural framework for sequential frame synthesis but require heavy computation to achieve high fidelity. Diffusion distillation can compress these models into efficient few-step variants, but existing video distillation approaches largely adapt image-specific methods that neglect temporal dependencies. These techniques often excel in image generation but underperform in video synthesis, exhibiting reduced motion coherence, error accumulation over long sequences, and a latency-quality trade-off. We identify two factors that result in these limitations: insufficient utilization of temporal context during step reduction and implicit prediction of subsequent noise levels in next-chunk prediction (i.e., exposure bias). To address these issues, we propose Diagonal Distillation, which operates orthogonally to existing approaches and better exploits temporal information across both video chunks and denoising steps. Central to our approach is an asymmetric generation strategy: more steps early, fewer steps later. This design allows later chunks to inherit rich appearance information from thoroughly processed early chunks, while using partially denoised chunks as conditional inputs for subsequent synthesis. By aligning the implicit prediction of subsequent noise levels during chunk generation with the actual inference conditions, our approach mitigates error propagation and reduces oversaturation in long-range sequences. We further incorporate implicit optical flow modeling to preserve motion quality under strict step constraints. Our method generates a 5-second video in 2.61 seconds (up to 31 FPS), achieving a 277.3x speedup over the undistilled model.
Abstract (translated)
大型预训练扩散模型在生成视频的质量上有了显著的提升，然而它们在实时流媒体中的应用仍然受到限制。自回归模型为序列帧合成提供了一个自然框架，但要达到高保真度则需要大量计算资源。扩散蒸馏可以将这些大型模型压缩成高效的几步变体版本，但是现有的视频蒸馏方法大多借鉴了特定于图像的方法，忽视了时间依赖性。虽然这些技术在图像生成中表现出色，但在视频合成中的表现较差，表现为运动连贯性降低、长时间序列中的误差积累以及延迟与质量之间的权衡问题。 我们发现导致这些问题的两个关键因素是：减少步骤过程中对时间上下文利用不足和下一片段预测（即曝光偏差）中隐式预测随后噪声水平的问题。为了解决这些问题，我们提出了对角蒸馏方法，这种方法与现有的方法形成了补充，并能更好地利用视频片段以及去噪步之间的时空信息。我们的方法的核心是对称生成策略：早期步骤多、后期步骤少。这种设计使后续片段能够从先前充分处理的片段中继承丰富的视觉信息，同时将部分去噪后的片段作为条件输入用于后续合成。通过在片段生成期间与实际推断条件对齐隐式预测随后噪声水平，我们的方法减轻了误差传播，并减少了长时间序列中的过度饱和问题。 此外，我们还结合了隐式光流建模以在严格的步骤限制下保持运动质量。我们的方法可以在2.61秒内生成5秒钟的视频（最高可达31帧每秒），比未蒸馏模型的速度提高了277.3倍。
URL
https://arxiv.org/abs/2603.09488
PDF
-
HDR-NSFF: High Dynamic Range Neural Scene Flow Fields
2026-03-09 12:36:46Shin Dong-Yeon, Kim Jun-Seong, Kwon Byung-Ki, Tae-Hyun OhAbstract
Radiance of real-world scenes typically spans a much wider dynamic range than what standard cameras can capture. While conventional HDR methods merge alternating-exposure frames, these approaches are inherently constrained to 2D pixel-level alignment, often leading to ghosting artifacts and temporal inconsistency in dynamic scenes. To address these limitations, we present HDR-NSFF, a paradigm shift from 2D-based merging to 4D spatio-temporal modeling. Our framework reconstructs dynamic HDR radiance fields from alternating-exposure monocular videos by representing the scene as a continuous function of space and time, and is compatible with both neural radiance field and 4D Gaussian Splatting (4DGS) based dynamic representations. This unified end-to-end pipeline explicitly models HDR radiance, 3D scene flow, geometry, and tone-mapping, ensuring physical plausibility and global coherence. We further enhance robustness by (i) extending semantic-based optical flow with DINO features to achieve exposure-invariant motion estimation, and (ii) incorporating a generative prior as a regularizer to compensate for limited observation in monocular captures and saturation-induced information loss. To evaluate HDR space-time view synthesis, we present the first real-world HDR-GoPro dataset specifically designed for dynamic HDR scenes. Experiments demonstrate that HDR-NSFF recovers fine radiance details and coherent dynamics even under challenging exposure variations, thereby achieving state-of-the-art performance in novel space-time view synthesis. Project page: this https URL
Abstract (translated)
真实世界的场景亮度范围通常远超出标准相机能够捕捉的范围。虽然传统的高动态范围（HDR）方法通过合并不同曝光度的照片来解决这个问题，但这些方法本质上局限于二维像素级别的对齐，这往往会导致动态场景中的鬼影效应和时间上的一致性问题。为了克服这些限制，我们提出了HDR-NSFF，这是一种从基于2D融合转向4D时空建模的范式转变。 我们的框架通过将场景表示为时间和空间上的连续函数来从交替曝光的单目视频中重建动态HDR辐射场，并且与神经辐射场和4DGaussian Splatting（4DGS）基的动态表示兼容。这一统一的端到端流水线显式地建模了HDR辐射、3D场景流、几何结构及色调映射，确保物理上的可信度和全局一致性。 为了增强鲁棒性，我们通过(i) 使用DINO特征扩展基于语义的光流以实现曝光不变性的运动估计；(ii) 作为正则化器纳入生成先验来弥补单目捕捉中的有限观测以及饱和导致的信息损失。 为评估HDR时空视图合成的效果，我们首次提出了针对动态HDR场景的真实世界HDR-GoPro数据集。实验表明，HDR-NSFF即使在挑战性的曝光变化下也能恢复精细的辐射细节和连贯的动力学，并因此达到了新型时空视图合成中的最先进性能。 项目页面：[此URL](this%20https%20URL)
URL
https://arxiv.org/abs/2603.08313
PDF
-
Event-based Motion & Appearance Fusion for 6D Object Pose Tracking
2026-03-09 11:34:50Zhichao Li, Chiara Bartolozzi, Lorenzo Natale, Arren GloverAbstract
Object pose tracking is a fundamental and essential task for robotics to perform tasks in the home and industrial settings. The most commonly used sensors to do so are RGB-D cameras, which can hit limitations in highly dynamic environments due to motion blur and frame-rate constraints. Event cameras have remarkable features such as high temporal resolution and low latency, which make them a potentially ideal vision sensors for object pose tracking at high speed. Even so, there are still only few works on 6D pose tracking with event cameras. In this work, we take advantage of the high temporal resolution and propose a method that uses both a propagation step fused with a pose correction strategy. Specifically, we use 6D object velocity obtained from event-based optical flow for pose propagation, after which, a template-based local pose correction module is utilized for pose correction. Our learning-free method has comparable performance to the state-of-the-art algorithms, and in some cases out performs them for fast-moving objects. The results indicate the potential for using event cameras in highly-dynamic scenarios where the use of deep network approaches are limited by low update rates.
Abstract (translated)
物体姿态跟踪是机器人在家庭和工业环境中执行任务的基础且必不可少的任务。最常用的传感器是RGB-D相机，但在动态强烈的环境下，这些设备可能会因运动模糊和帧率限制而受到限制。事件相机具有高时间分辨率和低延迟等显著特性，使其成为高速物体姿态跟踪的理想视觉传感器。尽管如此，目前利用事件相机进行6D姿态追踪的研究仍然很少。 在这项工作中，我们充分利用了事件相机的高时间分辨率，并提出了一种结合传播步骤与姿态修正策略的方法。具体来说，我们使用从基于事件的光流中获得的6D物体速度来执行姿态传播，在此之后，采用基于模板的局部姿态校正模块来进行姿态修正。我们的无学习方法在性能上可媲美当前最先进的算法，并且对于快速移动的对象，在某些情况下甚至超越了它们的表现。这些结果表明，事件相机在更新率较低的传统深度网络受限的应用场景中具有巨大的潜力。 通过这种方法，我们不仅展示了如何克服传统RGB-D传感器的限制，还为高速度和动态环境中的物体姿态跟踪开辟了一条新的途径。这为进一步探索事件相机在机器人任务中的应用奠定了基础，并可能开启一系列新的应用场景和技术突破。
URL
https://arxiv.org/abs/2603.08264
PDF
-
Fine-Grained 3D Facial Reconstruction for Micro-Expressions
2026-03-07 05:34:57Che Sun, Xinjie Zhang, Rui Gao, Xu Chen, Yuwei Wu, Yunde JiaAbstract
Recent advances in 3D facial expression reconstruction have demonstrated remarkable performance in capturing macro-expressions, yet the reconstruction of micro-expressions remains unexplored. This novel task is particularly challenging due to the subtle, transient, and low-intensity nature of micro-expressions, which complicate the extraction of stable and discriminative features essential for accurate reconstruction. In this paper, we propose a fine-grained micro-expression reconstruction method that integrates a global dynamic feature capturing stable facial motion patterns with a locally-enriched feature incorporating multiple informative cues from 2D motions, facial priors and 3D facial geometry. Specifically, we devise a plug-and-play dynamic-encoded module to extract micro-expression feature for global facial action, allowing it to leverage prior knowledge from abundant macro-expression data to mitigate the scarcity of micro-expression data. Subsequently, a dynamic-guided mesh deformation module is designed for extracting aggregated local features from dense optical flow, sparse landmark cues and facial mesh geometry, which adaptively refines fine-grained facial micro-expression without compromising global 3D geometry. Extensive experiments on micro-expression datasets demonstrate that our method consistently outperforms state-of-the-art methods in both geometric accuracy and perceptual detail.
Abstract (translated)
最近在3D面部表情重建方面取得的进展展示了捕捉宏观表情方面的出色性能，然而，对微观表情（micro-expressions）的重建研究尚处于探索阶段。这一新颖的任务尤其具有挑战性，因为微表情通常非常微妙、短暂且强度低，这使得提取稳定而有区分度的特征变得复杂，这些特征对于准确地进行微表情重建至关重要。 在本文中，我们提出了一种细粒度的微表情重构方法，该方法结合了用于捕获面部整体运动模式的全局动态特性以及通过2D运动、面部先验知识和3D面部几何结构获取的局部丰富特征。具体而言，我们设计了一个即插即用的动态编码模块来提取面向全球面部动作的微表情特征，并利用宏表情数据中的先前知识以缓解微表情数据稀缺的问题。随后，我们还设计了动态引导网格变形模块，用于从密集光流、稀疏地标线索和面部网格几何结构中提取聚合局部特性，从而在不损害全局3D几何的情况下适应性地细化细微的面部微表情。 通过在多个微表情数据集上进行广泛的实验，我们的方法在几何精度和感知细节方面始终优于现有最先进的技术。
URL
https://arxiv.org/abs/2603.07043
PDF
-
SurgSync: Time-Synchronized Multi-Modal Data Collection Framework and Dataset for Surgical Robotics
2026-03-06 22:38:45Haoying Zhou, Chang Liu, Yimeng Wu, Junlin Wu, Zijian Wu, Yu Chung Lee, Sara Martuscelli, Spetimiu E. Salcudean, Gregory S. Fischer, Peter KazanzidesAbstract
Most existing robotic surgery systems adopt a human-in-the-loop paradigm, often with the surgeon directly teleoperating the robotic system. Adding intelligence to these robots would enable higher-level control, such as supervised autonomy or even full autonomy. However, artificial intelligence (AI) requires large amounts of training data, which is currently lacking. This work proposes SurgSync, a multi-modal data collection framework with offline and online synchronization to support training and real-time inference, respectively. The framework is implemented on a da Vinci Research Kit (dVRK) and introduces (1) dual-mode (online/offline-matching) synchronized recorders, (2) a modern stereo endoscope to achieve image quality on par with clinical systems, and (3) additional sensors such as a side-view camera and a novel capacitive contact sensor to provide ground truth contact data. The framework also incorporates a post-processing toolbox for tasks such as depth estimation, optical flow, and a practical kinematic reprojection method using Gaussian heatmap. User studies with participants of varying skill levels are performed with ex-vivo tissue to provide clinically realistic data, and a network for surgical skill assessment is employed to demonstrate utilization of the collected data. Through the user study experiments, we obtained a dataset of 214 validated instances across multiple canonical training tasks. All software and data are available at this http URL.
Abstract (translated)
大多数现有的机器人手术系统采用人在环路（human-in-the-loop）的模式，通常是由外科医生直接操控机器人系统。向这些机器人添加智能功能将能够实现更高层次的控制，比如监督自主性或完全自主性。然而，人工智能（AI）需要大量的训练数据，而目前这方面的数据严重不足。为此，本研究提出了一种名为SurgSync的多模态数据采集框架，该框架包括离线和在线同步两个部分，分别用于支持训练和实时推断。该框架在达芬奇机器人手术系统的研发套件（da Vinci Research Kit, dVRK）上实现，并引入了以下三个关键组件： 1. 双模式（在线/离线匹配）的同步记录器； 2. 一种现代立体内窥镜，能够在图像质量方面与临床系统相媲美； 3. 额外传感器，如侧视摄像头和创新的电容接触传感器，提供真实的接触数据。 此外，该框架还集成了一个后期处理工具包，用于执行深度估计、光学流等任务，并采用高斯热图（Gaussian heatmap）来实现实际运动学重投影方法。通过使用离体组织进行技能水平各异参与者的用户研究，获取了临床真实的数据；同时部署了一个手术技能评估网络以展示收集数据的利用情况。通过这些用户实验研究，我们获得了涵盖多个典型训练任务的214个验证实例的数据集。所有软件和数据均可在提供的网址上获得。
URL
https://arxiv.org/abs/2603.06919
PDF
-
An Extended Topological Model For High-Contrast Optical Flow
2026-03-06 20:18:01Brad Turow, Jose A. PereaAbstract
In this paper, we identify low-dimensional models for dense core subsets in the space of $3\times 3$ high-contrast optical flow patches sampled from the Sintel dataset. In particular, we leverage the theory of approximate and discrete circle bundles to identify a 3-manifold whose boundary is a previously proposed optical flow torus, together with disjoint circles corresponding to pairs of binary step-edge range image patches. The 3-manifold model we introduce provides an explanation for why the previously-proposed torus model could not be verified with direct methods (e.g., a straightforward persistent homology computation). We also demonstrate that nearly all optical flow patches in the top 1 percent by contrast norm are found near the family of binary step-edge circles described above, rather than the optical flow torus, and that these frequently occurring patches are concentrated near motion boundaries (which are of particular importance for computer vision tasks such as object segmentation and tracking). Our findings offer insights on the subtle interplay between topology and geometry in inference for visual data.
Abstract (translated)
在这篇论文中，我们为Sintel数据集中采样的高对比度$3\times 3$光学流补丁的空间中的密集核心子集识别出了低维模型。具体来说，我们利用近似和离散圆束理论来识别出一个三维流形，其边界是一个先前提出的光学流环面，并且与二进制阶跃边缘范围图像补丁对相对应的分离圆圈相交。 我们引入的三维流形模型解释了为什么之前的环面模型不能通过直接方法（例如，简单的持久同调计算）来验证。此外，我们也展示了几乎所有按对比度范数排名前1%的光学流补丁都靠近上述二进制阶跃边缘圆家族，而非光学流环面上。并且，这些频繁出现的补丁集中在运动边界附近（这对于如目标分割和跟踪等计算机视觉任务特别重要）。 我们的发现提供了关于在视觉数据推断中拓扑与几何之间微妙相互作用的理解。
URL
https://arxiv.org/abs/2603.06853
PDF
-
RealWonder: Real-Time Physical Action-Conditioned Video Generation
2026-03-05 18:22:54Wei Liu, Ziyu Chen, Zizhang Li, Yue Wang, Hong-Xing Yu, Jiajun WuAbstract
Current video generation models cannot simulate physical consequences of 3D actions like forces and robotic manipulations, as they lack structural understanding of how actions affect 3D scenes. We present RealWonder, the first real-time system for action-conditioned video generation from a single image. Our key insight is using physics simulation as an intermediate bridge: instead of directly encoding continuous actions, we translate them through physics simulation into visual representations (optical flow and RGB) that video models can process. RealWonder integrates three components: 3D reconstruction from single images, physics simulation, and a distilled video generator requiring only 4 diffusion steps. Our system achieves 13.2 FPS at 480x832 resolution, enabling interactive exploration of forces, robot actions, and camera controls on rigid objects, deformable bodies, fluids, and granular materials. We envision RealWonder opens new opportunities to apply video models in immersive experiences, AR/VR, and robot learning. Our code and model weights are publicly available in our project website: this https URL
Abstract (translated)
当前的视频生成模型无法模拟3D动作（如力和机器人操作）对物理世界的连续影响，因为它们缺乏对动作如何影响3D场景的结构理解。我们提出了一种名为RealWonder的新系统，这是首个基于单张图像进行动作条件下的实时视频生成系统的解决方案。 我们的关键洞察在于使用物理学仿真作为中间桥梁：不是直接编码连续的动作，而是通过物理学仿真将其转换为视觉表示（光流和RGB），这些是视频模型能够处理的。 RealWonder集成三大组件：从单一图片重建3D场景、物理仿真以及一个仅需4次扩散步骤的精简视频生成器。 该系统在480x832分辨率下可达到13.2帧每秒的速度，支持刚体对象、可变形物体、流体和颗粒材料上的力、机器人动作及相机控制交互式的探索。我们设想RealWonder将为视频模型在沉浸式体验、AR/VR以及机器人学习中的应用开辟新的机会。 我们的代码和模型权重可在项目网站上公开获取：[此URL](this https URL)
URL
https://arxiv.org/abs/2603.05449
PDF
-
FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generation
2026-03-05 07:41:34Ganggui Ding, Hao Chen, Xiaogang XuAbstract
Large pre-trained video diffusion models excel in video frame interpolation but struggle to generate high fidelity frames due to reliance on intrinsic generative priors, limiting detail preservation from start and end frames. Existing methods often depend on motion control for temporal consistency, yet dense optical flow is error-prone, and sparse points lack structural context. In this paper, we propose FC-VFI for faithful and consistent video frame interpolation, supporting \(4\times\)x and \(8\times\) interpolation, boosting frame rates from 30 FPS to 120 and 240 FPS at \(2560\times 1440\)resolution while preserving visual fidelity and motion consistency. We introduce a temporal modeling strategy on the latent sequences to inherit fidelity cues from start and end frames and leverage semantic matching lines for structure-aware motion guidance, improving motion consistency. Furthermore, we propose a temporal difference loss to mitigate temporal inconsistencies. Extensive experiments show FC-VFI achieves high performance and structural integrity across diverse scenarios.
Abstract (translated)
大型预训练视频扩散模型在视频帧插值方面表现出色，但在生成高保真度的帧时却由于依赖于内在的生成先验而难以保留细节。现有的方法通常依靠运动控制来保持时间一致性，然而密集光流容易出错，稀疏点则缺乏结构上下文信息。在这篇论文中，我们提出了FC-VFI（Faithful and Consistent Video Frame Interpolation）模型，旨在实现忠实且一致的视频帧插值，并支持4倍和8倍的插值率，在2560×1440分辨率下将帧率从30 FPS提升至120 FPS及240 FPS的同时保持视觉保真度与运动一致性。我们引入了一种时序建模策略，用于在潜在序列中继承起始帧和结束帧的保真度线索，并利用语义匹配线来为结构感知运动引导提供支持，从而提高运动一致性。此外，我们还提出了一种时间差异损失函数以减少时间上的不一致现象。通过广泛的实验验证，FC-VFI模型在各种场景下均展现出高水准的表现及结构完整性。
URL
https://arxiv.org/abs/2603.04899
PDF
-
ArtHOI: Articulated Human-Object Interaction Synthesis by 4D Reconstruction from Video Priors
2026-03-04 17:58:04Zihao Huang, Tianqi Liu, Zhaoxi Chen, Shaocong Xu, Saining Zhang, Lixing Xiao, Zhiguo Cao, Wei Li, Hao Zhao, Ziwei LiuAbstract
Synthesizing physically plausible articulated human-object interactions (HOI) without 3D/4D supervision remains a fundamental challenge. While recent zero-shot approaches leverage video diffusion models to synthesize human-object interactions, they are largely confined to rigid-object manipulation and lack explicit 4D geometric reasoning. To bridge this gap, we formulate articulated HOI synthesis as a 4D reconstruction problem from monocular video priors: given only a video generated by a diffusion model, we reconstruct a full 4D articulated scene without any 3D supervision. This reconstruction-based approach treats the generated 2D video as supervision for an inverse rendering problem, recovering geometrically consistent and physically plausible 4D scenes that naturally respect contact, articulation, and temporal coherence. We introduce ArtHOI, the first zero-shot framework for articulated human-object interaction synthesis via 4D reconstruction from video priors. Our key designs are: 1) Flow-based part segmentation: leveraging optical flow as a geometric cue to disentangle dynamic from static regions in monocular video; 2) Decoupled reconstruction pipeline: joint optimization of human motion and object articulation is unstable under monocular ambiguity, so we first recover object articulation, then synthesize human motion conditioned on the reconstructed object states. ArtHOI bridges video-based generation and geometry-aware reconstruction, producing interactions that are both semantically aligned and physically grounded. Across diverse articulated scenes (e.g., opening fridges, cabinets, microwaves), ArtHOI significantly outperforms prior methods in contact accuracy, penetration reduction, and articulation fidelity, extending zero-shot interaction synthesis beyond rigid manipulation through reconstruction-informed synthesis.
Abstract (translated)
合成物理上合理的人体与物体交互（HOI）场景，而无需3D或4D的监督数据，仍然是一个基本挑战。尽管最近的一些零样本方法利用视频扩散模型来合成人体与物体之间的互动，但这些方法主要局限于对刚性对象的操作，并缺乏显式的四维几何推理能力。为了弥合这一差距，我们把关节式HOI生成定义为一项单目视频先验下的四维重建任务：仅使用由扩散模型生成的视频作为输入，无需任何3D监督数据，即可重构出完整的四维关节场景。基于这种重建的方法将生成的2D视频视为反向渲染问题中的监督信号，在保证几何一致性和物理合理性的前提下尊重接触、关节和时间连续性。 我们引入了ArtHOI——第一个通过从视频先验中进行4D重建来实现零样本关节式人体与物体交互合成的框架。我们的关键设计包括： 1. 基于流的人体部位分割：利用光流作为几何线索，将动态区域从单目视频中的静态区域中分离出来； 2. 解耦重构流水线：在单目模糊性下同时优化人类运动和对象关节是不稳定的，因此我们首先恢复物体的关节状态，然后基于重建的对象状态条件生成人体动作。 ArtHOI通过视频生成与几何感知重建之间的桥梁产生了语义对齐且物理实证的互动场景。在各种复杂的关节式场景（例如打开冰箱、橱柜、微波炉）中，ArtHOI在接触准确性、穿透减少和关节真实度上明显优于先前的方法，在没有监督数据的情况下扩展了零样本交互合成能力，超越了刚性操作范围，并通过重建指导合成来实现这一目标。
URL
https://arxiv.org/abs/2603.04338
PDF
-
Architecture and evaluation protocol for transformer-based visual object tracking in UAV applications
2026-03-04 10:12:12Augustin Borne (ISL, Hochschule Karlsruhe -- Technik und Wirtschaft Karlsruhe University of Applied Sciences, IRIMAS), Pierre Notin (ISL), Christophe Hennequin (ISL), Sebastien Changey (ISL), Stephane Bazeille (IRIMAS), Christophe Cudel (IRIMAS), Franz QuintAbstract
Object tracking from Unmanned Aerial Vehicles (UAVs) is challenged by platform dynamics, camera motion, and limited onboard resources. Existing visual trackers either lack robustness in complex scenarios or are too computationally demanding for real-time embedded use. We propose an Modular Asynchronous Tracking Architecture (MATA) that combines a transformer-based tracker with an Extended Kalman Filter, integrating ego-motion compensation from sparse optical flow and an object trajectory model. We further introduce a hardware-independent, embedded oriented evaluation protocol and a new metric called Normalized time to Failure (NT2F) to quantify how long a tracker can sustain a tracking sequence without external help. Experiments on UAV benchmarks, including an augmented UAV123 dataset with synthetic occlusions, show consistent improvements in Success and NT2F metrics across multiple tracking processing frequency. A ROS 2 implementation on a Nvidia Jetson AGX Orin confirms that the evaluation protocol more closely matches real-time performance on embedded systems.
Abstract (translated)
来自无人飞行器（UAV）的对象跟踪面临平台动态、相机运动和有限的机载资源等挑战。现有的视觉追踪方法要么在复杂场景中不够鲁棒，要么对实时嵌入式应用来说计算需求过高。我们提出了一种模块化异步跟踪架构（MATA），该架构结合了基于变压器的追踪器和扩展卡尔曼滤波器，并集成了稀疏光流带来的自我运动补偿以及对象轨迹模型。此外，我们还引入了一种硬件无关、面向嵌入式设备的评估协议，以及一种新的度量标准——归一化失效时间（NT2F），用以量化追踪器在没有外部帮助的情况下能够维持跟踪序列的时间长度。实验结果表明，在包括增强版UAV123数据集（该数据集中合成了一些遮挡情况）在内的多个无人机基准测试中，MATA方法在成功率和NT2F度量标准上都有显著的改进，并且这些改进在不同的处理频率下是一致的。基于Nvidia Jetson AGX Orin设备上的ROS 2实现表明，这种评估协议更贴近嵌入式系统中的实时性能。 这段文字描述了一种新的对象跟踪架构以及相关的实验结果和评价方法，适用于无人飞行器（UAV）等场景下的视觉追踪任务。
URL
https://arxiv.org/abs/2603.03904
PDF
-
FREE-Edit: Using Editing-aware Injection in Rectified Flow Models for Zero-shot Image-Driven Video Editing
2026-03-01 16:01:44Maomao Li, Yunfei Liu, Yu LiAbstract
Image-driven video editing aims to propagate edit contents from the modified first frame to the rest frames. The existing methods usually invert the source video to noise using a pre-trained image-to-video (I2V) model and then guide the sampling process using the edited first frame. Generally, a popular choice for maintaining motion and layout from the source video is intervening in the denoising process by injecting attention during reconstruction. However, such injection often leads to unsatisfactory results, where excessive injection leads to conflicting semantics from the source video while insufficient injection brings limited source representation. Recognizing this, we propose an Editing-awaRE (REE) injection method to modulate injection intensity of each token. Specifically, we first compute the pixel difference between the source and edited first frame to form a corresponding editing mask. Next, we track the editing area throughout the entire video by using optical flow to warp the first-frame mask. Then, editing-aware feature injection intensity for each token is generated accordingly, where injection is not conducted on editing areas. Building upon REE injection, we further propose a zero-shot image-driven video editing framework with recent-emerging rectified-Flow models, dubbed FREE-Edit. Without fine-tuning or training, our FREE-Edit demonstrates effectiveness in various image-driven video editing scenarios, showing its capability to produce higher-quality outputs compared with existing techniques. Project page: this https URL.
Abstract (translated)
图像驱动的视频编辑旨在将修改后的第一帧上的编辑内容传播到其余所有帧上。现有方法通常会使用预训练的图像到视频（I2V）模型将源视频逆向转换为噪声，然后通过编辑过的首帧来引导采样过程。为了从源视频中保持运动和布局的一致性，一种常见的做法是在去噪过程中注入注意力以进行重建。然而，这种注入方式往往会导致不尽人意的结果：过度注入会使得原始视频中的语义与编辑内容产生冲突，而不足的注入则无法充分表示源视频的内容。 为了应对这一挑战，我们提出了一种名为Editing-awaRE（REE）的注入方法，该方法可以调节每个标记的注入强度。具体来说，首先计算源帧和编辑后的第一帧之间的像素差异，以形成对应的编辑掩码。接着，通过使用光学流技术来变形首帧掩码，我们可以跟踪整个视频中的编辑区域。根据这一过程生成了针对每个标记的、具有编辑意识的功能注入强度，在此过程中不会在编辑区域内进行任何功能注入。 基于REE注入方法，我们进一步提出了一种零样本图像驱动的视频编辑框架——FREE-Edit，并结合了最近出现的矫正流模型（rectified-Flow models）。无需微调或训练，我们的FREE-Edit框架能够在各种图像驱动的视频编辑场景中展现其有效性，并且能够产生比现有技术更高的质量输出。 项目页面：[请在此处替换为实际链接]
URL
https://arxiv.org/abs/2603.01164
PDF
-
Micro-expression Recognition Based on Dual-branch Feature Extraction and Fusion
2026-02-27 11:52:50Mingjie Zhang, Bo Li, Wanting Liu, Hongyan Cui, Yue Li, Qingwen Li, Hong Li, Ge GaoAbstract
Micro-expressions, characterized by transience and subtlety, pose challenges to existing optical flow-based recognition methods. To address this, this paper proposes a dual-branch micro-expression feature extraction network integrated with parallel attention. Key contributions include: 1) a residual network designed to alleviate gradient anishing and network degradation; 2) an Inception network constructed to enhance model representation and suppress interference from irrelevant regions; 3) an adaptive feature fusion module developed to integrate dual-branch features. Experiments on the CASME II dataset demonstrate that the proposed method achieves 74.67% accuracy, outperforming LBP-TOP (by 11.26%), MSMMT (by 3.36%), and other comparative methods.
Abstract (translated)
微表情由于其短暂性和微妙性，对现有的基于光学流的识别方法构成了挑战。为解决这一问题，本文提出了一种集成了并行注意力机制的双分支微表情特征提取网络。主要贡献包括：1）设计了一个残差网络来缓解梯度消失和网络退化的问题；2）构建了一个Inception网络以增强模型表示能力，并抑制无关区域的干扰；3）开发了一个自适应特征融合模块，用于整合双分支特征。在CASME II数据集上的实验表明，所提出的方法实现了74.67%的准确率，优于LBP-TOP（高出11.26%）、MSMMT（高出3.36%）和其他比较方法。
URL
https://arxiv.org/abs/2602.23950
PDF
-
SelfOccFlow: Towards end-to-end self-supervised 3D Occupancy Flow prediction
2026-02-27 10:42:01Xavier Timoneda, Markus Herb, Fabian Duerr, Daniel GoehringAbstract
Estimating 3D occupancy and motion at the vehicle's surroundings is essential for autonomous driving, enabling situational awareness in dynamic environments. Existing approaches jointly learn geometry and motion but rely on expensive 3D occupancy and flow annotations, velocity labels from bounding boxes, or pretrained optical flow models. We propose a self-supervised method for 3D occupancy flow estimation that eliminates the need for human-produced annotations or external flow supervision. Our method disentangles the scene into separate static and dynamic signed distance fields and learns motion implicitly through temporal aggregation. Additionally, we introduce a strong self-supervised flow cue derived from features' cosine similarities. We demonstrate the efficacy of our 3D occupancy flow method on SemanticKITTI, KITTI-MOT, and nuScenes.
Abstract (translated)
在自动驾驶中，估算车辆周围环境的三维占用情况和运动状态对于实现情境感知至关重要，尤其是在动态环境中。现有的方法能够同时学习几何形状和运动，但依赖于昂贵的3D占用标注、来自边界框的速度标签或预训练的光流模型。我们提出了一种无需人工生成注释或外部流动监督的自我监督方法来估算三维占用流。该方法将场景分解为独立的静态和动态有符号距离字段，并通过时间聚合隐式学习运动。此外，我们引入了一个强大的自我监督流动线索，来源于特征余弦相似度。我们在SemanticKITTI、KITTI-MOT和nuScenes数据集上展示了我们的3D占用流估算方法的有效性。
URL
https://arxiv.org/abs/2602.23894
PDF
-
DMAligner: Enhancing Image Alignment via Diffusion Model Based View Synthesis
2026-02-26 14:00:07Xinglong Luo, Ao Luo, Zhengning Wang, Yueqi Yang, Chaoyu Feng, Lei Lei, Bing Zeng, Shuaicheng LiuAbstract
Image alignment is a fundamental task in computer vision with broad applications. Existing methods predominantly employ optical flow-based image warping. However, this technique is susceptible to common challenges such as occlusions and illumination variations, leading to degraded alignment visual quality and compromised accuracy in downstream tasks. In this paper, we present DMAligner, a diffusion-based framework for image alignment through alignment-oriented view synthesis. DMAligner is crafted to tackle the challenges in image alignment from a new perspective, employing a generation-based solution that showcases strong capabilities and avoids the problems associated with flow-based image warping. Specifically, we propose a Dynamics-aware Diffusion Training approach for learning conditional image generation, synthesizing a novel view for image alignment. This incorporates a Dynamics-aware Mask Producing (DMP) module to adaptively distinguish dynamic foreground regions from static backgrounds, enabling the diffusion model to more effectively handle challenges that classical methods struggle to solve. Furthermore, we develop the Dynamic Scene Image Alignment (DSIA) dataset using Blender, which includes 1,033 indoor and outdoor scenes with over 30K image pairs tailored for image alignment. Extensive experimental results demonstrate the superiority of the proposed approach on DSIA benchmarks, as well as on a series of widely-used video datasets for qualitative comparisons. Our code is available at this https URL.
Abstract (translated)
图像对齐是计算机视觉中的基本任务，具有广泛的应用。现有方法主要采用基于光学流的图像扭曲技术。然而，这种方法容易受到遮挡和光照变化等常见问题的影响，导致对齐视觉质量下降，并且在下游任务中准确性降低。本文介绍了DMAligner，这是一种通过面向对齐视角合成的扩散框架进行图像对齐的方法。DMAligner 旨在从一个新的角度解决图像对齐中的挑战，采用基于生成的方法，展示出强大的能力并避免了与流基图像扭曲相关的问题。 具体而言，我们提出了一个感知动态性的扩散训练方法，用于学习条件下的图像生成，为图像对齐合成新视角。这包括了一个感知动态性掩码产生的（DMP）模块，能够自适应地区分动态前景区域和静态背景，使扩散模型更有效地处理传统方法难以解决的挑战。 此外，我们使用Blender开发了Dynamic Scene Image Alignment (DSIA) 数据集，其中包含1,033个室内和室外场景以及超过30K张图像对，专为图像对齐设计。广泛的实验结果展示了该方法在DSIA基准测试中的优越性，并且在一系列广泛使用的视频数据集中进行定性比较时也表现良好。 我们的代码可在[此链接](https://这个URL需要替换为你提供的具体URL)获取。
URL
https://arxiv.org/abs/2602.23022
PDF
-
Velocity and stroke rate reconstruction of canoe sprint team boats based on panned and zoomed video recordings
2026-02-26 12:31:30Julian Ziegler, Daniel Matthes, Finn Gerdts, Patrick Frenzel, Torsten Warnke, Matthias Englert, Tina Koevari, Mirco FuchsAbstract
Pacing strategies, defined by velocity and stroke rate profiles, are essential for peak performance in canoe sprint. While GPS is the gold standard for analysis, its limited availability necessitates automated video-based solutions. This paper presents an extended framework for reconstructing performance metrics from panned and zoomed video recordings across all sprint disciplines (K1-K4, C1-C2) and distances (200m-500m). Our method utilizes YOLOv8 for buoy and athlete detection, leveraging the known buoy grid to estimate homographies. We generalized the estimation of the boat position by means of learning a boat-specific athlete offset using a U-net based boat tip calibration. Further, we implement a robust tracking scheme using optical flow to adapt to multi-athlete boat types. Finally, we introduce methods to extract stroke rate information from either pose estimations or the athlete bounding boxes themselves. Evaluation against GPS data from elite competitions yields a velocity RRMSE of 0.020 +- 0.011 (rho = 0.956) and a stroke rate RRMSE of 0.022 +- 0.024 (rho = 0.932). The methods provide coaches with highly accurate, automated feedback without requiring on-boat sensors or manual annotation.
Abstract (translated)
划船竞速中的速度和划桨频率策略对于达到巅峰表现至关重要。虽然全球定位系统（GPS）是分析的最佳标准，但由于其有限的可用性，需要自动化的视频解决方案。本文提出了一种扩展框架，用于从平移和缩放视频记录中重建所有冲刺项目（K1-K4, C1-C2）及距离（200米至500米）的性能指标。 我们的方法利用YOLOv8检测浮标和运动员，并通过已知的浮标网格来估计同构映射。我们通过学习特定于船的运动员偏移量，采用基于U-net的船尖校准方法，推广了对船位置的估算。此外，我们实施了一种使用光流实现的鲁棒跟踪方案，以适应多运动员类型的船只。 最后，我们引入了几种从姿势估计或运动员边界框自身提取划桨频率信息的方法。与来自精英比赛的GPS数据对比评估显示，速度的相对均方根误差（RRMSE）为0.020 ± 0.011 (ρ = 0.956)，划桨频率的RRMSE为0.022 ± 0.024 (ρ = 0.932)。 这些方法为教练提供了高精度且自动化的反馈，无需船上传感器或手动标注。
URL
https://arxiv.org/abs/2602.22941
PDF
-
GSTurb: Gaussian Splatting for Atmospheric Turbulence Mitigation
2026-02-26 09:37:27Hanliang Du, Zhangji Lu, Zewei Cai, Qijian Tang, Qifeng Yu, Xiaoli LiuAbstract
Atmospheric turbulence causes significant image degradation due to pixel displacement (tilt) and blur, particularly in long-range imaging applications. In this paper, we propose a novel framework for atmospheric turbulence mitigation, GSTurb, which integrates optical flow-guided tilt correction and Gaussian splatting for modeling non-isoplanatic blur. The framework employs Gaussian parameters to represent tilt and blur, and optimizes them across multiple frames to enhance restoration. Experimental results on the ATSyn-static dataset demonstrate the effectiveness of our method, achieving a peak PSNR of 27.67 dB and SSIM of 0.8735. Compared to the state-of-the-art method, GSTurb improves PSNR by 1.3 dB (a 4.5% increase) and SSIM by 0.048 (a 5.8% increase). Additionally, on real datasets, including the TSRWGAN Real-World and CLEAR datasets, GSTurb outperforms existing methods, showing significant improvements in both qualitative and quantitative performance. These results highlight that combining optical flow-guided tilt correction with Gaussian splatting effectively enhances image restoration under both synthetic and real-world turbulence conditions. The code for this method will be available at this https URL.
Abstract (translated)
大气湍流会导致图像退化，主要原因是像素位移（倾斜）和模糊，尤其是在长距离成像应用中。本文提出了一种新的大气湍流缓解框架GSTurb，该框架结合了光学流引导的倾斜校正和高斯喷射来建模非同像平面模糊。此框架使用高斯参数表示倾斜和模糊，并在多帧之间优化这些参数以增强图像恢复效果。 实验结果表明，在ATSyn-static数据集上，我们的方法达到了峰值PSNR为27.67 dB和SSIM为0.8735的性能。相较于最先进的方法，GSTurb将PSNR提高了1.3 dB（增幅4.5%），并将SSIM提高了0.048（增幅5.8%）。此外，在真实数据集TSRWGAN Real-World和CLEAR上，GSTurb也优于现有方法，无论是在定性还是定量性能方面都表现出显著的改进。这些结果表明，结合光学流引导的倾斜校正与高斯喷射可以有效地增强合成湍流和现实世界湍流条件下的图像恢复。 该方法的相关代码将在此网址提供：[此链接](https://this-url.com)（请将此处的链接替换为实际提供的GitHub或其他存储库链接）。
URL
https://arxiv.org/abs/2602.22800
PDF
-
DAGS-SLAM: Dynamic-Aware 3DGS SLAM via Spatiotemporal Motion Probability and Uncertainty-Aware Scheduling
2026-02-25 07:14:07Li Zhang, Yu-An Liu, Xijia Jiang, Conghao Huang, Danyang Li, Yanyong ZhangAbstract
Mobile robots and IoT devices demand real-time localization and dense reconstruction under tight compute and energy budgets. While 3D Gaussian Splatting (3DGS) enables efficient dense SLAM, dynamic objects and occlusions still degrade tracking and mapping. Existing dynamic 3DGS-SLAM often relies on heavy optical flow and per-frame segmentation, which is costly for mobile deployment and brittle under challenging illumination. We present DAGS-SLAM, a dynamic-aware 3DGS-SLAM system that maintains a spatiotemporal motion probability (MP) state per Gaussian and triggers semantics on demand via an uncertainty-aware scheduler. DAGS-SLAM fuses lightweight YOLO instance priors with geometric cues to estimate and temporally update MP, propagates MP to the front-end for dynamic-aware correspondence selection, and suppresses dynamic artifacts in the back-end via MP-guided optimization. Experiments on public dynamic RGB-D benchmarks show improved reconstruction and robust tracking while sustaining real-time throughput on a commodity GPU, demonstrating a practical speed-accuracy tradeoff with reduced semantic invocations toward mobile deployment.
Abstract (translated)
移动机器人和物联网设备在计算资源和能源有限的情况下，需要实时定位和密集重建。虽然3D高斯点阵（3D Gaussian Splatting，简称3DGS）能够实现高效的稠密SLAM（即时定位与地图构建），但是动态物体和遮挡仍然会降低跟踪和建图的质量。现有的动态3DGS-SLAM系统通常依赖于重量级的光流计算和逐帧分割，这在移动部署中成本高昂且在照明条件恶劣的情况下稳定性较差。 我们提出了一个名为DAGS-SLAM（动态感知3DGS-SLAM）的系统，该系统为每个高斯点维护一个时空运动概率（Motion Probability, MP）状态，并通过不确定性感知调度器按需触发语义处理。DAGS-SLAM将轻量级YOLO实例先验与几何线索相结合以估计和临时更新MP，在前端传播MP以便动态感知对应选择，同时在后端利用MP引导优化来抑制动态伪影。 实验显示，DAGS-SLAM在公共的动态RGB-D基准测试中显著提升了重建质量和跟踪鲁棒性，并能在商品级GPU上维持实时吞吐量。这表明通过减少语义调用，DAGS-SLAM实现了移动部署所需的实用速度精度权衡。
URL
https://arxiv.org/abs/2602.21644
PDF
