Abstract
Extracting the true dynamical variables of a system from high-dimensional video is challenging due to distracting visual factors such as background motion, occlusions, and texture changes. We propose LyTimeT, a two-phase framework for interpretable variable extraction that learns robust and stable latent representations of dynamical systems. In Phase 1, LyTimeT employs a spatio-temporal TimeSformer-based autoencoder that uses global attention to focus on dynamically relevant regions while suppressing nuisance variation, enabling distraction-robust latent state learning and accurate long-horizon video prediction. In Phase 2, we probe the learned latent space, select the most physically meaningful dimensions using linear correlation analysis, and refine the transition dynamics with a Lyapunov-based stability regularizer to enforce contraction and reduce error accumulation during roll-outs. Experiments on five synthetic benchmarks and four real-world dynamical systems, including chaotic phenomena, show that LyTimeT achieves mutual information and intrinsic dimension estimates closest to ground truth, remains invariant under background perturbations, and delivers the lowest analytical mean squared error among CNN-based (TIDE) and transformer-only baselines. Our results demonstrate that combining spatio-temporal attention with stability constraints yields predictive models that are not only accurate but also physically interpretable.
Abstract (translated)
从高维视频中提取系统的真正动态变量是一个挑战，因为会受到诸如背景运动、遮挡和纹理变化等分散注意力的视觉因素的影响。我们提出了一种名为LyTimeT的两阶段框架，该框架旨在学习鲁棒且稳定的潜在表示来提取可解释的变量。在第一阶段，LyTimeT采用基于时空Transformer（TimeSformer）的自编码器，利用全局注意机制聚焦于动态相关的区域并抑制干扰变化，从而实现抗分心的隐态学习和准确的长时视频预测。 第二阶段中，我们探究已学得的潜在空间，使用线性相关分析选择最具有物理意义的维度，并通过基于李雅普诺夫稳定性正则化的约束来优化转换动力学，以确保收缩并减少在展开过程中的误差累积。实验结果表明，在五个合成基准和四个真实世界动态系统（包括混沌现象）上，LyTimeT能够提供与实际情况最为接近的互信息和固有维度估计值，并且在背景扰动下保持不变性，同时在基于CNN（TIDE）和其他仅基于Transformer的方法中具有最低的解析均方误差。 我们的研究结果表明，结合时空注意力机制与稳定性约束可以生成不仅准确而且物理可解释的预测模型。
URL
https://arxiv.org/abs/2510.19716
