Abstract
Infrared and visible video fusion plays a critical role in intelligent surveillance and low-light monitoring. However, maintaining temporal stability while preserving spatial detail remains a fundamental challenge. Existing methods either focus on frame-wise enhancement with limited temporal modeling or rely on heavy spatio-temporal aggregation that often sacrifices high-frequency details. In this paper, we propose FTPFusion, a frequency-aware infrared and visible video fusion method based on temporal perturbation and sparse cross-modal interaction. Specifically, FTPFusion decomposes the feature representations into high-frequency and low-frequency components for collaborative modeling. The high-frequency branch performs sparse cross-modal spatio-temporal interaction to capture motion-related context and complementary details. The low-frequency branch introduces a temporal perturbation strategy to enhance robustness against complex video variations, such as flickering, jitter, and local misalignment. Furthermore, we design an offset-aware temporal consistency constraint to explicitly stabilize cross-frame representations under temporal disturbances. Extensive experiments on multiple public benchmarks demonstrate that FTPFusion consistently outperforms state-of-the-art methods across multiple metrics in both spatial fidelity and temporal consistency. The source code will be available at this https URL.
Abstract (translated)
红外与可见光视频融合在智能监控与低照度监测中至关重要。然而，如何在保持空间细节的同时维持时间稳定性仍是一项根本挑战。现有方法要么侧重于帧级增强而时间建模能力有限，要么依赖沉重的时空聚合机制，往往牺牲高频细节。本文提出FTPFusion，一种基于时间扰动与稀疏跨模态交互的频域感知红外与可见光视频融合方法。具体而言，FTPFusion将特征表示分解为高频与低频分量进行协同建模：高频分支执行稀疏跨模态时空交互以捕获运动相关上下文与互补细节；低频分支引入时间扰动策略以增强对闪烁、抖动及局部错位等复杂视频变化的鲁棒性。此外，我们设计了偏移感知的时间一致性约束，以在时间扰动下显式稳定跨帧表示。在多个公开基准数据集上的大量实验表明，FTPFusion在空间保真度与时间一致性多项指标上均持续优于最新方法。源代码将在以下网址公开。
URL
https://arxiv.org/abs/2604.01900
