Paper Reading AI Learner

DIFFUMA: High-Fidelity Spatio-Temporal Video Prediction via Dual-Path Mamba and Diffusion Enhancement

2025-07-09 10:51:54
Xinyu Xie, Weifeng Cao, Jun Shi, Yangyang Hu, Hui Liang, Wanyong Liang, Xiaoliang Qian

Abstract

Spatio-temporal video prediction plays a pivotal role in critical domains, ranging from weather forecasting to industrial automation. However, in high-precision industrial scenarios such as semiconductor manufacturing, the absence of specialized benchmark datasets severely hampers research on modeling and predicting complex processes. To address this challenge, we make a twofold this http URL, we construct and release the Chip Dicing Lane Dataset (CHDL), the first public temporal image dataset dedicated to the semiconductor wafer dicing process. Captured via an industrial-grade vision system, CHDL provides a much-needed and challenging benchmark for high-fidelity process modeling, defect detection, and digital twin this http URL, we propose DIFFUMA, an innovative dual-path prediction architecture specifically designed for such fine-grained dynamics. The model captures global long-range temporal context through a parallel Mamba module, while simultaneously leveraging a diffusion module, guided by temporal features, to restore and enhance fine-grained spatial details, effectively combating feature degradation. Experiments demonstrate that on our CHDL benchmark, DIFFUMA significantly outperforms existing methods, reducing the Mean Squared Error (MSE) by 39% and improving the Structural Similarity (SSIM) from 0.926 to a near-perfect 0.988. This superior performance also generalizes to natural phenomena datasets. Our work not only delivers a new state-of-the-art (SOTA) model but, more importantly, provides the community with an invaluable data resource to drive future research in industrial AI.

Abstract (translated)

时空视频预测在诸如天气预报和工业自动化等关键领域中扮演着核心角色。然而,在半导体制造等高精度的工业场景中,缺乏专门的基准数据集严重阻碍了对复杂过程建模与预测的研究进展。为解决这一挑战,我们进行了两个方面的努力:首先,构建并发布了用于半导体晶圆切割工艺的第一个公开时间图像数据集——Chip Dicing Lane 数据集(CHDL)。该数据集通过工业级视觉系统获取,提供了高保真度流程建模、缺陷检测和数字孪生所急需的且具有挑战性的基准。其次,我们提出了 DIFFUMA 架构,一种专门设计用于此类精细动态预测的创新双路径架构。DIFFUMA 通过并行的 Mamba 模块捕获全局长时序上下文信息,并同时利用由时间特征引导的扩散模块恢复和增强细粒度空间细节,从而有效对抗特征退化问题。 实验表明,在我们的 CHDL 基准测试中,DIFFUMA 显著优于现有方法,将均方误差(MSE)降低了 39%,并将结构相似性指数(SSIM)从 0.926 提升到了接近完美的 0.988。这一卓越性能也推广到自然现象数据集上。我们的工作不仅提供了新的最先进的(SOTA)模型,而且更重要的是为社区提供了一个宝贵的数据资源,以推动工业 AI 的未来研究进展。

URL

https://arxiv.org/abs/2507.06738

PDF

https://arxiv.org/pdf/2507.06738.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot