Surveillance
-
YOLOv11 Demystified: A Practical Guide to High-Performance Object Detection
2026-04-03 12:43:19Nikhileswara Rao SulakeAbstract
YOLOv11 is the latest iteration in the You Only Look Once (YOLO) series of real-time object detectors, introducing novel architectural modules to improve feature extraction and small-object detection. In this paper, we present a detailed analysis of YOLOv11, including its backbone, neck, and head components. The model key innovations, the C3K2 blocks, Spatial Pyramid Pooling - Fast (SPPF), and C2PSA (Cross Stage Partial with Spatial Attention) modules enhance spatial feature processing while preserving speed. We compare YOLOv11 performance to prior YOLO versions on standard benchmarks, highlighting improvements in mean Average Precision (mAP) and inference speed. Our results demonstrate that YOLOv11 achieves superior accuracy without sacrificing real-time capabilities, making it well-suited for applications in autonomous driving, surveillance, and video this http URL work formalizes YOLOv11 in a research context, providing a clear reference for future studies.
Abstract (translated)
YOLOv11是You Only Look Once（YOLO）系列实时目标检测器的最新迭代版本，引入了新型架构模块以提升特征提取和小目标检测能力。本文详细分析了YOLOv11，涵盖其主干网络、颈部网络和头部网络组件。模型的关键创新——C3K2模块、空间金字塔池化快速版（SPPF）以及C2PSA（带空间注意力的跨阶段部分）模块——在保持速度的同时增强了空间特征处理能力。我们将YOLOv11在标准基准测试上的性能与先前YOLO版本进行了对比，突出了平均精度均值（mAP）和推理速度的改进。结果表明，YOLOv11在无需牺牲实时性的前提下实现了更优精度，使其非常适合自动驾驶、监控和视频分析等应用。本研究将YOLOv11置于研究语境中，为未来研究提供了清晰参考。
URL
https://arxiv.org/abs/2604.03349
PDF
-
CMCC-ReID: Cross-Modality Clothing-Change Person Re-Identification
2026-04-03 07:19:30Haoxuan Xu, Hanzi Wang, Guanglin NiuAbstract
Person Re-Identification (ReID) faces severe challenges from modality discrepancy and clothing variation in long-term surveillance scenario. While existing studies have made significant progress in either Visible-Infrared ReID (VI-ReID) or Clothing-Change ReID (CC-ReID), real-world surveillance system often face both challenges simultaneously. To address this overlooked yet realistic problem, we define a new task, termed Cross-Modality Clothing-Change Re-Identification (CMCC-ReID), which targets pedestrian matching across variations in both modality and clothing. To advance research in this direction, we construct a new benchmark SYSU-CMCC, where each identity is captured in both visible and infrared domains with distinct outfits, reflecting the dual heterogeneity of long-term surveillance. To tackle CMCC-ReID, we propose a Progressive Identity Alignment Network (PIA) that progressively mitigates the issues of clothing variation and modality discrepancy. Specifically, a Dual-Branch Disentangling Learning (DBDL) module separates identity-related cues from clothing-related factors to achieve clothing-agnostic representation, and a Bi-Directional Prototype Learning (BPL) module performs intra-modality and inter-modality contrast in the embedding space to bridge the modality gap while further suppressing clothing interference. Extensive experiments on the SYSU-CMCC dataset demonstrate that PIA establishes a strong baseline for this new task and significantly outperforms existing methods.
Abstract (translated)
行人重识别（ReID）在长期监控场景中面临模态差异与衣着变化的双重严峻挑战。尽管现有研究在可见光-红外ReID（VI-ReID）或衣着变化ReID（CC-ReID）单一方面已取得显著进展，但实际监控系统往往需同时应对这两种问题。针对这一被忽视却极具现实意义的问题，我们定义了一项新任务——跨模态衣着变化重识别（CMCC-ReID），旨在实现跨模态与衣着变化双重变异下的行人匹配。为推进该方向研究，我们构建了新基准数据集SYSU-CMCC，其中每个身份均以不同衣着在可见光与红外双模态下采集，以反映长期监控中的双重异构性。为应对CMCC-ReID挑战，我们提出渐进式身份对齐网络（PIA），通过逐步缓解衣着变化与模态差异问题。具体而言，双分支解耦学习（DBDL）模块将身份相关特征与衣着相关因素分离，以获得与衣着无关的表征；双向原型学习（BPL）模块在嵌入空间中进行模态内与模态间对比，在弥合模态差距的同时进一步抑制衣着干扰。在SYSU-CMCC数据集上的大量实验表明，PIA为该新任务建立了强基线，并显著超越现有方法。
URL
https://arxiv.org/abs/2604.02808
PDF
-
IndoorCrowd: A Multi-Scene Dataset for Human Detection, Segmentation, and Tracking with an Automated Annotation Pipeline
2026-04-02 13:38:31Sebastian-Ion Nae, Radu Moldoveanu, Alexandra Stefania Ghita, Adina Magda FloreaAbstract
Understanding human behaviour in crowded indoor environments is central to surveillance, smart buildings, and human-robot interaction, yet existing datasets rarely capture real-world indoor complexity at scale. We introduce IndoorCrowd, a multi-scene dataset for indoor human detection, instance segmentation, and multi-object tracking, collected across four campus locations (ACS-EC, ACS-EG, IE-Central, R-Central). It comprises $31$ videos ($9{,}913$ frames at $5$fps) with human-verified, per-instance segmentation masks. A $620$-frame control subset benchmarks three foundation-model auto-annotators: SAM3, GroundingSAM, and EfficientGroundingSAM, against human labels using Cohen's $\kappa$, AP, precision, recall, and mask IoU. A further $2{,}552$-frame subset supports multi-object tracking with continuous identity tracks in MOTChallenge format. We establish detection, segmentation, and tracking baselines using YOLOv8n, YOLOv26n, and RT-DETR-L paired with ByteTrack, BoT-SORT, and OC-SORT. Per-scene analysis reveals substantial difficulty variation driven by crowd density, scale, and occlusion: ACS-EC, with $79.3\%$ dense frames and a mean instance scale of $60.8$px, is the most challenging scene. The project page is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2604.02032
PDF
-
FTPFusion: Frequency-Aware Infrared and Visible Video Fusion with Temporal Perturbation
2026-04-02 11:08:14Xilai Li, Chusheng Fang, Xiaosong LiAbstract
Infrared and visible video fusion plays a critical role in intelligent surveillance and low-light monitoring. However, maintaining temporal stability while preserving spatial detail remains a fundamental challenge. Existing methods either focus on frame-wise enhancement with limited temporal modeling or rely on heavy spatio-temporal aggregation that often sacrifices high-frequency details. In this paper, we propose FTPFusion, a frequency-aware infrared and visible video fusion method based on temporal perturbation and sparse cross-modal interaction. Specifically, FTPFusion decomposes the feature representations into high-frequency and low-frequency components for collaborative modeling. The high-frequency branch performs sparse cross-modal spatio-temporal interaction to capture motion-related context and complementary details. The low-frequency branch introduces a temporal perturbation strategy to enhance robustness against complex video variations, such as flickering, jitter, and local misalignment. Furthermore, we design an offset-aware temporal consistency constraint to explicitly stabilize cross-frame representations under temporal disturbances. Extensive experiments on multiple public benchmarks demonstrate that FTPFusion consistently outperforms state-of-the-art methods across multiple metrics in both spatial fidelity and temporal consistency. The source code will be available at this https URL.
Abstract (translated)
红外与可见光视频融合在智能监控与低照度监测中至关重要。然而，如何在保持空间细节的同时维持时间稳定性仍是一项根本挑战。现有方法要么侧重于帧级增强而时间建模能力有限，要么依赖沉重的时空聚合机制，往往牺牲高频细节。本文提出FTPFusion，一种基于时间扰动与稀疏跨模态交互的频域感知红外与可见光视频融合方法。具体而言，FTPFusion将特征表示分解为高频与低频分量进行协同建模：高频分支执行稀疏跨模态时空交互以捕获运动相关上下文与互补细节；低频分支引入时间扰动策略以增强对闪烁、抖动及局部错位等复杂视频变化的鲁棒性。此外，我们设计了偏移感知的时间一致性约束，以在时间扰动下显式稳定跨帧表示。在多个公开基准数据集上的大量实验表明，FTPFusion在空间保真度与时间一致性多项指标上均持续优于最新方法。源代码将在以下网址公开。
URL
https://arxiv.org/abs/2604.01900
PDF
-
Dense Point-to-Mask Optimization with Reinforced Point Selection for Crowd Instance Segmentation
2026-04-02 08:02:49Hongru Chen, Jiyang Huang, Jia Wan, Antoni B. ChanAbstract
Crowd instance segmentation is a crucial task with a wide range of applications, including surveillance and transportation. Currently, point labels are common in crowd datasets, while region labels (e.g., boxes) are rare and inaccurate. The masks obtained through segmentation help to improve the accuracy of region labels and resolve the correspondence between individual location coordinates and crowd density maps. However, directly applying currently popular large foundation models such as SAM does not yield ideal results in dense crowds. To this end, we first propose Dense Point-to-Mask Optimization (DPMO), which integrates SAM with the Nearest Neighbor Exclusive Circle (NNEC) constraint to generate dense instance segmentation from point annotations. With DPMO and manual correction, we obtain mask annotations from the existing point annotations for traditional crowd datasets. Then, to predict instance segmentation in dense crowds, we propose a Reinforced Point Selection (RPS) framework trained with Group Relative Policy Optimization (GRPO), which selects the best predicted point from a sampling of the initial point prediction. Through extensive experiments, we achieve state-of-the-art crowd instance segmentation performance on ShanghaiTech, UCF-QNRF, JHU-CROWD++, and NWPU-Crowd datasets. Furthermore, we design new loss functions supervised by masks that boost counting performance across different models, demonstrating the significant role of mask annotations in enhancing counting accuracy.
Abstract (translated)
人群实例分割是一项具有广泛应用（包括监控和交通领域）的关键任务。目前，人群数据集中普遍采用点标注，而区域标注（如边界框）较为罕见且不够精确。通过分割获得的掩码有助于提升区域标注的准确性，并解决个体位置坐标与人群密度图之间的对应关系。然而，直接将当前流行的大型基础模型（如SAM）应用于密集人群场景时无法获得理想结果。为此，我们首先提出密集点-掩码优化方法（DPMO），该方法将SAM与最近邻互斥圆（NNEC）约束相结合，从点标注生成密集实例分割掩码。借助DPMO和人工修正，我们为传统人群数据集从现有点标注中获得了掩码标注。随后，为预测密集人群中的实例分割，我们提出了基于组相对策略优化（GRPO）训练的强化点选择（RPS）框架，该框架从初始点预测的采样结果中选择最优预测点。通过大量实验，我们在ShanghaiTech、UCF-QNRF、JHU-CROWD++和NWPU-Crowd数据集上达到了最先进的人群实例分割性能。此外，我们设计了由掩码监督的新损失函数，可提升不同模型的计数性能，证明了掩码标注在增强计数准确性方面的重要作用。
URL
https://arxiv.org/abs/2604.01742
PDF
-
Robust Multi-Agent Reinforcement Learning for Small UAS Separation Assurance under GPS Degradation and Spoofing
2026-03-30 18:26:59Alex Zongo, Filippos Fotiadis, Ufuk Topcu, Peng WeiAbstract
We address robust separation assurance for small Unmanned Aircraft Systems (sUAS) under GPS degradation and spoofing via Multi-Agent Reinforcement Learning (MARL). In cooperative surveillance, each aircraft (or agent) broadcasts its GPS-derived position; when such position broadcasts are corrupted, the entire observed air traffic state becomes unreliable. We cast this state observation corruption as a zero-sum game between the agents and an adversary: with probability R, the adversary perturbs the observed state to maximally degrade each agent's safety performance. We derive a closed-form expression for this adversarial perturbation, bypassing adversarial training entirely and enabling linear-time evaluation in the state dimension. We show that this expression approximates the true worst-case adversarial perturbation with second-order accuracy. We further bound the safety performance gap between clean and corrupted observations, showing that it degrades at most linearly with the corruption probability under Kullback-Leibler regularization. Finally, we integrate the closed-form adversarial policy into a MARL policy gradient algorithm to obtain a robust counter-policy for the agents. In a high-density sUAS simulation, we observe near-zero collision rates under corruption levels up to 35%, outperforming a baseline policy trained without adversarial perturbations.
Abstract (translated)
本文通过多智能体强化学习（MARL）解决了小型无人机系统（sUAS）在GPS降级与欺骗条件下的鲁棒性防撞保障问题。在协同监视场景中，每架无人机（或智能体）广播其GPS定位信息；当此类位置广播受到污染时，整个观测到的空中交通状态将变得不可靠。我们将这种状态观测污染建模为智能体与攻击者之间的零和博弈：以概率R，攻击者会扰动观测状态以最大程度降低各智能体的安全性能。我们推导出该对抗性扰动的闭式表达，完全规避对抗训练，并实现状态维度上的线性时间复杂度评估。证明该表达式能以二阶精度逼近真实最坏情况对抗扰动。进一步，我们量化了清洁观测与污染观测之间的安全性能差距，证明在库尔贝克-莱布勒正则化下，该差距最多随污染概率线性退化。最终，我们将闭式对抗策略集成至MARL策略梯度算法，为智能体生成鲁棒对抗策略。在高密度小型无人机系统仿真中，即使在35%污染水平下，该方法仍实现近零碰撞率，优于未进行对抗训练的基准策略。
URL
https://arxiv.org/abs/2603.28900
PDF
-
Critic-Free Deep Reinforcement Learning for Maritime Coverage Path Planning on Irregular Hexagonal Grids
2026-03-30 12:56:38Carlos S. Sep\'ulveda, Gonzalo A. RuzAbstract
Maritime surveillance missions, such as search and rescue and environmental monitoring, rely on the efficient allocation of sensing assets over vast and geometrically complex areas. Traditional Coverage Path Planning (CPP) approaches depend on decomposition techniques that struggle with irregular coastlines, islands, and exclusion zones, or require computationally expensive re-planning for every instance. We propose a Deep Reinforcement Learning (DRL) framework to solve CPP on hexagonal grid representations of irregular maritime areas. Unlike conventional methods, we formulate the problem as a neural combinatorial optimization task where a Transformer-based pointer policy autoregressively constructs coverage tours. To overcome the instability of value estimation in long-horizon routing problems, we implement a critic-free Group-Relative Policy Optimization (GRPO) scheme. This method estimates advantages through within-instance comparisons of sampled trajectories rather than relying on a value function. Experiments on 1,000 unseen synthetic maritime environments demonstrate that a trained policy achieves a 99.0% Hamiltonian success rate, more than double the best heuristic (46.0%), while producing paths 7% shorter and with 24% fewer heading changes than the closest baseline. All three inference modes (greedy, stochastic sampling, and sampling with 2-opt refinement) operate under 50~ms per instance on a laptop GPU, confirming feasibility for real-time on-board deployment.
Abstract (translated)
海洋监视任务（如搜救和环境监测）依赖于在广阔且几何结构复杂的区域上高效分配感知资源。传统的覆盖路径规划（CPP）方法依赖于分解技术，但这些技术难以处理不规则的海岸线、岛屿和禁飞区，或需要对每个实例进行计算成本高昂的重新规划。我们提出了一种深度强化学习（DRL）框架，用于解决基于不规则海域六边形网格表示的CPP问题。与常规方法不同，我们将问题 formulate 为一个神经组合优化任务，其中基于Transformer的指针策略通过自回归方式构建覆盖路径。为了克服长时域路径规划中价值估计的不稳定性，我们 implement 了一种无评判器的组相对策略优化（GRPO）方案。该方法通过实例内采样轨迹的比较来估计优势，而非依赖价值函数。在1000个未见过的合成海洋环境上的实验表明，训练后的策略实现了99.0%的哈密顿成功率，是最优启发式算法（46.0%）的两倍以上，同时生成的路径比最近基线方法短7%，航向变更减少24%。所有三种推理模式（贪婪模式、随机采样及结合2-opt优化的采样）在笔记本电脑GPU上每实例耗时均低于50毫秒，证实了其在实时机载部署中的可行性。
URL
https://arxiv.org/abs/2603.28385
PDF
-
From Pixels to Privacy: Temporally Consistent Video Anonymization via Token Pruning for Privacy Preserving Action Recognition
2026-03-27 11:57:51Nazia Aslam, Abhisek Ray, Joakim Bruslund Haurum, Lukas Esterle, Kamal NasrollahiAbstract
Recent advances in large-scale video models have significantly improved video understanding across domains such as surveillance, healthcare, and entertainment. However, these models also amplify privacy risks by encoding sensitive attributes, including facial identity, race, and gender. While image anonymization has been extensively studied, video anonymization remains relatively underexplored, even though modern video models can leverage spatiotemporal motion patterns as biometric identifiers. To address this challenge, we propose a novel attention-driven spatiotemporal video anonymization framework based on systematic disentanglement of utility and privacy features. Our key insight is that attention mechanisms in Vision Transformers (ViTs) can be explicitly structured to separate action-relevant information from privacy-sensitive content. Building on this insight, we introduce two task-specific classification tokens, an action CLS token and a privacy CLS token, that learn complementary representations within a shared Transformer backbone. We contrast their attention distributions to compute a utility-privacy score for each spatiotemporal tubelet, and keep the top-k tubelets with the highest scores. This selectively prunes tubelets dominated by privacy cues while preserving those most critical for action recognition. Extensive experiments demonstrate that our approach maintains action recognition performance comparable to models trained on raw videos, while substantially reducing privacy leakage. These results indicate that attention-driven spatiotemporal pruning offers an effective and principled solution for privacy-preserving video analytics.
Abstract (translated)
大规模视频模型的最新进展显著提升了监控、医疗和娱乐等领域的视频理解能力。然而，这些模型也通过编码面部身份、种族和性别等敏感属性而放大了隐私风险。尽管图像匿名化技术已得到广泛研究，视频匿名化仍相对缺乏探索——而现代视频模型能够利用时空运动模式作为生物特征标识符。为应对这一挑战，我们提出了一种基于效用与隐私特征系统性解耦的、新型注意力驱动时空视频匿名化框架。我们的核心见解是：视觉Transformer（ViTs）中的注意力机制可被显式结构化，以分离与动作相关的信息与隐私敏感内容。基于此见解，我们引入了两个任务特定的分类标记——动作CLS标记与隐私CLS标记，使它们能在共享Transformer主干网络中学习互补表征。我们通过对比它们的注意力分布，为每个时空小片段计算效用-隐私分数，并保留分数最高的前k个小片段。这种方法能选择性地剪除由隐私线索主导的小片段，同时保留对动作识别最关键的部分。大量实验表明，我们的方法在保持与原始视频训练模型相当的动作识别性能的同时，显著降低了隐私泄露风险。这些结果表明，注意力驱动的时空剪枝为隐私保护视频分析提供了有效且 principled 的解决方案。
URL
https://arxiv.org/abs/2603.26336
PDF
-
Provably Contractive and High-Quality Denoisers for Convergent Restoration
2026-03-27 08:35:55Shubhi Shukla, Pravin NairAbstract
Image restoration, the recovery of clean images from degraded measurements, has applications in various domains like surveillance, defense, and medical imaging. Despite achieving state-of-the-art (SOTA) restoration performance, existing convolutional and attention-based networks lack stability guarantees under minor shifts in input, exposing a robustness accuracy trade-off. We develop provably contractive (global Lipschitz $< 1$) denoiser networks that considerably reduce this gap. Our design composes proximal layers obtained from unfolding techniques, with Lipschitz-controlled convolutional refinements. By contractivity, our denoiser guarantees that input perturbations of strength $\|\delta\|\le\varepsilon$ induce at most $\varepsilon$ change at the output, while strong baselines such as DnCNN and Restormer can exhibit larger deviations under the same perturbations. On image denoising, the proposed model is competitive with unconstrained SOTA denoisers, reporting the tightest gap for a provably 1-Lipschitz model and establishing that such gaps are indeed achievable by contractive denoisers. Moreover, the proposed denoisers act as strong regularizers for image restoration that provably effect convergence in Plug-and-Play algorithms. Our results show that enforcing strict Lipschitz control does not inherently degrade output quality, challenging a common assumption in the literature and moving the field toward verifiable and stable vision models. Codes and pretrained models are available at this https URL
Abstract (translated)
图像恢复旨在从退化观测中重建清晰图像，在监控、国防和医学成像等领域具有广泛应用。尽管现有基于卷积和注意力的网络已实现最先进的恢复性能，但它们在输入发生微小偏移时缺乏稳定性保证，暴露出鲁棒性与精度之间的权衡。我们开发了可证明具有压缩性（全局Lipschitz常数<1）的去噪网络，显著缩小了这一差距。我们的设计通过展开技术获得近端层，并结合Lipschitz约束的卷积精修进行组合。基于压缩性，我们的去噪器保证：当输入扰动强度满足$\|\delta\|\le\varepsilon$时，输出变化不超过$\varepsilon$；而DnCNN和Restormer等强基线模型在相同扰动下可能产生更大偏差。在图像去噪任务中，所提模型与无约束的最先进去噪器具有竞争力，为可证明1-Lipschitz模型实现了最紧凑的精度差距，证实了压缩性去噪器确实能够达到此类性能。此外，提出的去噪器作为图像恢复的强正则器，可证明能促进即插即用算法的收敛。我们的研究表明，施加严格的Lipschitz约束并不会固有降低输出质量，这挑战了文献中的常见假设，推动领域向可验证、稳定的视觉模型发展。代码与预训练模型见以下网址：https URL
URL
https://arxiv.org/abs/2603.26168
PDF
-
Experimental study on surveillance video-based indoor occupancy measurement with occupant-centric control
2026-03-27 05:19:42Irfan Qaisar, Kailai Sun, Qingshan Jia, Qianchuan ZhaoAbstract
Accurate occupancy information is essential for closed-loop occupant-centric control (OCC) in smart buildings. However, existing vision-based occupancy measurement methods often struggle to provide stable and accurate measurements in real indoor environments, and their implications for downstream HVAC control remain insufficiently studied. To achieve Net Zero emissions by 2050, this paper presents an experimental study of large language models (LLMs)-enhanced vision-based indoor occupancy measurement and its impact on OCC-enabled HVAC operation. Detection-only, tracking-based, and LLM-based refinement pipelines are compared under identical conditions using real surveillance data collected from a research laboratory in China, with frame-level manual ground-truth annotations. Results show that tracking-based methods improve temporal stability over detection-only measurement, while LLM-based refinement further improves occupancy measurement performance and reduces false unoccupied prediction. The best-performing pipeline, YOLOv8+DeepSeek, achieves an accuracy of 0.8824 and an F1-score of 0.9320. This pipeline is then integrated into an HVAC supervisory model predictive control framework in OpenStudio-EnergyPlus. Experimental results demonstrate that the proposed framework can support more efficient OCC operation, achieving a substantial HVAC energy-saving potential of 17.94%. These findings provide an effective methodology and practical foundation for future research in AI-enhanced smart building operations.
Abstract (translated)
准确的 occupancy 信息对于智能建筑中以 occupant 为中心的闭环控制（OCC）至关重要。然而，现有的基于视觉的 occupancy 测量方法在实际室内环境中往往难以提供稳定且精确的测量结果，且其对下游 HVAC 控制的影响尚未得到充分研究。为实现 2050 年净零排放目标，本文提出了一项实验研究，探讨大语言模型（LLMs）增强的基于视觉的室内 occupancy 测量方法及其对支持 OCC 的 HVAC 运行的影响。研究在相同条件下，使用来自中国某研究实验室的真实监控数据（包含帧级人工标注的真值）对比了仅检测、基于跟踪以及基于 LLM 优化的三种流程。结果表明，基于跟踪的方法相较于仅检测测量提升了时间稳定性，而基于 LLM 的优化进一步提高了 occupancy 测量性能并减少了误报“无人”的情况。表现最佳的流程 YOLOv8+DeepSeek 达到了 0.8824 的准确率和 0.9320 的 F1 分数。随后，将该流程集成至 OpenStudio-EnergyPlus 中的 HVAC 监督模型预测控制（MPC）框架内。实验结果表明，所提出的框架能够支持更高效的 OCC 运行，实现了高达 17.94% 的 HVAC 节能潜力。这些发现为未来 AI 增强型智能建筑运行研究提供了有效的方法论和实用基础。
URL
https://arxiv.org/abs/2603.26081
PDF
-
Face2Parts: Exploring Coarse-to-Fine Inter-Regional Facial Dependencies for Generalized Deepfake Detection
2026-03-27 03:12:24Kutub Uddin, Nusrat Tasnim, Byung Tae OhAbstract
Multimedia data, particularly images and videos, is integral to various applications, including surveillance, visual interaction, biometrics, evidence gathering, and advertising. However, amateur or skilled counterfeiters can simulate them to create deepfakes, often for slanderous motives. To address this challenge, several forensic methods have been developed to ensure the authenticity of the content. The effectiveness of these methods depends on their focus, with challenges arising from the diverse nature of manipulations. In this article, we analyze existing forensic methods and observe that each method has unique strengths in detecting deepfake traces by focusing on specific facial regions, such as the frame, face, lips, eyes, or nose. Considering these insights, we propose a novel hybrid approach called Face2Parts based on hierarchical feature representation ($HFR$) that takes advantage of coarse-to-fine information to improve deepfake detection. The proposed method involves extracting features from the frame, face, and key facial regions (i.e., lips, eyes, and nose) separately to explore the coarse-to-fine relationships. This approach enables us to capture inter-dependencies among facial regions using a channel-attention mechanism and deep triplet learning. We evaluated the proposed method on benchmark deepfake datasets in both intra-, inter-dataset, and inter-manipulation settings. The proposed method achieves an average AUC of 98.42\% on FF++, 79.80\% on CDF1, 85.34\% on CDF2, 89.41\% on DFD, 84.07\% on DFDC, 95.62\% on DTIM, 80.76\% on PDD, and 100\% on WLDR, respectively. The results demonstrate that our approach generalizes effectively and achieves promising performance to outperform the existing methods.
Abstract (translated)
多媒体数据（尤其是图像和视频）在监控、视觉交互、生物识别、证据收集和广告等多种应用中不可或缺。然而，业余或专业伪造者可能模拟这些数据制作深度伪造内容，往往出于诽谤目的。为应对这一挑战，已开发出多种取证方法以确保内容真实性。这些方法的有效性取决于其检测重点，而伪造手段的多样性也带来了相应挑战。本文分析了现有取证方法，发现每种方法通过聚焦特定面部区域（如整体画面、面部、嘴唇、眼睛或鼻子）在检测深度伪造痕迹方面具有独特优势。基于这些见解，我们提出了一种名为Face2Parts的新型混合方法，该方法采用分层特征表示（HFR），利用由粗到细的信息提升深度伪造检测效果。该方法分别从画面整体、面部及关键面部区域（即嘴唇、眼睛和鼻子）提取特征，以探索由粗到细的特征关联。此方法通过通道注意力机制和深度三元组学习，捕捉面部区域间的相互依赖关系。我们在多个基准深度伪造数据集上评估了该方法，涵盖域内、跨域及跨伪造手法场景。实验结果显示，该方法在FF++数据集上平均AUC达98.42%，CDF1数据集上为79.80%，CDF2数据集上为85.34%，DFD数据集上为89.41%，DFDC数据集上为84.07%，DTIM数据集上为95.62%，PDD数据集上为80.76%，WLDR数据集上达到100%。结果表明，该方法具有良好泛化能力，性能优于现有方法。
URL
https://arxiv.org/abs/2603.26036
PDF
-
LEMMA: Laplacian pyramids for Efficient Marine SeMAntic Segmentation
2026-03-26 17:38:16Ishaan Gakhar, Laven Srivastava, Sankarshanaa Sagaram, Aditya Kasliwal, Ujjwal VermaAbstract
Semantic segmentation in marine environments is crucial for the autonomous navigation of unmanned surface vessels (USVs) and coastal Earth Observation events such as oil spills. However, existing methods, often relying on deep CNNs and transformer-based architectures, face challenges in deployment due to their high computational costs and resource-intensive nature. These limitations hinder the practicality of real-time, low-cost applications in real-world marine settings. To address this, we propose LEMMA, a lightweight semantic segmentation model designed specifically for accurate remote sensing segmentation under resource constraints. The proposed architecture leverages Laplacian Pyramids to enhance edge recognition, a critical component for effective feature extraction in complex marine environments for disaster response, environmental surveillance, and coastal monitoring. By integrating edge information early in the feature extraction process, LEMMA eliminates the need for computationally expensive feature map computations in deeper network layers, drastically reducing model size, complexity and inference time. LEMMA demonstrates state-of-the-art performance across datasets captured from diverse platforms while reducing trainable parameters and computational requirements by up to 71x, GFLOPs by up to 88.5\%, and inference time by up to 84.65\%, as compared to existing models. Experimental results highlight its effectiveness and real-world applicability, including 93.42\% IoU on the Oil Spill dataset and 98.97\% mIoU on Mastr1325.
Abstract (translated)
海洋环境中的语义分割对于无人水面艇（USV）的自主导航以及海岸带地球观测事件（如油污泄漏）至关重要。然而，现有方法通常依赖深度卷积神经网络（CNN）和基于Transformer的架构，因其高昂的计算成本和资源密集型特性，在部署时面临挑战。这些限制阻碍了其在真实海洋场景中实现实时、低成本应用的实用性。为解决此问题，我们提出了LEMMA——一个专门针对资源受限条件下精确遥感分割而设计的轻量级语义分割模型。该架构利用拉普拉斯金字塔增强边缘识别能力，这是复杂海洋环境中灾难响应、环境监测和海岸监控有效特征提取的关键环节。通过在特征提取早期整合边缘信息，LEMMA消除了深层网络层中对计算量大的特征图运算的需求，显著降低了模型规模、复杂度和推理时间。与现有模型相比，LEMMA在来自不同平台采集的数据集上展现出最先进性能，同时将可训练参数和计算需求最多减少71倍，GFLOPs最多降低88.5%，推理时间最多缩短84.65%。实验结果凸显了其有效性和实际应用价值，在油污数据集上达到93.42%的IoU，在Mastr1325数据集上达到98.97%的mIoU。
URL
https://arxiv.org/abs/2603.25689
PDF
-
GridVAD: Open-Set Video Anomaly Detection via Spatial Reasoning over Stratified Frame Grids
2026-03-26 14:08:41Mohamed Eltahir, Ahmed O. Ibrahim, Obada Siralkhatim, Tabarak Abdallah, Sondos MohamedAbstract
Vision-Language Models (VLMs) are powerful open-set reasoners, yet their direct use as anomaly detectors in video surveillance is fragile: without calibrated anomaly priors, they alternate between missed detections and hallucinated false alarms. We argue the problem is not the VLM itself but how it is used. VLMs should function as anomaly proposers, generating open-set candidate descriptions that are then grounded and tracked by purpose-built spatial and temporal modules. We instantiate this propose-ground-propagate principle in GridVAD, a training-free pipeline that produces pixel-level anomaly masks without any domain-specific training. A VLM reasons over stratified grid representations of video clips to generate natural-language anomaly proposals. Self-Consistency Consolidation (SCC) filters hallucinations by retaining only proposals that recur across multiple independent samplings. Grounding DINO anchors each surviving proposal to a bounding box, and SAM2 propagates it as a dense mask through the anomaly interval. The per-clip VLM budget is fixed at M+1 calls regardless of video length, where M can be set according to the proposals needed. On UCSD Ped2, GridVAD achieves the highest Pixel-AUROC (77.59) among all compared methods, surpassing even the partially fine-tuned TAO (75.11) and outperforms other zero-shot approaches on object-level RBDC by over 5x. Ablations reveal that SCC provides a controllable precision-recall tradeoff: filtering improves all pixel level metrics at a modest cost in object-level recall. Efficiency experiments show GridVAD is 2.7x more call-efficient than uniform per-frame VLM querying while additionally producing dense segmentation this http URL and qualitative video results are available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2603.25467
PDF
-
SDD-YOLO: A Small-Target Detection Framework for Ground-to-Air Anti-UAV Surveillance with Edge-Efficient Deployment
2026-03-26 09:17:39Pengyu Chen, Haotian Sa, Yiwei Hu, Yuhan Cheng, Junbo WangAbstract
Detecting small unmanned aerial vehicles (UAVs) from a ground-to-air (G2A) perspective presents significant challenges, including extremely low pixel occupancy, cluttered aerial backgrounds, and strict real-time constraints. Existing YOLO-based detectors are primarily optimized for general object detection and often lack adequate feature resolution for sub-pixel targets, while introducing complexities during deployment. In this paper, we propose SDD-YOLO, a small-target detection framework tailored for G2A anti-UAV surveillance. To capture fine-grained spatial details critical for micro-targets, SDD-YOLO introduces a P2 high-resolution detection head operating at 4 times downsampling. Furthermore, we integrate the recent architectural advancements from YOLO26, including a DFL-free, NMS-free architecture for streamlined inference, and the MuSGD hybrid training strategy with ProgLoss and STAL, which substantially mitigates gradient oscillation on sparse small-target signals. To support our evaluation, we construct DroneSOD-30K, a large-scale G2A dataset comprising approximately 30,000 annotated images covering diverse meteorological conditions. Experiments demonstrate that SDD-YOLO-n achieves a mAP@0.5 of 86.0% on DroneSOD-30K, surpassing the YOLOv5n baseline by 7.8 percentage points. Extensive inference analysis shows our model attains 226 FPS on an NVIDIA RTX 5090 and 35 FPS on an Intel Xeon CPU, demonstrating exceptional efficiency for future edge deployment.
Abstract (translated)
从地对空（G2A）视角检测小型无人机（UAV）面临显著挑战，包括极低的像素占比、杂乱的空中背景以及严格的实时性要求。现有基于YOLO的检测器主要针对通用目标检测进行优化，往往缺乏对亚像素目标的足够特征分辨率，同时在部署时引入复杂性。本文提出SDD-YOLO，一个专为G2A反无人机监控设计的小目标检测框架。为捕捉微目标所需的关键细粒度空间细节，SDD-YOLO引入了在4倍下采样率下运行的P2高分辨率检测头。此外，我们整合了YOLO26的最新架构进展，包括无DFL、无NMS的简化推理架构，以及结合渐进损失（ProgLoss）和空间时间自适应学习（STAL）的MuSGD混合训练策略，该策略显著缓解了稀疏小目标信号上的梯度振荡问题。为支持评估，我们构建了DroneSOD-30K数据集，这是一个包含约3万张标注图像的大规模G2A数据集，覆盖多样气象条件。实验表明，SDD-YOLO-n在DroneSOD-30K上达到86.0%的mAP@0.5，比YOLOv5n基线高出7.8个百分点。广泛的推理分析显示，我们的模型在NVIDIA RTX 5090上达到226 FPS，在Intel Xeon CPU上达到35 FPS，展现了面向未来边缘部署的卓越效率。
URL
https://arxiv.org/abs/2603.25218
PDF
-
TIGFlow-GRPO: Trajectory Forecasting via Interaction-Aware Flow Matching and Reward-Driven Optimization
2026-03-26 01:59:10Xuepeng Jing, Wenhuan Lu, Hao Meng, Zhizhi Yu, Jianguo WeiAbstract
Human trajectory forecasting is important for intelligent multimedia systems operating in visually complex environments, such as autonomous driving and crowd surveillance. Although Conditional Flow Matching (CFM) has shown strong ability in modeling trajectory distributions from spatio-temporal observations, existing approaches still focus primarily on supervised fitting, which may leave social norms and scene constraints insufficiently reflected in generated trajectories. To address this issue, we propose TIGFlow-GRPO, a two-stage generative framework that aligns flow-based trajectory generation with behavioral rules. In the first stage, we build a CFM-based predictor with a Trajectory-Interaction-Graph (TIG) module to model fine-grained visual-spatial interactions and strengthen context encoding. This stage captures both agent-agent and agent-scene relations more effectively, providing more informative conditional features for subsequent alignment. In the second stage, we perform Flow-GRPO post-training,where deterministic flow rollout is reformulated as stochastic ODE-to-SDE sampling to enable trajectory exploration, and a composite reward combines view-aware social compliance with map-aware physical feasibility. By evaluating trajectories explored through SDE rollout, GRPO progressively steers multimodal predictions toward behaviorally plausible futures. Experiments on the ETH/UCY and SDD datasets show that TIGFlow-GRPO improves forecasting accuracy and long-horizon stability while generating trajectories that are more socially compliant and physically feasible. These results suggest that the proposed framework provides an effective way to connect flow-based trajectory modeling with behavior-aware alignment in dynamic multimedia environments.
Abstract (translated)
人类轨迹预测对于在视觉复杂环境中运行的智能多媒体系统（如自动驾驶和人群监控）至关重要。尽管条件流匹配（CFM）已展现出从时空观测中建模轨迹分布的强大能力，但现有方法仍主要侧重于监督拟合，可能导致生成轨迹未能充分反映社会规范与场景约束。针对此问题，我们提出TIGFlow-GRPO——一个将基于流的轨迹生成与行为规则对齐的两阶段生成框架。第一阶段构建基于CFM的预测器，并引入轨迹交互图（TIG）模块以建模细粒度视觉-空间交互并强化上下文编码，该阶段能更有效地捕捉智能体间及智能体与场景的关系，为后续对齐提供信息量更丰富的条件特征。第二阶段执行流式GRPO后训练，其中将确定性流展开重构为随机ODE到SDE采样以实现轨迹探索，并通过组合奖励将视角感知的社会合规性与地图感知的物理可行性相结合。通过评估SDE展开探索的轨迹，GRPO逐步将多模态预测引导至行为合理的未来。在ETH/UCY和SDD数据集上的实验表明，TIGFlow-GRPO在提升预测精度与长时稳定性的同时，能生成更具社会合规性与物理可行性的轨迹。这些结果说明，所提框架为动态多媒体环境中基于流的轨迹建模与行为感知对齐提供了有效连接方式。
URL
https://arxiv.org/abs/2603.24936
PDF
-
AirSimAG: A High-Fidelity Simulation Platform for Air-Ground Collaborative Robotics
2026-03-24 11:17:53Yangjie Cui, Xin Dong, Boyang Gao, Jinwu Xiang, Daochun Li, Zhan TuAbstract
As spatial intelligence continues to evolve, heterogeneous multi-agent systems-particularly the collaboration between Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs), have demonstrated strong potential in complex applications such as search and rescue, urban surveillance, and environmental monitoring. However, existing simulation platforms are primarily designed for single-agent dynamics and lack dedicated frameworks for interactive air-ground collaborative simulation. In this paper, we present AirsimAG, a high-fidelity air-ground collaborative simulation platform built upon an extensively customized AirSim framework. The platform enables synchronized multi-agent simulation and supports heterogeneous sensing and control interfaces for UAV-UGV systems. To demonstrate its capabilities, we design a set of representative air-ground collaborative tasks, including mapping, planning, tracking, formation, and exploration. We further provide quantitative analyses based on these tasks to illustrate the platform effectiveness in supporting multi-agent coordination and cross-modal data consistency. The AirsimAG simulation platform is publicly available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2603.23079
PDF
-
ForeSea: AI Forensic Search with Multi-modal Queries for Video Surveillance
2026-03-24 07:15:28Hyojin Park, Yi Li, Janghoon Cho, Sungha Choi, Jungsoo Lee, Taotao Jing, Shuai Zhang, Munawar Hayat, Dashan Gao, Ning Bi, Fatih PorikliAbstract
Despite decades of work, surveillance still struggles to find specific targets across long, multi-camera video. Prior methods -- tracking pipelines, CLIP based models, and VideoRAG -- require heavy manual filtering, capture only shallow attributes, and fail at temporal reasoning. Real-world searches are inherently multimodal (e.g., "When does this person join the fight?" with the person's image), yet this setting remains underexplored. Also, there are no proper benchmarks to evaluate those setting - asking video with multimodal queries. To address this gap, we introduce ForeSeaQA, a new benchmark specifically designed for video QA with image-and-text queries and timestamped annotations of key events. The dataset consists of long-horizon surveillance footage paired with diverse multimodal questions, enabling systematic evaluation of retrieval, temporal grounding, and multimodal reasoning in realistic forensic conditions. Not limited to this benchmark, we propose ForeSea, an AI forensic search system with a 3-stage, plug-and-play pipeline. (1) A tracking module filters irrelevant footage; (2) a multimodal embedding module indexes the remaining clips; and (3) during inference, the system retrieves top-K candidate clips for a Video Large Language Model (VideoLLM) to answer queries and localize events. On ForeSeaQA, ForeSea improves accuracy by 3.5% and temporal IoU by 11.0 over prior VideoRAG models. To our knowledge, ForeSeaQA is the first benchmark to support complex multimodal queries with precise temporal grounding, and ForeSea is the first VideoRAG system built to excel in this setting.
Abstract (translated)
尽管历经数十载研究，监控系统在长时段、多摄像头视频中定位特定目标仍面临挑战。先前的方案——包括追踪流水线、基于CLIP的模型及VideoRAG——均需大量人工筛选，仅能捕捉表层特征，且无法进行时序推理。现实中的搜索本质上是多模态的（例如：结合人物图像询问"此人何时加入斗殴？"），但此类场景尚未得到充分探索。此外，目前缺乏针对多模态查询视频检索场景的合适基准测试。为填补这一空白，我们推出了ForeSeaQA——首个专为图像-文本联合查询视频问答设计的新基准，包含带时间戳的关键事件标注。该数据集包含长时段监控视频与多样化多模态问题，可在真实取证条件下系统评估检索、时序定位与多模态推理能力。除基准外，我们还提出了ForeSea——一个三阶段即插即用式AI取证搜索系统：(1) 追踪模块过滤无关片段；(2) 多模态嵌入模块对剩余片段建立索引；(3) 推理阶段系统检索Top-K候选片段，交由视频大语言模型（VideoLLM）回答问题并定位事件。在ForeSeaQA上，ForeSea相比先前VideoRAG模型将准确率提升3.5%，时序交并比提升11.0。据我们所知，ForeSeaQA是首个支持复杂多模态查询与精确时序定位的基准，而ForeSea则是首个专为该场景优化的VideoRAG系统。
URL
https://arxiv.org/abs/2603.22872
PDF
-
Detecting Corporate AI-Washing via Cross-Modal Semantic Inconsistency Learning
2026-03-24 01:30:36Zhanjie Wen, Jingqiao GuoAbstract
Corporate AI-washing-the strategic misrepresentation of AI capabilities via exaggerated or fabricated cross-channel disclosures-has emerged as a systemic threat to capital market information integrity with the widespread adoption of generative AI. Existing detection methods rely on single-modal text frequency analysis, suffering from vulnerability to adversarial reformulation and cross-channel obfuscation. This paper presents AWASH, a multimodal framework that redefines AI-washing detection as cross-modal claim-evidence reasoning (instead of surface-level similarity measurement), built on AW-Bench-the first large-scale trimodal benchmark for this task, including 88412 aligned annual report text, disclosure image, and earnings call video triplets from 4892 A-share listed firms during 2019Q1-2025Q2. We propose the Cross-Modal Inconsistency Detection (CMID) network, integrating a tri-modal encoder, a structured natural language inference module for claim-evidence entailment reasoning, and an operational grounding layer that cross-validates AI claims against verifiable physical evidence (patent filing trajectories, AI-specific talent recruitment, compute infrastructure proxies). Evaluated against six competitive baselines, CMID achieves an F1 score of 0.882 and an AUC-ROC of 0.921, outperforming the strongest text-only baseline by 17.4 percentage points and the latest multimodal competitor by 11.3 percentage points. A pre-registered user study with 14 regulatory analysts verifies that CMID-generated evidence reports cut case review time by 43% while increasing true positive detection rates by 28%. These findings confirm the technical superiority and practical applicability of structured multimodal reasoning for large-scale corporate disclosure surveillance.
Abstract (translated)
URL
https://arxiv.org/abs/2604.09644
PDF
-
EgoGroups: A Benchmark For Detecting Social Groups of People in the Wild
2026-03-23 17:43:49Jeffri Murrugarra-Llerena, Pranav Chitale, Zicheng Liu, Kai Ao, Yujin Ham, Guha Balakrishnan, Paola Cascante-BonillaAbstract
Social group detection, or the identification of humans involved in reciprocal interpersonal interactions (e.g., family members, friends, and customers and merchants), is a crucial component of social intelligence needed for agents transacting in the world. The few existing benchmarks for social group detection are limited by low scene diversity and reliance on third-person camera sources (e.g., surveillance footage). Consequently, these benchmarks generally lack real-world evaluation on how groups form and evolve in diverse cultural contexts and unconstrained settings. To address this gap, we introduce EgoGroups, a first-person view dataset that captures social dynamics in cities around the world. EgoGroups spans 65 countries covering low, medium, and high-crowd settings under four weather/time-of-day conditions. We include dense human annotations for person and social groups, along with rich geographic and scene metadata. Using this dataset, we performed an extensive evaluation of state-of-the-art VLM/LLMs and supervised models on their group detection capabilities. We found several interesting findings, including VLMs and LLMs can outperform supervised baselines in a zero-shot setting, while crowd density and cultural regions clearly influence model performance.
Abstract (translated)
社交群体检测，即识别参与相互人际互动（如家庭成员、朋友、顾客与商家）的人类，是智能体在现实世界中开展交互所需社交智能的关键组成部分。现有少数社交群体检测基准数据集受限于场景多样性不足以及过度依赖第三人称视角影像源（如监控录像），因此这些基准通常缺乏在多元文化背景和无约束环境下对群体形成与演变过程的真实世界评估。为填补此空白，我们推出了EgoGroups——一个捕捉全球城市社交动态的第一人称视角数据集。该数据集覆盖65个国家，包含低、中、高人流密度场景，并涵盖四种天气/时段条件。数据包含密集的人员与社交群体人工标注，以及丰富的地理与场景元数据。基于此数据集，我们对当前最先进的视觉语言模型/大语言模型及有监督模型进行了广泛的群体检测能力评估。研究发现若干有趣结论：视觉语言模型与大语言模型在零样本设置下可超越有监督基线模型，而人群密度与文化区域则显著影响模型性能。
URL
https://arxiv.org/abs/2603.22249
PDF
-
Adversarial Camouflage
2026-03-23 12:00:24Pawe{\l} Borsukiewicz, Daniele Lunghi, Melissa Tessa, Jacques Klein, Tegawend\'e F. Bissyand\'eAbstract
While the rapid development of facial recognition algorithms has enabled numerous beneficial applications, their widespread deployment has raised significant concerns about the risks of mass surveillance and threats to individual privacy. In this paper, we introduce \textit{Adversarial Camouflage} as a novel solution for protecting users' privacy. This approach is designed to be efficient and simple to reproduce for users in the physical world. The algorithm starts by defining a low-dimensional pattern space parameterized by color, shape, and angle. Optimized patterns, once found, are projected onto semantically valid facial regions for evaluation. Our method maximizes recognition error across multiple architectures, ensuring high cross-model transferability even against black-box systems. It significantly degrades the performance of all tested state-of-the-art face recognition models during simulations and demonstrates promising results in real-world human experiments, while revealing differences in model robustness and evidence of attack transferability across architectures.
Abstract (translated)
尽管人脸识别算法的快速发展催生了众多有益应用，但其大规模部署也引发了关于大规模监控风险和个人隐私威胁的重大担忧。本文提出\textit{对抗性伪装}作为保护用户隐私的新方案。该方法设计高效且便于物理世界用户复现。算法首先定义由颜色、形状和角度参数化的低维模式空间，优化后的模式会被投影至语义有效的人脸区域进行评估。我们的方法能在多种架构上最大化识别错误率，确保即使面对黑盒系统也具有高跨模型迁移性。在模拟测试中，该方法显著降低了所有 tested 最先进人脸识别模型的性能，并在真实人体实验中展现出前景性结果，同时揭示了不同模型鲁棒性的差异及跨架构攻击迁移性的证据。
URL
https://arxiv.org/abs/2603.21867
PDF
