Contour
-
Identity-Aware U-Net: Fine-grained Cell Segmentation via Identity-Aware Representation Learning
2026-04-07 08:43:04Rui XiaoAbstract
Precise segmentation of objects with highly similar shapes remains a challenging problem in dense prediction, especially in scenarios with ambiguous boundaries, overlapping instances, and weak inter-instance visual differences. While conventional segmentation models are effective at localizing object regions, they often lack the discriminative capacity required to reliably distinguish a target object from morphologically similar distractors. In this work, we study fine-grained object segmentation from an identity-aware perspective and propose Identity-Aware U-Net (IAU-Net), a unified framework that jointly models spatial localization and instance discrimination. Built upon a U-Net-style encoder-decoder architecture, our method augments the segmentation backbone with an auxiliary embedding branch that learns discriminative identity representations from high-level features, while the main branch predicts pixel-accurate masks. To enhance robustness in distinguishing objects with near-identical contours or textures, we further incorporate triplet-based metric learning, which pulls target-consistent embeddings together and separates them from hard negatives with similar morphology. This design enables the model to move beyond category-level segmentation and acquire a stronger capability for precise discrimination among visually similar objects. Experiments on benchmarks including cell segmentation demonstrate promising results, particularly in challenging cases involving similar contours, dense layouts, and ambiguous boundaries.
Abstract (translated)
URL
https://arxiv.org/abs/2604.09702
PDF
-
SBF: An Effective Representation to Augment Skeleton for Video-based Human Action Recognition
2026-04-04 04:41:08Zhuoxuan Peng, Yiyi Ding, Yang Lin, S. -H. Gary ChanAbstract
Many modern video-based human action recognition (HAR) approaches use 2D skeleton as the intermediate representation in their prediction pipelines. Despite overall encouraging results, these approaches still struggle in many common scenes, mainly because the skeleton does not capture critical action-related information pertaining to the depth of the joints, contour of the human body, and interaction between the human and objects. To address this, we propose an effective approach to augment skeleton with a representation capturing action-related information in the pipeline of HAR. The representation, termed Scale-Body-Flow (SBF), consists of three distinct components, namely a scale map volume given by the scale (and hence depth information) of each joint, a body map outlining the human subject, and a flow map indicating human-object interaction given by pixel-wise optical flow values. To predict SBF, we further present SFSNet, a novel segmentation network supervised by the skeleton and optical flow without extra annotation overhead beyond the existing skeleton extraction. Extensive experiments across different datasets demonstrate that our pipeline based on SBF and SFSNet achieves significantly higher HAR accuracy with similar compactness and efficiency as compared with the state-of-the-art skeleton-only approaches.
Abstract (translated)
许多现代基于视频的人体动作识别（HAR）方法在预测流程中使用2D骨架作为中间表示。尽管整体结果令人鼓舞，但这些方法在许多常见场景中仍面临挑战，主要原因在于骨架无法捕捉与关节深度、人体轮廓以及人与物体交互相关的关键动作信息。为解决这一问题，我们提出了一种有效方法，在HAR流程中通过一种能捕捉动作相关信息的表示来增强骨架。该表示称为尺度-体素-流（SBF），包含三个不同组件：由每个关节的尺度（因而包含深度信息）给出的尺度图体积、勾勒人体主体的体素图，以及通过像素级光流值表示的人-物交互流图。为预测SBF，我们进一步提出了SFSNet——一种通过骨架和光流进行监督的新型分割网络，无需在现有骨架提取之外增加额外标注开销。在不同数据集上的大量实验表明，基于SBF和SFSNet的流程相比最先进的纯骨架方法，在保持相似紧凑性和效率的同时，显著提升了HAR准确率。
URL
https://arxiv.org/abs/2604.03590
PDF
-
Automated Segmentation and Tracking of Group Housed Pigs Using Foundation Models
2026-04-03 19:50:08Ye Bi, Bimala Acharya, David Rosero, Juan SteibelAbstract
Foundation models (FM) are reshaping computer vision by reducing reliance on task-specific supervised learning and leveraging general visual representations learned at scale. In precision livestock farming, most pipelines remain dominated by supervised learning models that require extensive labeled data, repeated retraining, and farm-specific tuning. This study presents an FM-centered workflow for automated monitoring of group-housed nursery pigs, in which pretrained vision-language FM serve as general visual backbones and farm-specific adaptation is achieved through modular post-processing. Grounding-DINO was first applied to 1,418 annotated images to establish a baseline detection performance. While detection accuracy was high under daytime conditions, performance degraded under night-vision and heavy occlusion, motivating the integration of temporal tracking logic. Building on these detections, short-term video segmentation with Grounded-SAM2 was evaluated on 550 one-minute video clips; after post-processing, over 80% of 4,927 active tracks were fully correct, with most remaining errors arising from inaccurate masks or duplicated labels. To support identity consistency over an extended time, we further developed a long-term tracking pipeline integrating initialization, tracking, matching, mask refinement, re-identification, and post-hoc quality control. This system was evaluated on a continuous 132-minute video and maintained stable identities throughout. On 132 uniformly sampled ground-truth frames, the system achieved a mean region similarity (J) of 0.83, contour accuracy (F) of 0.92, J&F of 0.87, MOTA of 0.99, and MOTP of 90.7%, with no identity switches. Overall, this work demonstrates how FM prior knowledge can be combined with lightweight, task-specific logic to enable scalable, label-efficient, and long-duration monitoring in pig production.
Abstract (translated)
基础模型（FM）正通过减少对特定任务监督学习的依赖并利用大规模学习的通用视觉表征，重塑计算机视觉领域。在精准畜牧业中，大多数流程仍以需要大量标注数据、反复重新训练和农场特定调优的监督学习模型为主。本研究提出了一种以FM为核心的工作流程，用于群养保育猪的自动化监测，其中预训练的视觉-语言FM作为通用视觉主干网络，并通过模块化后处理实现农场特定适应。首先在1418张标注图像上应用Grounding-DINO建立基线检测性能。尽管白天条件下检测准确率较高，但在夜视和严重遮挡场景下性能下降，因此集成了时序追踪逻辑。基于这些检测结果，在550段一分钟视频片段上评估了Grounded-SAM2的短期视频分割效果；经过后处理，4927条活跃追踪轨迹中超过80%完全正确，剩余错误主要源于掩码不准确或标签重复。为实现长期身份一致性，进一步开发了集成初始化、追踪、匹配、掩码精炼、重识别和事后质量控制的长期追踪流程。该系统在连续132分钟视频上进行了评估，全程保持稳定的身份识别。在132个均匀采样的真实帧上，系统达到平均区域相似度（J）0.83、轮廓准确率（F）0.92、J&F值0.87、MOTA指标0.99、MOTP指标90.7%，且无身份切换。总体而言，本研究展示了如何将FM先验知识与轻量级任务特定逻辑相结合，实现猪生产中可扩展、标签高效和长时程监测。
URL
https://arxiv.org/abs/2604.03426
PDF
-
From Elevation Maps To Contour Lines: SVM and Decision Trees to Detect Violin Width Reduction
2026-04-02 18:16:52Phil\'emon Beghin, Anne-Emmanuelle Ceulemans, Fran\c{c}ois GlineurAbstract
We explore the automatic detection of violin width reduction using 3D photogrammetric meshes. We compare SVM and Decision Trees applied to a geometry-based raw representation built from elevation maps with a more targeted, feature-engineered approach relying on parametric contour lines fitting. Although elevation maps occasionally achieve strong results, their performance does not surpass that of the contour-based inputs.
Abstract (translated)
我们探索利用3D摄影测量网格自动检测小提琴宽度缩减的方法。比较了基于高程图构建的几何原始表示上应用的支持向量机与决策树，以及依赖参数化轮廓线拟合的更具针对性的特征工程方法。尽管高程图偶尔能取得显著效果，但其性能并未超越基于轮廓的输入方式。
URL
https://arxiv.org/abs/2604.02446
PDF
-
Contour-Guided Query-Based Feature Fusion for Boundary-Aware and Generalizable Cardiac Ultrasound Segmentation
2026-03-30 07:17:40Zahid Ullah, Sieun Choi, Jihie KimAbstract
Accurate cardiac ultrasound segmentation is essential for reliable assessment of ventricular function in intelligent healthcare systems. However, echocardiographic images are challenging due to low contrast, speckle noise, irregular boundaries, and domain shifts across devices and patient populations. Existing methods, largely based on appearance-driven learning, often fail to preserve boundary precision and structural consistency under these conditions. To address these issues, we propose a Contour-Guided Query Refinement Network (CGQR-Net) for boundary-aware cardiac ultrasound segmentation. The framework integrates multi-resolution feature representations with contour-derived structural priors. An HRNet backbone preserves high-resolution spatial details while capturing multi-scale context. A coarse segmentation is first generated, from which anatomical contours are extracted and encoded into learnable query embeddings. These contour-guided queries interact with fused feature maps via cross-attention, enabling structure-aware refinement that improves boundary delineation and reduces noise artifacts. A dual-head supervision strategy jointly optimizes segmentation and boundary prediction to enforce structural consistency. The proposed method is evaluated on the CAMUS dataset and further validated on the CardiacNet dataset to assess cross-dataset generalization. Experimental results demonstrate improved segmentation accuracy, enhanced boundary precision, and robust performance across varying imaging conditions. These results highlight the effectiveness of integrating contour-level structural information with feature-level representations for reliable cardiac ultrasound segmentation.
Abstract (translated)
URL
https://arxiv.org/abs/2603.28110
PDF
-
Evaluating Large and Lightweight Vision Models for Irregular Component Segmentation in E-Waste Disassembly
2026-03-28 23:06:36Xinyao Zhang, Chang Liu, Xiao Liang, Minghui Zheng, Sara BehdadAbstract
Precise segmentation of irregular and densely arranged components is essential for robotic disassembly and material recovery in electronic waste (e-waste) recycling. This study evaluates the impact of model architecture and scale on segmentation performance by comparing SAM2, a transformer-based vision model, with the lightweight YOLOv8 network. Both models were trained and tested on a newly collected dataset of 1,456 annotated RGB images of laptop components including logic boards, heat sinks, and fans, captured under varying illumination and orientation conditions. Data augmentation techniques, such as random rotation, flipping, and cropping, were applied to improve model robustness. YOLOv8 achieved higher segmentation accuracy (mAP50 = 98.8%, mAP50-95 = 85%) and stronger boundary precision than SAM2 (mAP50 = 8.4%). SAM2 demonstrated flexibility in representing diverse object structures but often produced overlapping masks and inconsistent contours. These findings show that large pre-trained models require task-specific optimization for industrial applications. The resulting dataset and benchmarking framework provide a foundation for developing scalable vision algorithms for robotic e-waste disassembly and circular manufacturing systems.
Abstract (translated)
URL
https://arxiv.org/abs/2603.27441
PDF
-
Make It Up: Fake Images, Real Gains in Generalized Few-shot Semantic Segmentation
2026-03-28 09:22:50Guohuan Xie, Xin He, Dingying Fan, Le Zhang, Ming-Ming Cheng, Yun LiuAbstract
Generalized few-shot semantic segmentation (GFSS) is fundamentally limited by the coverage of novel-class appearances under scarce annotations. While diffusion models can synthesize novel-class images at scale, practical gains are often hindered by insufficient coverage and noisy supervision when masks are unavailable or unreliable. We propose Syn4Seg, a generation-enhanced GFSS framework designed to expand novel-class coverage while improving pseudo-label quality. Syn4Seg first maximizes prompt-space coverage by constructing an embedding-deduplicated prompt bank for each novel class, yielding diverse yet class-consistent synthetic images. It then performs support-guided pseudo-label estimation via a two-stage refinement that i) filters low-consistency regions to obtain high-precision seeds and ii) relabels uncertain pixels with image-adaptive prototypes that combine global (support) and local (image) statistics. Finally, we refine only boundary-band and unlabeled pixels using a constrained SAM-based update to improve contour fidelity without overwriting high-confidence interiors. Extensive experiments on PASCAL-$5^i$ and COCO-$20^i$ demonstrate consistent improvements in both 1-shot and 5-shot settings, highlighting synthetic data as a scalable path for GFSS with reliable masks and precise boundaries.
Abstract (translated)
广义小样本语义分割（GFSS）从根本上受限于稀缺标注下新类外观的覆盖范围。尽管扩散模型能够大规模合成新类图像，但当掩码不可用或不可靠时，覆盖不足和噪声监督往往制约实际效果。我们提出Syn4Seg，一个生成增强型GFSS框架，旨在扩展新类覆盖范围同时提升伪标签质量。Syn4Seg首先通过为每个新类构建嵌入去重的提示词库，最大化提示空间覆盖，生成多样且类别一致的合成图像。随后执行支持引导的伪标签估计，采用两阶段细化：i）过滤低一致性区域以获得高精度种子点；ii）利用结合全局（支持集）与局部（图像）统计的图像自适应原型，对不确定像素进行重标注。最后，仅对边界带和未标注像素执行基于约束SAM的更新，在不覆盖高置信度内部区域的前提下提升轮廓精度。在PASCAL-$5^i$和COCO-$20^i$上的大量实验表明，该方法在1-shot和5-shot设置下均实现稳定提升，凸显了合成数据作为GFSS可扩展路径的潜力，可同时保证可靠掩码与精确边界。
URL
https://arxiv.org/abs/2603.27206
PDF
-
Training-free Detection and 6D Pose Estimation of Unseen Surgical Instruments
2026-03-26 09:28:19Jonas Hein, Lilian Calvet, Matthias Seibold, Siyu Tang, Marc Pollefeys, Philipp F\"urnstahlAbstract
Purpose: Accurate detection and 6D pose estimation of surgical instruments are crucial for many computer-assisted interventions. However, supervised methods lack flexibility for new or unseen tools and require extensive annotated data. This work introduces a training-free pipeline for accurate multi-view 6D pose estimation of unseen surgical instruments, which only requires a textured CAD model as prior knowledge. Methods: Our pipeline consists of two main stages. First, for detection, we generate object mask proposals in each view and score their similarity to rendered templates using a pre-trained feature extractor. Detections are matched across views, triangulated into 3D instance candidates, and filtered using multi-view geometric consistency. Second, for pose estimation, a set of pose hypotheses is iteratively refined and scored using feature-metric scores with cross-view attention. The best hypothesis undergoes a final refinement using a novel multi-view, occlusion-aware contour registration, which minimizes reprojection errors of unoccluded contour points. Results: The proposed method was rigorously evaluated on real-world surgical data from the MVPSP dataset. The method achieves millimeter-accurate pose estimates that are on par with supervised methods under controlled conditions, while maintaining full generalization to unseen instruments. These results demonstrate the feasibility of training-free, marker-less detection and tracking in surgical scenes, and highlight the unique challenges in surgical environments. Conclusion: We present a novel and flexible pipeline that effectively combines state-of-the-art foundational models, multi-view geometry, and contour-based refinement for high-accuracy 6D pose estimation of surgical instruments without task-specific training. This approach enables robust instrument tracking and scene understanding in dynamic clinical environments.
Abstract (translated)
目的：精确检测和6D姿态估计对于许多计算机辅助介入手术至关重要。然而，监督方法缺乏对新工具或未见工具的灵活性，且需要大量标注数据。本研究提出了一种针对未见手术器械的免训练多视图6D姿态估计流程，仅需带纹理的CAD模型作为先验知识。 方法：我们的流程包含两个主要阶段。首先，在检测阶段，我们在每个视图中生成目标掩码提案，并使用预训练的特征提取器评估其与渲染模板的相似性。检测结果跨视图匹配，三角化为3D实例候选，并通过多视图几何一致性进行过滤。其次，在姿态估计阶段，一组姿态假设通过跨视图注意力的特征度量分数进行迭代优化和评分。最佳假设采用一种新颖的多视图、感知遮挡的轮廓配准方法进行最终优化，该方法最小化未遮挡轮廓点的重投影误差。 结果：所提方法在MVPSP数据集的真实手术数据上进行了严格评估。该方法实现了毫米级精度的姿态估计，在受控条件下与监督方法相当，同时保持对未见器械的完全泛化能力。这些结果证明了手术场景中免训练、无标记检测与跟踪的可行性，并凸显了手术环境中的独特挑战。 结论：我们提出了一种新颖灵活的流程，有效结合了最先进的基座模型、多视图几何和基于轮廓的优化，无需任务特定训练即可实现手术器械的高精度6D姿态估计。该方法支持动态临床环境中的稳健器械跟踪与场景理解。
URL
https://arxiv.org/abs/2603.25228
PDF
-
SilLang: Improving Gait Recognition with Silhouette Language Encoding
2026-03-25 06:15:29Ruiyi Zhan, Guozhen Peng, Canyu Chen, Jian Lei, Annan LiAbstract
Gait silhouettes, which can be encoded into binary gait codes, are widely adopted to representing motion patterns of pedestrian. Recent approaches commonly leverage visual backbones to encode gait silhouettes, achieving successful performance. However, they primarily focus on continuous visual features, overlooking the discrete nature of binary silhouettes that inherently share a discrete encoding space with natural language. Large Language Models (LLMs) have demonstrated exceptional capability in extracting discriminative features from discrete sequences and modeling long-range dependencies, highlighting their potential to capture temporal motion patterns by identifying subtle variations. Motivated by these observations, we explore bridging binary gait silhouettes and natural language within a binary encoding space. However, the encoding spaces of text tokens and binary gait silhouettes remain misaligned, primarily due to differences in token frequency and density. To address this issue, we propose the Contour-Velocity Tokenizer, which encodes binary gait silhouettes while reshaping their distribution to better align with the text token space. We then establish a dual-branch framework termed Silhouette Language Model, which enhances visual silhouettes by integrating discrete linguistic embeddings derived from LLMs. Implemented on mainstream gait backbones, SilLang consistently improves state-of-the-art methods across SUSTech1K, GREW, and Gait3D.
Abstract (translated)
步态剪影可被编码为二值步态编码，广泛用于表征行人运动模式。近期方法通常利用视觉主干网络对步态剪影进行编码，取得了良好性能。然而这些方法主要关注连续视觉特征，忽略了二值剪影的离散特性——其本质上与自然语言共享离散编码空间。大语言模型（LLMs）已展现出从离散序列中提取判别特征及建模长程依赖的卓越能力，凸显了其通过捕捉细微变化来理解时序运动模式的潜力。受此启发，我们尝试在二值编码空间内建立步态剪影与自然语言的关联。然而文本标记与二值步态剪影的编码空间仍存在错配，主要源于标记频率与密度的差异。为此，我们提出轮廓-速度分词器，在编码二值步态剪影的同时重塑其分布以更好对齐文本标记空间。进而构建名为剪影语言模型的双分支框架，通过融合源自大语言模型的离散语言嵌入来增强视觉剪影表征。该方法在主流步态主干网络上实现，于SUSTech1K、GREW及Gait3D数据集上持续提升现有最优方法性能。
URL
https://arxiv.org/abs/2603.23976
PDF
-
Multimodal Industrial Anomaly Detection via Geometric Prior
2026-03-24 03:34:08Min Li, Jinghui He, Gang Li, Jiachen Li, Jin Wan, Delong HanAbstract
The purpose of multimodal industrial anomaly detection is to detect complex geometric shape defects such as subtle surface deformations and irregular contours that are difficult to detect in 2D-based methods. However, current multimodal industrial anomaly detection lacks the effective use of crucial geometric information like surface normal vectors and 3D shape topology, resulting in low detection accuracy. In this paper, we propose a novel Geometric Prior-based Anomaly Detection network (GPAD). Firstly, we propose a point cloud expert model to perform fine-grained geometric feature extraction, employing differential normal vector computation to enhance the geometric details of the extracted features and generate geometric prior. Secondly, we propose a two-stage fusion strategy to efficiently leverage the complementarity of multimodal data as well as the geometric prior inherent in 3D points. We further propose attention fusion and anomaly regions segmentation based on geometric prior, which enhance the model's ability to perceive geometric defects. Extensive experiments show that our multimodal industrial anomaly detection model outperforms the State-of-the-art (SOTA) methods in detection accuracy on both MVTec-3D AD and Eyecandies datasets.
Abstract (translated)
多模态工业异常检测旨在检测复杂的几何形状缺陷，例如基于二维方法难以检测的细微表面变形和不规则轮廓。然而，当前多模态工业异常检测缺乏对表面法向量和三维形状拓扑等关键几何信息的有效利用，导致检测精度较低。本文提出了一种新颖的基于几何先验的异常检测网络（GPAD）。首先，我们提出点云专家模型进行细粒度几何特征提取，采用微分法向量计算增强提取特征的几何细节并生成几何先验。其次，我们提出两阶段融合策略，以高效利用多模态数据的互补性以及三维点云固有的几何先验。我们还进一步提出了基于几何先验的注意力融合与异常区域分割，增强了模型对几何缺陷的感知能力。大量实验表明，我们的多模态工业异常检测模型在MVTec-3D AD和Eyecandies数据集上的检测精度均优于最先进（SOTA）方法。
URL
https://arxiv.org/abs/2603.22757
PDF
-
GTSR: Subsurface Scattering Awared 3D Gaussians for Translucent Surface Reconstruction
Abstract
Reconstructing translucent objects from multi-view images is a difficult problem. Previously, researchers have used differentiable path tracing and the neural implicit field, which require relatively large computational costs. Recently, many works have achieved good reconstruction results for opaque objects based on a 3DGS pipeline with much higher efficiency. However, such methods have difficulty dealing with translucent objects, because they do not consider the optical properties of translucent objects. In this paper, we propose a novel 3DGS-based pipeline (GTSR) to reconstruct the surface geometry of translucent objects. GTSR combines two sets of Gaussians, surface and interior Gaussians, which are used to model the surface and scattering color when lights pass translucent objects. To render the appearance of translucent objects, we introduce a method that uses the Fresnel term to blend two sets of Gaussians. Furthermore, to improve the reconstructed details of non-contour areas, we introduce the Disney BSDF model with deferred rendering to enhance constraints of the normal and depth. Experimental results demonstrate that our method outperforms baseline reconstruction methods on the NeuralTO Syn dataset while showing great real-time rendering performance. We also extend the dataset with new translucent objects of varying material properties and demonstrate our method can adapt to different translucent materials.
Abstract (translated)
从多视角图像重建半透明物体是一个难题。此前，研究者们采用可微路径追踪与神经隐式场等方法，但这些方案需要较高的计算成本。近期，许多基于3D高斯泼溅（3DGS）管线的方法在高效重建不透明物体方面取得了良好效果。然而，此类方法难以处理半透明物体，因其未考虑半透明物体的光学特性。本文提出一种基于3DGS的新管线（GTSR），用于重建半透明物体的表面几何结构。GTSR结合了两组高斯元——表面高斯元与内部高斯元，分别用于建模半透明物体表面的反射光与光线穿透后的散射颜色。为渲染半透明物体外观，我们引入基于菲涅尔项混合两组高斯元的方法。此外，为提升非轮廓区域的重建细节，我们采用迪士尼双向散射分布函数（BSDF）模型结合延迟渲染技术，增强对法线与深度的约束。实验结果表明，在NeuralTO Syn合成数据集上，本方法优于基线重建方法，同时展现出优秀的实时渲染性能。我们还扩展了包含不同材质特性的新半透明物体数据集，并验证了本方法可适应多种半透明材质。
URL
https://arxiv.org/abs/2603.22036
PDF
-
SHARP: Spectrum-aware Highly-dynamic Adaptation for Resolution Promotion in Remote Sensing Synthesis
2026-03-23 10:25:45Bingxuan Zhao, Qing Zhou, Chuang Yang, Qi WangAbstract
Text-to-image generation powered by Diffusion Transformers (DiTs) has made remarkable strides, yet remote sensing (RS) synthesis lags behind due to two barriers: the absence of a domain-specialized DiT prior and the prohibitive cost of training at the large resolutions that RS applications demand. Training-free resolution promotion via Rotary Position Embedding (RoPE) rescaling offers a practical remedy, but every existing method applies a static positional scaling rule throughout the denoising process. This uniform compression is particularly harmful for RS imagery, whose substantially denser medium- and high-frequency energy encodes the fine structures critical for aerial-scene realism, such as vehicles, building contours, and road markings. Addressing both challenges requires a domain-specialized generative prior coupled with a denoising-aware positional adaptation strategy. To this end, we fine-tune FLUX on over 100,000 curated RS images to build a strong domain prior (RS-FLUX), and propose Spectrum-aware Highly-dynamic Adaptation for Resolution Promotion (SHARP), a training-free method that introduces a rational fractional time schedule k_rs(t) into RoPE. SHARP applies strong positional promotion during the early layout-formation stage and progressively relaxes it during detail recovery, aligning extrapolation strength with the frequency-progressive nature of diffusion denoising. Its resolution-agnostic formulation further enables robust multi-scale generation from a single set of hyperparameters. Extensive experiments across six square and rectangular resolutions show that SHARP consistently outperforms all training-free baselines on CLIP Score, Aesthetic Score, and HPSv2, with widening margins at more aggressive extrapolation factors and negligible computational overhead. Code and weights are available at this https URL.
Abstract (translated)
基于扩散Transformer（DiT）的文生图技术已取得显著进展，但遥感（RS）图像合成仍面临两大瓶颈：缺乏领域专用的DiT先验模型，以及训练高分辨率RS应用所需模型的 prohibitive 成本。通过旋转位置嵌入（RoPE）重缩放实现的无训练分辨率提升提供了一种实用方案，但现有方法均在去噪全程采用静态位置缩放规则。这种均匀压缩对遥感图像尤为不利——其中高密度的中高频能量编码着对航拍场景真实性至关重要的精细结构（如车辆、建筑轮廓与道路标线）。要同时解决这两大挑战，需要领域专用的生成先验与感知去噪阶段的位置自适应策略。为此，我们在超过10万张精选遥感图像上微调FLUX模型，构建出强领域先验（RS-FLUX），并提出基于频谱感知的高动态自适应分辨率提升方法（SHARP）。该方法在RoPE中引入有理数分数时间调度k_rs(t)，在早期布局形成阶段施加强位置增强，并在细节恢复阶段逐步放松，使外推强度与扩散去噪的频率渐进特性相匹配。其分辨率无关的公式设计还能通过单一超参数集实现稳健的多尺度生成。在六种方形与矩形分辨率上的大量实验表明，SHARP在CLIP Score、美学评分与HPSv2指标上持续超越所有无训练基线方法，且在外推因子更激进时优势扩大，计算开销可忽略。代码与模型权重已公开于此https URL。
URL
https://arxiv.org/abs/2603.21783
PDF
-
Boundary-Aware Instance Segmentation in Microscopy Imaging
2026-03-22 12:59:22Thomas Mendelson, Joshua Francois, Galit Lahav, Tammy Riklin-RavivAbstract
Accurate delineation of individual cells in microscopy videos is essential for studying cellular dynamics, yet separating touching or overlapping instances remains a persistent challenge. Although foundation-model for segmentation such as SAM have broadened the accessibility of image segmentation, they still struggle to separate nearby cell instances in dense microscopy scenes without extensive prompting. We propose a prompt-free, boundary-aware instance segmentation framework that predicts signed distance functions (SDFs) instead of binary masks, enabling smooth and geometry-consistent modeling of cell contours. A learned sigmoid mapping converts SDFs into probability maps, yielding sharp boundary localization and robust separation of adjacent instances. Training is guided by a unified Modified Hausdorff Distance (MHD) loss that integrates region- and boundary-based terms. Evaluations on both public and private high-throughput microscopy datasets demonstrate improved boundary accuracy and instance-level performance compared to recent SAM-based and foundation-model approaches. Source code is available at: this https URL
Abstract (translated)
在显微视频中精确勾画单个细胞对于研究细胞动态至关重要，但分离接触或重叠的细胞实例仍是一项长期挑战。尽管SAM等基础分割模型已拓宽了图像分割的可及性，但在密集显微场景中，它们仍难以在无需大量提示的情况下分离邻近的细胞实例。我们提出了一种无需提示、边界感知的实例分割框架，该框架预测符号距离函数而非二值掩码，从而实现对细胞轮廓平滑且几何一致的建模。通过学习的sigmoid映射将符号距离函数转换为概率图，实现了相邻实例间清晰的边界定位与稳健分离。训练过程由统一的改进豪斯多夫距离损失指导，该损失整合了区域与边界相关项。在公共及私有高通量显微数据集上的评估表明，与近期基于SAM及基础模型的方法相比，本方法在边界精度与实例级性能上均有提升。源代码见：此https链接
URL
https://arxiv.org/abs/2603.21206
PDF
-
Representation-Level Adversarial Regularization for Clinically Aligned Multitask Thyroid Ultrasound Assessment
2026-03-22 07:14:43Dina Salama, Mohamed Mahmoud, Nourhan Bayasi, David Liu, Ilker HacihalilogluAbstract
Thyroid ultrasound is the first-line exam for assessing thyroid nodules and determining whether biopsy is warranted. In routine reporting, radiologists produce two coupled outputs: a nodule contour for measurement and a TI-RADS risk category based on sonographic criteria. Yet both contouring style and risk grading vary across readers, creating inconsistent supervision that can degrade standard learning pipelines. In this paper, we address this workflow with a clinically guided multitask framework that jointly predicts the nodule mask and TI-RADS category within a single model. To ground risk prediction in clinically meaningful evidence, we guide the classification embedding using a compact TI-RADS aligned radiomics target during training, while preserving complementary deep features for discriminative performance. However, under annotator variability, naive multitask optimization often fails not because the tasks are unrelated, but because their gradients compete within the shared representation. To make this competition explicit and controllable, we introduce RLAR, a representation-level adversarial gradient regularizer. Rather than performing parameter-level gradient surgery, RLAR uses each task's normalized adversarial direction in latent space as a geometric probe of task sensitivity and penalizes excessive angular alignment between task-specific adversarial directions. On a public TI-RADS dataset, our clinically guided multitask model with RLAR consistently improves risk stratification while maintaining segmentation quality compared to single-task training and conventional multitask baselines. Code and pretrained models will be released.
Abstract (translated)
甲状腺超声是评估甲状腺结节及判断是否需要活检的首选检查。在常规报告中，放射科医生会生成两种耦合输出：用于测量的结节轮廓，以及基于超声标准的TI-RADS风险分类。然而，不同阅片者的勾画风格和风险分级存在差异，导致监督不一致，可能削弱标准学习流程的效果。本文针对这一工作流程，提出一种临床引导的多任务框架，可在单一模型中联合预测结节掩膜与TI-RADS分类。为使风险预测基于有临床意义的证据，我们在训练中使用紧凑的TI-RADS对齐影像组学目标引导分类嵌入，同时保留互补的深度特征以提升判别性能。然而，在标注者存在变异的情况下，朴素的多任务优化常会失败——并非因为任务不相关，而是因为它们的梯度在共享表征中相互竞争。为使这种竞争显式且可控，我们引入RLAR（表征层面对抗梯度正则化器）。RLAR并非执行参数层面的梯度手术，而是利用各任务在隐空间归一化的对抗方向作为任务敏感性的几何探针，并对任务特定对抗方向间过大的角度对齐进行惩罚。在一个公开的TI-RADS数据集上，我们的临床引导多任务模型结合RLAR，在保持分割质量的同时，持续提升了风险分层效果，优于单任务训练和常规多任务基线。代码与预训练模型将公开发布。
URL
https://arxiv.org/abs/2603.21095
PDF
-
MEMO: Human-like Crisp Edge Detection Using Masked Edge Prediction
2026-03-21 12:21:29Jiaxin Cheng, Yue Wu, Yicong ZhouAbstract
Learning-based edge detection models trained with cross-entropy loss often suffer from thick edge predictions, which deviate from the crisp, single-pixel annotations typically provided by humans. While previous approaches to achieving crisp edges have focused on designing specialized loss functions or modifying network architectures, we show that a carefully designed training and inference strategy alone is sufficient to achieve human-like edge quality. In this work, we introduce the Masked Edge Prediction MOdel (MEMO), which produces both accurate and crisp edges using only cross-entropy loss. We first construct a large-scale synthetic edge dataset to pre-train MEMO, enhancing its generalization ability. Subsequent fine-tuning on downstream datasets requires only a lightweight module comprising 1.2\% additional parameters. During training, MEMO learns to predict edges under varying ratios of input masking. A key insight guiding our inference is that thick edge predictions typically exhibit a confidence gradient: high in the center and lower toward the boundaries. Leveraging this, we propose a novel progressive prediction strategy that sequentially finalizes edge predictions in order of prediction confidence, resulting in thinner and more precise contours. Our method achieves visually appealing, post-processing-free, human-like edge maps and outperforms prior methods on crispness-aware evaluations.
Abstract (translated)
基于交叉熵损失实现精准锐利的边缘检测，一直是计算机视觉领域的一项挑战。本文提出了MEMO模型，仅通过交叉熵损失即可生成既准确又锐利的边缘。我们首先构建了一个大规模合成边缘数据集用于预训练MEMO，以增强其泛化能力。随后在各类下游数据集上的微调仅需一个轻量级模块，该模块仅增加1.2%的参数。在训练过程中，MEMO学习在不同比例的输入掩码下预测边缘。其推理过程的核心思想是：粗边缘预测通常呈现置信度梯度——中心区域置信度高，向边缘边界逐渐降低。基于此，我们提出一种新颖的渐进式预测策略，该策略按预测置信度顺序逐步确定边缘，从而得到更细、更精确的轮廓。我们的方法能够生成无需后处理、视觉上媲美人工标注的优美边缘图，并在锐利度感知的评估指标上超越了以往方法。
URL
https://arxiv.org/abs/2603.20782
PDF
-
Gesture2Speech: How Far Can Hand Movements Shape Expressive Speech?
2026-03-20 10:17:10Lokesh Kumar, Nirmesh Shah, Ashishkumar P. Gudmalwar, Pankaj WasnikAbstract
Human communication seamlessly integrates speech and bodily motion, where hand gestures naturally complement vocal prosody to express intent, emotion, and emphasis. While recent text-to-speech (TTS) systems have begun incorporating multimodal cues such as facial expressions or lip movements, the role of hand gestures in shaping prosody remains largely underexplored. We propose a novel multimodal TTS framework, Gesture2Speech, that leverages visual gesture cues to modulate prosody in synthesized speech. Motivated by the observation that confident and expressive speakers coordinate gestures with vocal prosody, we introduce a multimodal Mixture-of-Experts (MoE) architecture that dynamically fuses linguistic content and gesture features within a dedicated style extraction module. The fused representation conditions an LLM-based speech decoder, enabling prosodic modulation that is temporally aligned with hand movements. We further design a gesture-speech alignment loss that explicitly models their temporal correspondence to ensure fine-grained synchrony between gestures and prosodic contours. Evaluations on the PATS dataset show that Gesture2Speech outperforms state-of-the-art baselines in both speech naturalness and gesture-speech synchrony. To the best of our knowledge, this is the first work to utilize hand gesture cues for prosody control in neural speech synthesis. Demo samples are available at this https URL
Abstract (translated)
人类交流无缝融合了言语与肢体动作，其中手势自然地与语音韵律相辅相成，以表达意图、情感和强调。尽管近期文本转语音（TTS）系统已开始整合面部表情或唇部动作等多模态线索，但手势在塑造韵律方面的作用仍 largely 未被探索。我们提出了一种新颖的多模态 TTS 框架 Gesture2Speech，该框架利用视觉手势线索来调节合成语音的韵律。受自信且富有表现力的演讲者会协调手势与语音韵律这一观察的启发，我们引入了一种多模态混合专家（MoE）架构，可在专用的风格提取模块内动态融合语言内容与手势特征。融合后的表征条件着一个基于大语言模型的语音解码器，从而实现与手部运动时间对齐的韵律调节。我们还设计了一种手势-语音对齐损失，显式建模它们的时间对应关系，以确保手势与韵律轮廓之间的细粒度同步。在 PATS 数据集上的评估表明，Gestuer2Speech 在语音自然度和手势-语音同步性两方面均优于最新基线方法。据我们所知，这是首个利用手部手势线索进行神经语音合成中韵律控制的工作。演示样本可在此 https URL 获取。
URL
https://arxiv.org/abs/2603.19831
PDF
-
Transparent Fragments Contour Estimation via Visual-Tactile Fusion for Autonomous Reassembly
2026-03-18 14:58:17Qihao Lin, Borui Chen, Yuping Zhou, Jianing Wu, Yulan Guo, Weishi Zheng, Chongkun XiaAbstract
The contour estimation of transparent fragments is very important for autonomous reassembly, especially in the fields of precision optical instrument repair, cultural relic restoration, and identification of other precious device broken accidents. Different from general intact transparent objects, the contour estimation of transparent fragments face greater challenges due to strict optical properties, irregular shapes and edges. To address this issue, a general transparent fragments contour estimation framework based on visual-tactile fusion is proposed in this paper. First, we construct the transparent fragment dataset named TransFrag27K, which includes a multiscene synthetic data of broken fragments from multiple types of transparent objects, and a scalable synthetic data generation pipeline. Secondly, we propose a visual grasping position detection network named TransFragNet to identify, locate and segment the sampling grasping position. And, we use a two-finger gripper with Gelsight Mini sensors to obtain reconstructed tactile information of the lateral edge of the fragments. By fusing this tactile information with visual cues, a visual-tactile fusion material classifier is proposed. Inspired by the way humans estimate a fragment's contour combining vision and touch, we introduce a general transparent fragment contour estimation framework based on visual-tactile fusion, demonstrates strong performance in real-world validation. Finally, a multi-dimensional similarity metrics based contour matching and reassembly algorithm is proposed, providing a reproducible benchmark for evaluating visual-tactile contour estimation and fragment reassembly. The experimental results demonstrate the validity of the proposed framework. The dataset and codes are available at this https URL.
Abstract (translated)
透明碎片的轮廓估计对于自主重组至关重要，尤其在精密光学仪器修复、文物复原及其他珍贵设备破损事故鉴定等领域。与一般完好的透明物体不同，透明碎片的轮廓估计面临更大挑战，因其具有严格的光学特性、不规则的形状与边缘。针对此问题，本文提出一种基于视觉-触觉融合的通用透明碎片轮廓估计框架。首先，我们构建了名为TransFrag27K的透明碎片数据集，该数据集包含多类型透明物体破碎碎片的多场景合成数据，以及一个可扩展的合成数据生成流程。其次，我们提出了名为TransFragNet的视觉抓取位置检测网络，用于识别、定位并分割采样抓取位置。同时，我们使用配备Gelsight Mini传感器的双指夹爪获取碎片侧边的重建触觉信息。通过融合该触觉信息与视觉线索，提出了一种视觉-触觉融合材料分类器。受人类结合视觉与触觉估计碎片轮廓的方式启发，我们引入了基于视觉-触觉融合的通用透明碎片轮廓估计框架，并在实际验证中展现出强劲性能。最后，提出了一种基于多维相似度度量的轮廓匹配与重组算法，为评估视觉-触觉轮廓估计与碎片重组提供了可复现的基准。实验结果表明了所提框架的有效性。数据集与代码已公开于此https URL。
URL
https://arxiv.org/abs/2603.20290
PDF
-
Geometric and Topological Deep Learning for Predicting Thermo-mechanical Performance in Cold Spray Deposition Process Modeling
2026-03-15 16:46:05Akshansh MishraAbstract
This study presents a geometric deep learning framework for predicting cold spray particle impact responses using finite element simulation data. A parametric dataset was generated through automated Abaqus simulations spanning a systematic range of particle velocity, particle temperature, and friction coefficient, yielding five output targets including maximum equivalent plastic strain, average contact plastic strain, maximum temperature, maximum von Mises stress, and deformation ratio. Four novel algorithms i.e. a GraphSAGE-style inductive graph neural network, a Chebyshev spectral graph convolution network, a topological data analysis augmented multilayer perceptron, and a geometric attention network were implemented and evaluated. Each input sample was treated as a node in a k-nearest-neighbour feature-space graph, enabling the models to exploit spatial similarity between process conditions during training. Three-dimensional feature space visualisations and two-dimensional contour projections confirmed the highly non-linear and velocity-dominated nature of the input-output relationships. Quantitative evaluation demonstrated that GraphSAGE and GAT consistently achieved R-square values exceeding 0.93 across most targets, with GAT attaining peak performance of R-square equal to 0.97 for maximum plastic strain. ChebSpectral and TDA-MLP performed considerably worse, yielding negative R-square values for several targets. These findings establish spatial graph-based neighbourhood aggregation as a robust and physically interpretable surrogate modelling strategy for cold spray process optimisation.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14478
PDF
-
Echoes Across Centuries: Phonetic Signatures of Persian Poets
2026-03-15 15:41:21Kourosh Shahnazari, Seyed Moein Ayyoubzadeh, Mohammadali KeshtparvarAbstract
This study examines phonetic texture in Persian poetry as a literary-historical phenomenon rather than a by-product of meter or a feature used only for classification. The analysis draws on a large corpus of 1,116,306 mesras from 31,988 poems written by 83 poets, restricted to five major classical meters to enable controlled comparison. Each line is converted into a grapheme-to-phoneme representation and analyzed using six phonetic metrics: hardness, sonority, sibilance, vowel ratio, phoneme entropy, and consonant-cluster ratio. Statistical models estimate poet-level differences while controlling for meter, poetic form, and line length. The results show that although meter and form explain a substantial portion of phonetic variation, they do not eliminate systematic differences between poets. Persian poetic sound therefore appears as conditioned variation within shared prosodic structures rather than as either purely individual style or simple metrical residue. A multidimensional stylistic map reveals several recurrent phonetic profiles, including high-sonority lyric styles, hardness-driven rhetorical or epic styles, sibilant mystical contours, and high-entropy complex textures. Historical analysis indicates that phonetic distributions shift across centuries, reflecting changes in genre prominence, literary institutions, and performance contexts rather than abrupt stylistic breaks. The study establishes a corpus-scale framework for phonetic analysis in Persian poetry and demonstrates how computational phonetics can contribute to literary-historical interpretation while remaining attentive to the formal structures that shape Persian verse.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14443
PDF
-
Deep EM with Hierarchical Latent Label Modelling for Multi-Site Prostate Lesion Segmentation
2026-03-15 15:03:20Wen Yan, Yipei Wang, Shiqi Huang, Natasha Thorley, Mark Emberton, Vasilis Stavrinides, Yipeng Hu, Dean BarrattAbstract
Label variability is a major challenge for prostate lesion segmentation. In multi-site datasets, annotations often reflect centre-specific contouring protocols, causing segmentation networks to overfit to local styles and generalise poorly to unseen sites in inference. We treat each observed annotation as a noisy observation of an underlying latent 'clean' lesion mask, and propose a hierarchical expectation-maximisation (HierEM) framework that alternates between: (1) inferring a voxel-wise posterior distribution over the latent mask, and (2) training a CNN using this posterior as a soft target and estimate site-specific sensitivity and specificity under a hierarchical prior. This hierarchical prior decomposes label-quality into a global mean with site- and case-level deviations, reducing site-specific bias by penalising the likelihood term contributed only by site deviations. Experiments on three cohorts demonstrate that the proposed hierarchical EM framework enhances cross-site generalisation compared to state-of-the-art methods. For pooled-dataset evaluation, the per-site mean DSC ranges from 29.50% to 39.69%; for leave-one-site-out generalisation, it ranges from 27.91% to 32.67%, yielding statistically significant improvements over comparison methods (p<0.039). The method also produces interpretable per-site latent label-quality estimates (sensitivity alpha ranges from 31.5% to 47.3% at specificity beta approximates 0.99), supporting post-hoc analyses of cross-site annotation variability. These results indicate that explicitly modelling site-dependent annotation can improve cross-site generalisation.
Abstract (translated)
URL
https://arxiv.org/abs/2603.14418
PDF
