Abstract
Localizing 3D objects using natural language is essential for robotic scene understanding. The descriptions often involve multiple spatial relationships to distinguish similar objects, making 3D-language alignment difficult. Current methods only model relationships for pairwise objects, ignoring the global perceptual significance of n-ary combinations in multi-modal relational understanding. To address this, we propose a novel progressive relational learning framework for 3D object grounding. We extend relational learning from binary to n-ary to identify visual relations that match the referential description globally. Given the absence of specific annotations for referred objects in the training data, we design a grouped supervision loss to facilitate n-ary relational learning. In the scene graph created with n-ary relationships, we use a multi-modal network with hybrid attention mechanisms to further localize the target within the n-ary combinations. Experiments and ablation studies on the ReferIt3D and ScanRefer benchmarks demonstrate that our method outperforms the state-of-the-art, and proves the advantages of the n-ary relational perception in 3D localization.
Abstract (translated)
使用自然语言定位3D对象对于机器人场景理解至关重要。这些描述通常涉及多个空间关系来区分相似的对象，从而使得3D和语言之间的对齐变得复杂。目前的方法仅针对成对物体建模其关系，忽略了在多模式关系理解中n-元组合的全局感知重要性。为了解决这个问题，我们提出了一种新颖的渐进式关系学习框架用于3D对象定位。我们将关系学习从二元扩展到n-元，以识别与指称描述相匹配的整体视觉关系。由于训练数据中缺乏具体针对参考对象的注释，我们设计了一个分组监督损失函数来促进n-元关系学习。在使用n-元关系构建的场景图中，我们采用一种具有混合注意力机制的多模式网络进一步在n-元组合内定位目标。我们在ReferIt3D和ScanRefer基准测试上的实验及消融研究证明了我们的方法优于最先进的方法，并且验证了n-元感知关系在3D定位中的优势。
URL
https://arxiv.org/abs/2510.10194
