Abstract
We introduce a new setting, Edit Transfer, where a model learns a transformation from just a single source-target example and applies it to a new query image. While text-based methods excel at semantic manipulations through textual prompts, they often struggle with precise geometric details (e.g., poses and viewpoint changes). Reference-based editing, on the other hand, typically focuses on style or appearance and fails at non-rigid transformations. By explicitly learning the editing transformation from a source-target pair, Edit Transfer mitigates the limitations of both text-only and appearance-centric references. Drawing inspiration from in-context learning in large language models, we propose a visual relation in-context learning paradigm, building upon a DiT-based text-to-image model. We arrange the edited example and the query image into a unified four-panel composite, then apply lightweight LoRA fine-tuning to capture complex spatial transformations from minimal examples. Despite using only 42 training samples, Edit Transfer substantially outperforms state-of-the-art TIE and RIE methods on diverse non-rigid scenarios, demonstrating the effectiveness of few-shot visual relation learning.
Abstract (translated)
我们提出了一种新的设置，称为编辑转移（Edit Transfer），在这种设置中，模型通过单一的源-目标示例学习转换，并将其应用到一个新的查询图像上。尽管基于文本的方法在语义操作方面表现出色，但它们往往难以处理精确的几何细节（如姿态和视角变化）。而参考基编辑方法通常侧重于风格或外观上的修改，在非刚性变换时表现不佳。通过从源-目标对中显式学习编辑转换，编辑转移减轻了仅基于文本和以外观为中心参考方法的局限性。 我们借鉴大型语言模型中的上下文学习概念，提出了一种视觉关系上下文学习范式，并在此基础上构建了一个DiT（Diffusion Model for Text-to-Image）为基础的文本到图像模型。我们将编辑示例与查询图像组合成一个统一的四面板复合图，然后采用轻量级LoRA微调方法来捕捉从少量样本中复杂的空间变换。尽管仅使用了42个训练样本，编辑转移在各种非刚性场景中的表现仍显著优于当前最先进的文本到图像（TIE）和参考基图像编辑（RIE）方法，证明了少样本视觉关系学习的有效性。
URL
https://arxiv.org/abs/2503.13327
