object replacement with reference object

#53
by kashish892098887787ai - opened

Hi Qwen team and community,

I’m working on an object-replacement task using Qwen-Image-Edit-2511, where I need to replace a selected object in an interior scene using a reference image.

The inputs are:

  • Source room image
  • Mask of the object to be replaced
  • Reference image of the target object

The desired output is the same room with only the selected object replaced by the target object.

The most important requirement for our use case is preserving the actual identity and design of the reference object — including its structure, shape, proportions, color, material and fine visual details.

I initially tried SDXL Inpainting + ControlNet + IP-Adapter, but the reference object's structure was often changed significantly.

I then tested Qwen-Image-Edit-2511 with ReCoEdit-RL, and the results are considerably better. In our current experiments, Qwen is able to preserve approximately 90% of the reference object's identity/design in many cases.

However, we are facing a performance limitation.

Our current inference takes approximately 4 minutes per image, and the model requires relatively high VRAM.

I would therefore like to ask:

Is there a recommended way to reduce Qwen-Image-Edit-2511's VRAM usage and inference time while maintaining similar reference-image fidelity?

For our application, we would ideally like to achieve:

  • Strong preservation of the reference object's exact design/identity
  • Different viewpoints between source and reference images
  • Correct scale, orientation and perspective
  • Realistic lighting and integration
  • Minimal changes outside the selected object
  • Lower VRAM consumption
  • Faster inference

We are currently considering approaches such as quantization, regional/crop-based editing, LoRA-based optimization, attention optimization, or a two-stage workflow for viewpoint/geometry adaptation followed by Qwen editing.

If there is a recommended configuration, optimization technique, or complementary model/workflow that can improve inference speed and memory usage without significantly reducing reference fidelity, I would really appreciate your suggestions.

I can share example source images, reference images, masks and Qwen outputs if needed.

Thanks!

Sign up or log in to comment