README.md

---
license: apache-2.0
---
# In-Context Editing LoRA (Qwen-Image-Edit-2511)

## Model Introduction

This is an interesting model that enables [Qwen-Image-Edit-2511](https://modelscope.cn/models/Qwen/Qwen-Image-Edit-2511) with In-Context Editing capabilities. You can input three images to the model: Image 1, Image 2, and Image 3, and the model will automatically apply the transformation from Image 1 to Image 2 onto Image 3.

For more details about training strategies and implementation, feel free to check our [technical blog](https://huggingface.co/blog/kelseye/qwen-image-edit-2511-icedit-lora).

## Demo Results

Image sources:

* Input Image 1: https://modelscope.cn/aigc/imageGeneration?tab=advanced&imageId=18968195
* Input Image 2: Generated via single-image editing using the editing model
* Input Image 3: https://modelscope.cn/aigc/imageGeneration?tab=advanced&imageId=18723032

Prompt: `Edit image 3 based on the transformation from image 1 to image 2.`

Negative prompt: `yellowish tint, AI-like appearance, unrealistic, ugly, oily skin, abnormal limbs, disproportionate limbs`

* Example 1: Expression reference

|Input Image 1|Input Image 2|Input Image 3|Output Image|
|-|-|-|-|
|![](./assets/image1_original.png)|![](./assets/image1_edit_1.png)|![](./assets/image2_original.png)|![](./assets/image2_edit_1.png)|

* Example 2: Style transfer

|Input Image 1|Input Image 2|Input Image 3|Output Image|
|-|-|-|-|
|![](./assets/image1_original.png)|![](./assets/image1_edit_2.png)|![](./assets/image2_original.png)|![](./assets/image2_edit_2.png)|

* Example 3: Adding entities

|Input Image 1|Input Image 2|Input Image 3|Output Image|
|-|-|-|-|
|![](./assets/image1_original.png)|![](./assets/image1_edit_3.png)|![](./assets/image2_original.png)|![](./assets/image2_edit_3.png)|

* Example 4: Local editing

|Input Image 1|Input Image 2|Input Image 3|Output Image|
|-|-|-|-|
|![](./assets/image1_original.png)|![](./assets/image1_edit_4.png)|![](./assets/image2_original.png)|![](./assets/image2_edit_4.png)|

## Inference Code

Install [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio):

```shell
git clone https://github.com/modelscope/DiffSynth-Studio.git  
cd DiffSynth-Studio
pip install -e .
```

Inference code:

```python
from diffsynth.pipelines.qwen_image import QwenImagePipeline, ModelConfig
from modelscope import snapshot_download
from PIL import Image
import torch
```

# Load models
```
pipe = QwenImagePipeline.from_pretrained(
    torch_dtype=torch.bfloat16,
    device="cuda",
    model_configs=[
        ModelConfig(model_id="Qwen/Qwen-Image-Edit-2511", origin_file_pattern="transformer/diffusion_pytorch_model*.safetensors"),
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="text_encoder/model*.safetensors"),
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="vae/diffusion_pytorch_model.safetensors"),
    ],
    processor_config=ModelConfig(model_id="Qwen/Qwen-Image-Edit", origin_file_pattern="processor/"),
)
lora = ModelConfig(
    model_id="DiffSynth-Studio/Qwen-Image-Edit-2511-ICEdit-LoRA",
    origin_file_pattern="model.safetensors"
)
pipe.load_lora(pipe.dit, lora)
```

# Load images
```
snapshot_download(
    "DiffSynth-Studio/Qwen-Image-Edit-2511-ICEdit-LoRA",
    local_dir="./data",
    allow_file_pattern="assets/*"
)
edit_image = [
    Image.open("data/assets/image1_original.png"),
    Image.open("data/assets/image1_edit_1.png"),
    Image.open("data/assets/image2_original.png")
]
prompt = "Edit image 3 based on the transformation from image 1 to image 2."
negative_prompt = "yellowish, AI-looking, unrealistic, ugly, oily skin, abnormal limbs, disproportionate limbs"
```
# Generate
```
image_4 = pipe(
    prompt=prompt, negative_prompt=negative_prompt,
    edit_image=edit_image,
    seed=1,
    num_inference_steps=50,
    height=1280,
    width=720,
    zero_cond_t=True,
)
image_4.save("image.png")
```