Transformers documentation

Qwen2.5-VL

Transformers

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v5.22.1).

Join the Hugging Face community

and get access to the augmented documentation experience

Collaborate on models, datasets and Spaces

Faster examples with accelerated inference

Switch between documentation themes

to get started

Model ini dipublikasikan di HF papers pada 2025-02-19 dan disumbangkan ke Hugging Face Transformers pada 2025-01-23.

Qwen2.5-VL

Qwen2.5-VL adalah model bahasa-visi multimodal, tersedia dalam 3B, 7B, dan 72B parameter, yang telah dilatih sebelumnya pada 4,1T token. Model ini memperkenalkan window attention pada encoder ViT untuk mempercepat pelatihan dan inferensi, pengambilan sampel FPS dinamis pada dimensi spasial dan temporal untuk pemahaman video yang lebih baik di berbagai laju pengambilan sampel, serta mekanisme MRoPE (multi-resolutional rotary positional encoding) yang ditingkatkan untuk lebih baik menangkap dan mempelajari dinamika temporal.

Anda dapat menemukan semua checkpoint asli Qwen2.5-VL di bawah koleksi Qwen2.5-VL.

Klik model Qwen2.5-VL di bilah sisi kanan untuk melihat lebih banyak contoh cara menerapkan Qwen2.5-VL ke berbagai tugas visi dan bahasa.

Setel use_kernels=True di from_pretrained() untuk mengganti lapisan yang didukung dengan kernel yang dioptimalkan dari Hub. Lihat Memuat kernel untuk mempelajari lebih lanjut.

Contoh di bawah ini menunjukkan cara menghasilkan teks berdasarkan gambar dengan Pipeline atau kelas AutoModel.

Pipeline

AutoModel

Quantization mengurangi beban memori model besar dengan merepresentasikan bobot dalam presisi yang lebih rendah. Lihat ringkasan Quantization untuk backend quantization lain yang tersedia.

Contoh di bawah menggunakan torchao untuk hanya mengkuantisasi bobot ke int4.

from transformers import Qwen2_5_VLForConditionalGeneration, TorchAoConfig


quantization_config = TorchAoConfig("int4_weight_only", group_size=128)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2.5-VL-7B-Instruct",
    device_map="auto",
    quantization_config=quantization_config
)

Catatan

Gunakan Qwen2.5-VL untuk input video dengan mengatur "type": "video" seperti yang ditunjukkan di bawah ini.

conversation = [
    {
        "role": "user",
        "content": [
            {"type": "video", "path": "/path/to/video.mp4"},
            {"type": "text", "text": "What happened in the video?"},
        ],
    }
]

inputs = processor.apply_chat_template(
    conversation,
    fps=1,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt"
).to(model.device)

# Inference: Generation of the output
output_ids = model.generate(**inputs, max_new_tokens=128)
generated_ids = [output_ids[len(input_ids):] for input_ids, output_ids in zip(inputs.input_ids, output_ids)]
output_text = processor.batch_decode(generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True)
print(output_text)

Gunakan Qwen2.5-VL untuk batch campuran input (gambar, video, teks). Tambahkan label saat menangani beberapa gambar atau video untuk referensi yang lebih baik, seperti yang ditunjukkan di bawah.

import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2.5-VL-7B-Instruct",
    device_map="auto",
    attn_implementation="sdpa"
)
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")
conversation = [
    {
        "role": "user",
        "content": [
            {"type": "image"}, 
            {"type": "text", "text": "Hello, how are you?"}
        ]
    },
    {
        "role": "assistant",
        "content": "I'm doing well, thank you for asking. How can I assist you today?"
    },
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "Can you describe these images and video?"}, 
            {"type": "image"}, 
            {"type": "image"}, 
            {"type": "video"}, 
            {"type": "text", "text": "These are from my vacation."}
        ]
    },
    {
        "role": "assistant",
        "content": "I'd be happy to describe the images and video for you. Could you please provide more context about your vacation?"
    },
    {
        "role": "user",
        "content": "It was a trip to the mountains. Can you see the details in the images and video?"
    }
]

# default:
prompt_without_id = processor.apply_chat_template(conversation, add_generation_prompt=True)
# Excepted output: '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\n<|vision_start|><|image_pad|><|vision_end|>Hello, how are you?<|im_end|>\n<|im_start|>assistant\nI'm doing well, thank you for asking. How can I assist you today?<|im_end|>\n<|im_start|>user\nCan you describe these images and video?<|vision_start|><|image_pad|><|vision_end|><|vision_start|><|image_pad|><|vision_end|><|vision_start|><|video_pad|><|vision_end|>These are from my vacation.<|im_end|>\n<|im_start|>assistant\nI'd be happy to describe the images and video for you. Could you please provide more context about your vacation?<|im_end|>\n<|im_start|>user\nIt was a trip to the mountains. Can you see the details in the images and video?<|im_end|>\n<|im_start|>assistant\n'


# add ids
prompt_with_id = processor.apply_chat_template(conversation, add_generation_prompt=True, add_vision_id=True)
# Excepted output: '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nPicture 1: <|vision_start|><|image_pad|><|vision_end|>Hello, how are you?<|im_end|>\n<|im_start|>assistant\nI'm doing well, thank you for asking. How can I assist you today?<|im_end|>\n<|im_start|>user\nCan you describe these images and video?Picture 2: <|vision_start|><|image_pad|><|vision_end|>Picture 3: <|vision_start|><|image_pad|><|vision_end|>Video 1: <|vision_start|><|video_pad|><|vision_end|>These are from my vacation.<|im_end|>\n<|im_start|>assistant\nI'd be happy to describe the images and video for you. Could you please provide more context about your vacation?<|im_end|>\n<|im_start|>user\nIt was a trip to the mountains. Can you see the details in the images and video?<|im_end|>\n<|im_start|>assistant\n'

Gunakan parameter min_pixels dan max_pixels di AutoProcessor untuk mengatur resolusi.

min_pixels = 224*224
max_pixels = 2048*2048
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct", min_pixels=min_pixels, max_pixels=max_pixels)

Resolusi yang lebih tinggi dapat membutuhkan lebih banyak komputasi, sedangkan mengurangi resolusi dapat menghemat memori sebagai berikut:

min_pixels = 256*28*28
max_pixels = 1024*28*28 
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct", min_pixels=min_pixels, max_pixels=max_pixels)

Transformers

Qwen2.5-VL

Catatan

Qwen2_5_VLConfig

kelas transformers.Qwen2_5_VLConfig

Qwen2_5_VLVisionConfig

kelas transformers.Qwen2_5_VLVisionConfig

Qwen2_5_VLTextConfig

kelas transformers.Qwen2_5_VLTextConfig

Qwen2_5_VLProcessor

kelas transformers.Qwen2_5_VLProcessor

__call__

Qwen2_5_VLTextModel

kelas transformers.Qwen2_5_VLTextModel

forward

Qwen2_5_VisionTransformerPretrainedModel

kelas transformers.Qwen2_5_VisionTransformerPretrainedModel

forward

Qwen2_5_VLModel

kelas transformers.Qwen2_5_VLModel

forward

get_video_features

get_image_features

Qwen2_5_VLForConditionalGeneration

kelas transformers.Qwen2_5_VLForConditionalGeneration

forward

get_video_features

get_image_features

call