Text-to-Image
Transformers
Safetensors
nielsr HF Staff commited on
Commit
2d51605
·
verified ·
1 Parent(s): 5ffb89f

Improve model card: add metadata and sample usage

Browse files

Hi! I'm Niels, part of the community science team at Hugging Face.

This PR improves the model card for MedITok by:
- Adding the `text-to-image` pipeline tag to improve discoverability.
- Adding `library_name: transformers` metadata, as the repository contains configuration files and architectures compatible with the Hugging Face Transformers library.
- Including a sample usage section for image feature extraction, sourced directly from the official GitHub repository.
- Linking the model to its corresponding paper on the Hub.

Please review and merge if this looks good to you!

Files changed (1) hide show
  1. README.md +36 -2
README.md CHANGED
@@ -1,5 +1,7 @@
1
  ---
2
  license: cc-by-4.0
 
 
3
  ---
4
 
5
  <div align="center">
@@ -14,13 +16,46 @@ license: cc-by-4.0
14
  </p>
15
 
16
  ## 🚀 Introduction
17
- MedITok is the first unified visual tokenizer for medical images. Trained on 30M medical images and 2M image-caption pairs via a two-stage representation learning framework, MedITok:
18
  - effectively encodes visual details and clinical semantics into a unified token space
19
  - achieves state-of-the-art performance across diverse medical imaging modalities and tasks.
20
  - can be incorporated into prevelant generative models (e.g., autoregressive architectures) for downstream medical image synthesis and interpretation.
21
 
22
  This work is supported by Shanghai Innovation Institute (SII).
23
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
 
25
  ## ✏️ Citation
26
 
@@ -31,5 +66,4 @@ This work is supported by Shanghai Innovation Institute (SII).
31
  journal={arXiv preprint arXiv:2505.19225},
32
  year={2025}
33
  }
34
-
35
  ```
 
1
  ---
2
  license: cc-by-4.0
3
+ library_name: transformers
4
+ pipeline_tag: text-to-image
5
  ---
6
 
7
  <div align="center">
 
16
  </p>
17
 
18
  ## 🚀 Introduction
19
+ MedITok is the first unified visual tokenizer for medical images, introduced in [Unified Medical Image Tokenizer for Autoregressive Synthesis and Understanding](https://huggingface.co/papers/2505.19225). Trained on 33M medical images and 2M image-caption pairs via a two-stage representation learning framework, MedITok:
20
  - effectively encodes visual details and clinical semantics into a unified token space
21
  - achieves state-of-the-art performance across diverse medical imaging modalities and tasks.
22
  - can be incorporated into prevelant generative models (e.g., autoregressive architectures) for downstream medical image synthesis and interpretation.
23
 
24
  This work is supported by Shanghai Innovation Institute (SII).
25
 
26
+ ## 🎯 Sample Usage
27
+
28
+ ### Image feature extraction
29
+ The following snippet demonstrates how to use the model for extracting features (requires the model implementation from the [official repository](https://github.com/masaaki-75/meditok)):
30
+
31
+ ```python
32
+ import torch
33
+ import numpy as np
34
+ from PIL import Image
35
+
36
+ def read_image(img, img_size=256):
37
+ if isinstance(img, str):
38
+ img = Image.open(img)
39
+
40
+ if isinstance(img, Image.Image):
41
+ img = img.convert('RGB')
42
+ if img.size[0] != img_size:
43
+ img = img.resize((img_size, img_size), Image.LANCZOS)
44
+ return img
45
+
46
+ def image_to_tensor(x):
47
+ # [H, W, C] -> [B, C, H, W]
48
+ x = torch.FloatTensor(np.array(x)).permute(2, 0, 1)
49
+ x = (x / 255.) * 2. - 1.
50
+ return x.unsqueeze(0)
51
+
52
+ # Assuming 'net' is the loaded MedITok model
53
+ img_path = 'assets/vis_imgs/sample1.png'
54
+ img = read_image(img_path)
55
+ x = image_to_tensor(img)
56
+ with torch.no_grad():
57
+ f = net.forward_features(x)
58
+ ```
59
 
60
  ## ✏️ Citation
61
 
 
66
  journal={arXiv preprint arXiv:2505.19225},
67
  year={2025}
68
  }
 
69
  ```