isaaccorley commited on
Commit
a87fa09
·
verified ·
1 Parent(s): 6e2f5a2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +46 -3
README.md CHANGED
@@ -1,3 +1,46 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: torchgeo
4
+ ---
5
+
6
+ Model rehosted from https://github.com/wangzhecheng/SkyScript
7
+
8
+ Original source: https://opendatasharing.s3.us-west-2.amazonaws.com/SkyScript/ckpt/SkyCLIP_ViT_L14_top30pct.zip
9
+
10
+ Model weights extracted below:
11
+
12
+ ```python
13
+ import os
14
+ import torch
15
+ import hashlib
16
+
17
+ import timm
18
+ from open_clip.factory import create_model_and_transforms
19
+ from timm.models.vision_transformer import _convert_openai_clip
20
+
21
+
22
+ path = "SkyCLIP_ViT_L14_top30pct/epoch_20.pt"
23
+ encoder = "ViT-L-14"
24
+ model, _, preprocess_val = create_model_and_transforms(encoder, path, weights_only=False)
25
+ print(preprocess_val)
26
+ model_timm = timm.create_model("vit_large_patch14_clip_224", pretrained=False, num_classes=768)
27
+ converted = _convert_openai_clip(model.state_dict(), model_timm)
28
+ model_timm.load_state_dict(converted, strict=True)
29
+
30
+ filename = "vit_large_patch14_224_skyclip_30pct.pth"
31
+ torch.save(model_timm.state_dict(), filename)
32
+ md5 = hashlib.md5(open(filename, "rb").read()).hexdigest()[:8]
33
+ os.rename(filename, filename.replace(".pth", f"-{md5}.pth"))
34
+ ```
35
+
36
+ The preprocessing transforms are:
37
+
38
+ ```
39
+ Compose(
40
+ Resize(size=224, interpolation=bicubic, max_size=None, antialias=True)
41
+ CenterCrop(size=(224, 224))
42
+ <function _convert_to_rgb at 0x33047ff60>
43
+ ToTensor()
44
+ Normalize(mean=(0.48145466, 0.4578275, 0.40821073), std=(0.26862954, 0.26130258, 0.27577711))
45
+ )
46
+ ```