probablybots commited on
Commit
74bcb6c
·
verified ·
1 Parent(s): 177ca8e

Rebrand AIDO.* -> GB.* (model card)

Browse files
Files changed (1) hide show
  1. README.md +15 -15
README.md CHANGED
@@ -1,21 +1,21 @@
1
  ---
2
  license: other
3
  ---
4
- # AIDO.Protein2StructureToken-16B
5
 
6
  [![License](https://img.shields.io/badge/license-GenBio_AI_Community_License-orange)](https://github.com/genbio-ai/ModelGenerator/blob/main/LICENSE)
7
 
8
- **AIDO.Protein2StructureToken-16B** is a fine-tuned version of [AIDO.Protein-16B](https://huggingface.co/genbio-ai/AIDO.Protein-16B), for protein structure prediction.
9
- This model uses amino acid sequences as input to predict tokens that can be decoded into 3D structures by [AIDO.StructureDecoder](https://huggingface.co/genbio-ai/AIDO.StructureDecoder).
10
  It surpasses existing state-of-the-art models, such as **ESM3-open**, in structure prediction tasks, demonstrating its robustness and capability in this domain.
11
 
12
  ## Model Architecture Details
13
 
14
- This model retains the architecture of AIDO.Protein-16B, a transformer encoder-only architecture with dense MLP layers replaced by sparse Mixture of Experts (MoE) layers.
15
  Each token activates 2 experts using a top-2 routing mechanism. A visual summary of the architecture is provided below:
16
 
17
  <center>
18
- <img src="https://huggingface.co/genbio-ai/AIDO.Protein-16B/resolve/main/proteinmoe_architecture.png" alt="AIDO.Protein-16B Architecture" style="width:70%; height:auto;" />
19
  </center>
20
 
21
 
@@ -54,7 +54,7 @@ The fine-tuning process used **0.4 trillion tokens**, using AlphaFold database w
54
  The input sequence should be single-chain amino acid sequences.
55
 
56
  - **Input Tokenization**: The sequences are tokenized at the amino acid level and terminated with a `[SEP]` token (id=34).
57
- - **Output Tokenization**: Each input token is converted into a structure token. The output can be decoded into 3D structures in PDB format using [AIDO.StructureDecoder](https://huggingface.co/genbio-ai/AIDO.StructureDecoder).
58
 
59
  ## Results
60
 
@@ -71,7 +71,7 @@ To reproduce the structure prediction results described above, follow these step
71
  2. Run the prediction command:
72
 
73
  ```bash
74
- mgen predict --config experiments/AIDO.StructureTokenizer/protein2structoken_16b.yaml
75
  ```
76
  This will pull the CASP14, CASP15, and CAMEO dataset from [genbio-ai/casp14-casp15-cameo-test-proteins](https://huggingface.co/datasets/genbio-ai/casp14-casp15-cameo-test-proteins), and predict the structure tokens from the amino acid sequence.
77
 
@@ -79,13 +79,13 @@ To reproduce the structure prediction results described above, follow these step
79
 
80
  ```bash
81
  # convert the predicted structures in tsv into one pt file
82
- python experiments/AIDO.StructureTokenizer/struct_token_format_conversion.py logs/protein2structoken_16b/predict_predictions.tsv logs/protein2structoken_16b/predict_predictions.pt
83
  # extract the codebook of the structure tokenizer
84
- python experiments/AIDO.StructureTokenizer/extract_structure_tokenizer_codebook.py --output_path logs/protein2structoken_16b/codebook.pt
85
  ```
86
  5. Run the decoding command to get 3D structures in PDB format (currently this script only supports single GPU inference):
87
  ```bash
88
- CUDA_VISIBLE_DEVICES=0 mgen predict --config experiments/AIDO.StructureTokenizer/decode.yaml \
89
  --data.init_args.config.struct_tokens_datasets_configs.name=protein2structoken_16b \
90
  --data.init_args.config.struct_tokens_datasets_configs.struct_tokens_path=logs/protein2structoken_16b/predict_predictions.pt \
91
  --data.init_args.config.struct_tokens_datasets_configs.codebook_path=logs/protein2structoken_16b/codebook.pt
@@ -93,15 +93,15 @@ To reproduce the structure prediction results described above, follow these step
93
  The outputs are in `logs/protstruct_decode/protein2structoken_16b_pdb_files/`
94
  6. You can compare the predicted structures with the ground truth PDBs in [genbio-ai/casp14-casp15-cameo-test-proteins](https://huggingface.co/datasets/genbio-ai/casp14-casp15-cameo-test-proteins/tree/main).
95
 
96
- Alternatively, you can provide your own input amino acid sequence in a CSV file. Here is one example csv at `experiments/AIDO.StructureTokenizer/protein2structoken_example_input.csv` in `ModelGenerator`:
97
  ```
98
  idx,aa_seq
99
  example,KEFWNLDKNLQLRLGIVFLG
100
  ```
101
  Here, `idx` is a unique name, and `aa_seq` is the amino acid sequence. To use this customized CSV file, replace the second step with
102
  ```bash
103
- mgen predict --config experiments/AIDO.StructureTokenizer/protein2structoken_16b.yaml \
104
- --data.init_args.path=experiments/AIDO.StructureTokenizer/ \
105
  --data.init_args.test_split_files=[protein2structoken_example_input.csv]
106
  ```
107
 
@@ -111,7 +111,7 @@ For more information, visit: [Model Generator](https://github.com/genbio-ai/mode
111
  mgen fit --model SequenceClassification --model.backbone aido_protein_16b --data SequenceClassificationDataModule --data.path <hf_or_local_path_to_your_dataset>
112
  mgen test --model SequenceClassification --model.backbone aido_protein_16b --data SequenceClassificationDataModule --data.path <hf_or_local_path_to_your_dataset>
113
  ```
114
- The usage of this model is the same as [AIDO.Protein-16B](https://huggingface.co/genbio-ai/AIDO.Protein-16B).
115
  You only need to change the `model.backbone` to `aido_protein2structoken`.
116
 
117
 
@@ -155,7 +155,7 @@ print(logits)
155
  ```
156
 
157
  ## Citation
158
- Please cite AIDO.Protein and AIDO.StructureTokenizer using the following BibTex codes:
159
  ```
160
  @inproceedings{zhang_balancing_2024,
161
  title = {Balancing Locality and Reconstruction in Protein Structure Tokenizer},
 
1
  ---
2
  license: other
3
  ---
4
+ # GB.Protein2StructureToken-16B
5
 
6
  [![License](https://img.shields.io/badge/license-GenBio_AI_Community_License-orange)](https://github.com/genbio-ai/ModelGenerator/blob/main/LICENSE)
7
 
8
+ **GB.Protein2StructureToken-16B** is a fine-tuned version of [GB.Protein-16B](https://huggingface.co/genbio-ai/GB.Protein-16B), for protein structure prediction.
9
+ This model uses amino acid sequences as input to predict tokens that can be decoded into 3D structures by [GB.StructureDecoder](https://huggingface.co/genbio-ai/GB.StructureDecoder).
10
  It surpasses existing state-of-the-art models, such as **ESM3-open**, in structure prediction tasks, demonstrating its robustness and capability in this domain.
11
 
12
  ## Model Architecture Details
13
 
14
+ This model retains the architecture of GB.Protein-16B, a transformer encoder-only architecture with dense MLP layers replaced by sparse Mixture of Experts (MoE) layers.
15
  Each token activates 2 experts using a top-2 routing mechanism. A visual summary of the architecture is provided below:
16
 
17
  <center>
18
+ <img src="https://huggingface.co/genbio-ai/GB.Protein-16B/resolve/main/proteinmoe_architecture.png" alt="GB.Protein-16B Architecture" style="width:70%; height:auto;" />
19
  </center>
20
 
21
 
 
54
  The input sequence should be single-chain amino acid sequences.
55
 
56
  - **Input Tokenization**: The sequences are tokenized at the amino acid level and terminated with a `[SEP]` token (id=34).
57
+ - **Output Tokenization**: Each input token is converted into a structure token. The output can be decoded into 3D structures in PDB format using [GB.StructureDecoder](https://huggingface.co/genbio-ai/GB.StructureDecoder).
58
 
59
  ## Results
60
 
 
71
  2. Run the prediction command:
72
 
73
  ```bash
74
+ mgen predict --config experiments/GB.StructureTokenizer/protein2structoken_16b.yaml
75
  ```
76
  This will pull the CASP14, CASP15, and CAMEO dataset from [genbio-ai/casp14-casp15-cameo-test-proteins](https://huggingface.co/datasets/genbio-ai/casp14-casp15-cameo-test-proteins), and predict the structure tokens from the amino acid sequence.
77
 
 
79
 
80
  ```bash
81
  # convert the predicted structures in tsv into one pt file
82
+ python experiments/GB.StructureTokenizer/struct_token_format_conversion.py logs/protein2structoken_16b/predict_predictions.tsv logs/protein2structoken_16b/predict_predictions.pt
83
  # extract the codebook of the structure tokenizer
84
+ python experiments/GB.StructureTokenizer/extract_structure_tokenizer_codebook.py --output_path logs/protein2structoken_16b/codebook.pt
85
  ```
86
  5. Run the decoding command to get 3D structures in PDB format (currently this script only supports single GPU inference):
87
  ```bash
88
+ CUDA_VISIBLE_DEVICES=0 mgen predict --config experiments/GB.StructureTokenizer/decode.yaml \
89
  --data.init_args.config.struct_tokens_datasets_configs.name=protein2structoken_16b \
90
  --data.init_args.config.struct_tokens_datasets_configs.struct_tokens_path=logs/protein2structoken_16b/predict_predictions.pt \
91
  --data.init_args.config.struct_tokens_datasets_configs.codebook_path=logs/protein2structoken_16b/codebook.pt
 
93
  The outputs are in `logs/protstruct_decode/protein2structoken_16b_pdb_files/`
94
  6. You can compare the predicted structures with the ground truth PDBs in [genbio-ai/casp14-casp15-cameo-test-proteins](https://huggingface.co/datasets/genbio-ai/casp14-casp15-cameo-test-proteins/tree/main).
95
 
96
+ Alternatively, you can provide your own input amino acid sequence in a CSV file. Here is one example csv at `experiments/GB.StructureTokenizer/protein2structoken_example_input.csv` in `ModelGenerator`:
97
  ```
98
  idx,aa_seq
99
  example,KEFWNLDKNLQLRLGIVFLG
100
  ```
101
  Here, `idx` is a unique name, and `aa_seq` is the amino acid sequence. To use this customized CSV file, replace the second step with
102
  ```bash
103
+ mgen predict --config experiments/GB.StructureTokenizer/protein2structoken_16b.yaml \
104
+ --data.init_args.path=experiments/GB.StructureTokenizer/ \
105
  --data.init_args.test_split_files=[protein2structoken_example_input.csv]
106
  ```
107
 
 
111
  mgen fit --model SequenceClassification --model.backbone aido_protein_16b --data SequenceClassificationDataModule --data.path <hf_or_local_path_to_your_dataset>
112
  mgen test --model SequenceClassification --model.backbone aido_protein_16b --data SequenceClassificationDataModule --data.path <hf_or_local_path_to_your_dataset>
113
  ```
114
+ The usage of this model is the same as [GB.Protein-16B](https://huggingface.co/genbio-ai/GB.Protein-16B).
115
  You only need to change the `model.backbone` to `aido_protein2structoken`.
116
 
117
 
 
155
  ```
156
 
157
  ## Citation
158
+ Please cite GB.Protein and GB.StructureTokenizer using the following BibTex codes:
159
  ```
160
  @inproceedings{zhang_balancing_2024,
161
  title = {Balancing Locality and Reconstruction in Protein Structure Tokenizer},