Rebrand AIDO.* -> GB.* (model card)
Browse files
README.md
CHANGED
|
@@ -1,21 +1,21 @@
|
|
| 1 |
---
|
| 2 |
license: other
|
| 3 |
---
|
| 4 |
-
#
|
| 5 |
|
| 6 |
[](https://github.com/genbio-ai/ModelGenerator/blob/main/LICENSE)
|
| 7 |
|
| 8 |
-
**
|
| 9 |
-
This model uses amino acid sequences as input to predict tokens that can be decoded into 3D structures by [
|
| 10 |
It surpasses existing state-of-the-art models, such as **ESM3-open**, in structure prediction tasks, demonstrating its robustness and capability in this domain.
|
| 11 |
|
| 12 |
## Model Architecture Details
|
| 13 |
|
| 14 |
-
This model retains the architecture of
|
| 15 |
Each token activates 2 experts using a top-2 routing mechanism. A visual summary of the architecture is provided below:
|
| 16 |
|
| 17 |
<center>
|
| 18 |
-
<img src="https://huggingface.co/genbio-ai/
|
| 19 |
</center>
|
| 20 |
|
| 21 |
|
|
@@ -54,7 +54,7 @@ The fine-tuning process used **0.4 trillion tokens**, using AlphaFold database w
|
|
| 54 |
The input sequence should be single-chain amino acid sequences.
|
| 55 |
|
| 56 |
- **Input Tokenization**: The sequences are tokenized at the amino acid level and terminated with a `[SEP]` token (id=34).
|
| 57 |
-
- **Output Tokenization**: Each input token is converted into a structure token. The output can be decoded into 3D structures in PDB format using [
|
| 58 |
|
| 59 |
## Results
|
| 60 |
|
|
@@ -71,7 +71,7 @@ To reproduce the structure prediction results described above, follow these step
|
|
| 71 |
2. Run the prediction command:
|
| 72 |
|
| 73 |
```bash
|
| 74 |
-
mgen predict --config experiments/
|
| 75 |
```
|
| 76 |
This will pull the CASP14, CASP15, and CAMEO dataset from [genbio-ai/casp14-casp15-cameo-test-proteins](https://huggingface.co/datasets/genbio-ai/casp14-casp15-cameo-test-proteins), and predict the structure tokens from the amino acid sequence.
|
| 77 |
|
|
@@ -79,13 +79,13 @@ To reproduce the structure prediction results described above, follow these step
|
|
| 79 |
|
| 80 |
```bash
|
| 81 |
# convert the predicted structures in tsv into one pt file
|
| 82 |
-
python experiments/
|
| 83 |
# extract the codebook of the structure tokenizer
|
| 84 |
-
python experiments/
|
| 85 |
```
|
| 86 |
5. Run the decoding command to get 3D structures in PDB format (currently this script only supports single GPU inference):
|
| 87 |
```bash
|
| 88 |
-
CUDA_VISIBLE_DEVICES=0 mgen predict --config experiments/
|
| 89 |
--data.init_args.config.struct_tokens_datasets_configs.name=protein2structoken_16b \
|
| 90 |
--data.init_args.config.struct_tokens_datasets_configs.struct_tokens_path=logs/protein2structoken_16b/predict_predictions.pt \
|
| 91 |
--data.init_args.config.struct_tokens_datasets_configs.codebook_path=logs/protein2structoken_16b/codebook.pt
|
|
@@ -93,15 +93,15 @@ To reproduce the structure prediction results described above, follow these step
|
|
| 93 |
The outputs are in `logs/protstruct_decode/protein2structoken_16b_pdb_files/`
|
| 94 |
6. You can compare the predicted structures with the ground truth PDBs in [genbio-ai/casp14-casp15-cameo-test-proteins](https://huggingface.co/datasets/genbio-ai/casp14-casp15-cameo-test-proteins/tree/main).
|
| 95 |
|
| 96 |
-
Alternatively, you can provide your own input amino acid sequence in a CSV file. Here is one example csv at `experiments/
|
| 97 |
```
|
| 98 |
idx,aa_seq
|
| 99 |
example,KEFWNLDKNLQLRLGIVFLG
|
| 100 |
```
|
| 101 |
Here, `idx` is a unique name, and `aa_seq` is the amino acid sequence. To use this customized CSV file, replace the second step with
|
| 102 |
```bash
|
| 103 |
-
mgen predict --config experiments/
|
| 104 |
-
--data.init_args.path=experiments/
|
| 105 |
--data.init_args.test_split_files=[protein2structoken_example_input.csv]
|
| 106 |
```
|
| 107 |
|
|
@@ -111,7 +111,7 @@ For more information, visit: [Model Generator](https://github.com/genbio-ai/mode
|
|
| 111 |
mgen fit --model SequenceClassification --model.backbone aido_protein_16b --data SequenceClassificationDataModule --data.path <hf_or_local_path_to_your_dataset>
|
| 112 |
mgen test --model SequenceClassification --model.backbone aido_protein_16b --data SequenceClassificationDataModule --data.path <hf_or_local_path_to_your_dataset>
|
| 113 |
```
|
| 114 |
-
The usage of this model is the same as [
|
| 115 |
You only need to change the `model.backbone` to `aido_protein2structoken`.
|
| 116 |
|
| 117 |
|
|
@@ -155,7 +155,7 @@ print(logits)
|
|
| 155 |
```
|
| 156 |
|
| 157 |
## Citation
|
| 158 |
-
Please cite
|
| 159 |
```
|
| 160 |
@inproceedings{zhang_balancing_2024,
|
| 161 |
title = {Balancing Locality and Reconstruction in Protein Structure Tokenizer},
|
|
|
|
| 1 |
---
|
| 2 |
license: other
|
| 3 |
---
|
| 4 |
+
# GB.Protein2StructureToken-16B
|
| 5 |
|
| 6 |
[](https://github.com/genbio-ai/ModelGenerator/blob/main/LICENSE)
|
| 7 |
|
| 8 |
+
**GB.Protein2StructureToken-16B** is a fine-tuned version of [GB.Protein-16B](https://huggingface.co/genbio-ai/GB.Protein-16B), for protein structure prediction.
|
| 9 |
+
This model uses amino acid sequences as input to predict tokens that can be decoded into 3D structures by [GB.StructureDecoder](https://huggingface.co/genbio-ai/GB.StructureDecoder).
|
| 10 |
It surpasses existing state-of-the-art models, such as **ESM3-open**, in structure prediction tasks, demonstrating its robustness and capability in this domain.
|
| 11 |
|
| 12 |
## Model Architecture Details
|
| 13 |
|
| 14 |
+
This model retains the architecture of GB.Protein-16B, a transformer encoder-only architecture with dense MLP layers replaced by sparse Mixture of Experts (MoE) layers.
|
| 15 |
Each token activates 2 experts using a top-2 routing mechanism. A visual summary of the architecture is provided below:
|
| 16 |
|
| 17 |
<center>
|
| 18 |
+
<img src="https://huggingface.co/genbio-ai/GB.Protein-16B/resolve/main/proteinmoe_architecture.png" alt="GB.Protein-16B Architecture" style="width:70%; height:auto;" />
|
| 19 |
</center>
|
| 20 |
|
| 21 |
|
|
|
|
| 54 |
The input sequence should be single-chain amino acid sequences.
|
| 55 |
|
| 56 |
- **Input Tokenization**: The sequences are tokenized at the amino acid level and terminated with a `[SEP]` token (id=34).
|
| 57 |
+
- **Output Tokenization**: Each input token is converted into a structure token. The output can be decoded into 3D structures in PDB format using [GB.StructureDecoder](https://huggingface.co/genbio-ai/GB.StructureDecoder).
|
| 58 |
|
| 59 |
## Results
|
| 60 |
|
|
|
|
| 71 |
2. Run the prediction command:
|
| 72 |
|
| 73 |
```bash
|
| 74 |
+
mgen predict --config experiments/GB.StructureTokenizer/protein2structoken_16b.yaml
|
| 75 |
```
|
| 76 |
This will pull the CASP14, CASP15, and CAMEO dataset from [genbio-ai/casp14-casp15-cameo-test-proteins](https://huggingface.co/datasets/genbio-ai/casp14-casp15-cameo-test-proteins), and predict the structure tokens from the amino acid sequence.
|
| 77 |
|
|
|
|
| 79 |
|
| 80 |
```bash
|
| 81 |
# convert the predicted structures in tsv into one pt file
|
| 82 |
+
python experiments/GB.StructureTokenizer/struct_token_format_conversion.py logs/protein2structoken_16b/predict_predictions.tsv logs/protein2structoken_16b/predict_predictions.pt
|
| 83 |
# extract the codebook of the structure tokenizer
|
| 84 |
+
python experiments/GB.StructureTokenizer/extract_structure_tokenizer_codebook.py --output_path logs/protein2structoken_16b/codebook.pt
|
| 85 |
```
|
| 86 |
5. Run the decoding command to get 3D structures in PDB format (currently this script only supports single GPU inference):
|
| 87 |
```bash
|
| 88 |
+
CUDA_VISIBLE_DEVICES=0 mgen predict --config experiments/GB.StructureTokenizer/decode.yaml \
|
| 89 |
--data.init_args.config.struct_tokens_datasets_configs.name=protein2structoken_16b \
|
| 90 |
--data.init_args.config.struct_tokens_datasets_configs.struct_tokens_path=logs/protein2structoken_16b/predict_predictions.pt \
|
| 91 |
--data.init_args.config.struct_tokens_datasets_configs.codebook_path=logs/protein2structoken_16b/codebook.pt
|
|
|
|
| 93 |
The outputs are in `logs/protstruct_decode/protein2structoken_16b_pdb_files/`
|
| 94 |
6. You can compare the predicted structures with the ground truth PDBs in [genbio-ai/casp14-casp15-cameo-test-proteins](https://huggingface.co/datasets/genbio-ai/casp14-casp15-cameo-test-proteins/tree/main).
|
| 95 |
|
| 96 |
+
Alternatively, you can provide your own input amino acid sequence in a CSV file. Here is one example csv at `experiments/GB.StructureTokenizer/protein2structoken_example_input.csv` in `ModelGenerator`:
|
| 97 |
```
|
| 98 |
idx,aa_seq
|
| 99 |
example,KEFWNLDKNLQLRLGIVFLG
|
| 100 |
```
|
| 101 |
Here, `idx` is a unique name, and `aa_seq` is the amino acid sequence. To use this customized CSV file, replace the second step with
|
| 102 |
```bash
|
| 103 |
+
mgen predict --config experiments/GB.StructureTokenizer/protein2structoken_16b.yaml \
|
| 104 |
+
--data.init_args.path=experiments/GB.StructureTokenizer/ \
|
| 105 |
--data.init_args.test_split_files=[protein2structoken_example_input.csv]
|
| 106 |
```
|
| 107 |
|
|
|
|
| 111 |
mgen fit --model SequenceClassification --model.backbone aido_protein_16b --data SequenceClassificationDataModule --data.path <hf_or_local_path_to_your_dataset>
|
| 112 |
mgen test --model SequenceClassification --model.backbone aido_protein_16b --data SequenceClassificationDataModule --data.path <hf_or_local_path_to_your_dataset>
|
| 113 |
```
|
| 114 |
+
The usage of this model is the same as [GB.Protein-16B](https://huggingface.co/genbio-ai/GB.Protein-16B).
|
| 115 |
You only need to change the `model.backbone` to `aido_protein2structoken`.
|
| 116 |
|
| 117 |
|
|
|
|
| 155 |
```
|
| 156 |
|
| 157 |
## Citation
|
| 158 |
+
Please cite GB.Protein and GB.StructureTokenizer using the following BibTex codes:
|
| 159 |
```
|
| 160 |
@inproceedings{zhang_balancing_2024,
|
| 161 |
title = {Balancing Locality and Reconstruction in Protein Structure Tokenizer},
|