Text Generation
Transformers
Safetensors
English
gpt2
boris
nmai
250M
conversational
text-generation-inference
KlondikeDev commited on
Commit
2ecd396
·
0 Parent(s):

Duplicate from KSP-NMAI/Boris-250M

Browse files
.gitattributes ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ benchmark.png filter=lfs diff=lfs merge=lfs -text
37
+ Boris-125M.png filter=lfs diff=lfs merge=lfs -text
38
+ Boris-250M.png filter=lfs diff=lfs merge=lfs -text
39
+ Screenshot[[:space:]]From[[:space:]]2026-07-23[[:space:]]13-05-50.png filter=lfs diff=lfs merge=lfs -text
40
+ benchmarks.png filter=lfs diff=lfs merge=lfs -text
Boris-250M.png ADDED

Git LFS Details

  • SHA256: 75b3ffda9d6392a034b78ba14f963f7171c4a43fbc9fe642c231edfd6df4006c
  • Pointer size: 131 Bytes
  • Size of remote file: 784 kB
README.md ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - HuggingFaceFW/fineweb-edu
5
+ - mlfoundations/dclm-baseline-1.0-parquet
6
+ language:
7
+ - en
8
+ tags:
9
+ - boris
10
+ - nmai
11
+ - 250M
12
+ ---
13
+
14
+ ![Boris](Boris-250M.png)
15
+
16
+ # Boris-250M
17
+ Boris-250M is a Large Language Model created by New Millennium Artificial Intelligence, trained on 5.01B tokens and contains 250 million parameters.
18
+
19
+ Trained for ~105:56:17 on one RTX 3060.
20
+
21
+ # Final Stats:
22
+ ### Loss - 3.0693
23
+ ### Grad Norm - 0.225
24
+ ### Learning Rate - 6.00e-05
25
+
26
+
27
+ ![Benchmarks](benchmarks.png)
28
+
29
+
30
+ ## Details
31
+
32
+ | | |
33
+ |---|---|
34
+ | Architecture | GPT-2 (pre-LN, learned positional embeddings, tied embeddings) |
35
+ | Parameters | ~250.4M |
36
+ | Layers / heads / d_model | 12 / 18 / 1152 |
37
+ | Context length | 1024 |
38
+ | Training tokens | 5.01B (20.0 tokens/param, Chinchilla-optimal) |
39
+ | Training data | 60% FineWeb-Edu / 40% DCLM |
40
+ | Vocab | 50304 (GPT-NeoX-20B BPE, padded) |
41
+ | Tokenizer | `EleutherAI/gpt-neox-20b` |
42
+ | Precision | trained in bf16 autocast with fp32 master weights |
43
+
44
+ Base model; not instruction-tuned or aligned. Outputs may be inaccurate or
45
+ offensive.
46
+
47
+ ## Copyright & License:
48
+
49
+ *Copyright 2026 Joseph Jones*
50
+
51
+ This project and all associated files (the "Work") are licensed under the Apache License, Version 2.0 (the "License"); you may not use this project except in compliance with the License. You may obtain a copy of the License at:
52
+
53
+ http://www.apache.org/licenses/LICENSE-2.0
54
+ Unless required by applicable law or agreed to in writing, software
55
+ distributed under the License is distributed on an "AS IS" BASIS,
56
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
57
+ See the License for the specific language governing permissions and
58
+ limitations under the License.
benchmarks.png ADDED

Git LFS Details

  • SHA256: f05976a9c579dac383e1b35e0aeda95793c17d1522bcaf15bae1e60776915752
  • Pointer size: 131 Bytes
  • Size of remote file: 190 kB
config.json ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "activation_function": "gelu",
3
+ "add_cross_attention": false,
4
+ "architectures": [
5
+ "GPT2LMHeadModel"
6
+ ],
7
+ "attn_pdrop": 0.0,
8
+ "bos_token_id": 0,
9
+ "dtype": "float32",
10
+ "embd_pdrop": 0.0,
11
+ "eos_token_id": 0,
12
+ "initializer_range": 0.02,
13
+ "layer_norm_epsilon": 1e-05,
14
+ "model_type": "gpt2",
15
+ "n_ctx": 1024,
16
+ "n_embd": 1152,
17
+ "n_head": 18,
18
+ "n_inner": null,
19
+ "n_layer": 12,
20
+ "n_positions": 1024,
21
+ "pad_token_id": null,
22
+ "reorder_and_upcast_attn": false,
23
+ "resid_pdrop": 0.0,
24
+ "scale_attn_by_inverse_layer_idx": false,
25
+ "scale_attn_weights": true,
26
+ "summary_activation": null,
27
+ "summary_first_dropout": 0.1,
28
+ "summary_proj_to_labels": true,
29
+ "summary_type": "cls_index",
30
+ "summary_use_proj": true,
31
+ "tie_word_embeddings": true,
32
+ "transformers_version": "5.13.1",
33
+ "use_cache": true,
34
+ "vocab_size": 50304
35
+ }
generation_config.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 0,
4
+ "eos_token_id": 0,
5
+ "output_attentions": false,
6
+ "output_hidden_states": false,
7
+ "transformers_version": "5.13.1",
8
+ "use_cache": true
9
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:61576c4e569c1a3e4c4983c1ba78b72d49a4c581120f88a4249fb9383bb74993
3
+ size 1001674568
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": "<|endoftext|>",
5
+ "eos_token": "<|endoftext|>",
6
+ "errors": "replace",
7
+ "is_local": false,
8
+ "local_files_only": false,
9
+ "model_max_length": 1000000000000000019884624838656,
10
+ "pad_token": "<|padding|>",
11
+ "tokenizer_class": "GPTNeoXTokenizer",
12
+ "trim_offsets": true,
13
+ "unk_token": "<|endoftext|>"
14
+ }