diff --git "a/train.log" "b/train.log"
new file mode 100644--- /dev/null
+++ "b/train.log"
@@ -0,0 +1,13172 @@
+[2025-06-30 06:36:55,989] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-30 06:36:58,463] [WARNING] [runner.py:215:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only.
+Detected VISIBLE_DEVICES=0,1,2,3,4,5,6,7: setting --include=localhost:0,1,2,3,4,5,6,7
+[2025-06-30 06:36:58,464] [INFO] [runner.py:607:main] cmd = /root/miniconda3/envs/openrlhf/bin/python -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMCwgMSwgMiwgMywgNCwgNSwgNiwgN119 --master_addr=127.0.0.1 --master_port=29500 --module --enable_each_rank_log=None openrlhf.cli.train_sft --max_len 4096 --dataset /apdcephfs/share_300000800/user/yulaizhao/codes/rl_grounding/0304/make_reward_sft/reward_data_180k_augmented.json --input_key query --output_key output --apply_chat_template --train_batch_size 128 --micro_train_batch_size 4 --pretrain /apdcephfs/share_300000800/user/dyu/share/model/pretrain/Qwen/Qwen2.5-7B-Instruct --save_path /apdcephfs/share_300000800/user/yulaizhao/rlhf/checkpoints/RLVR_sft_augmented_data_180k --save_steps -1 --logging_steps 1 --eval_steps -1 --zero_stage 3 --max_epochs 1 --bf16 --flash_attn --learning_rate 5e-6 --packing_samples --use_wandb true --wandb_project magic_spell --wandb_run_name RLVR_sft_augmented_data_180k_20250630-0636
+[2025-06-30 06:36:59,956] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-30 06:37:02,414] [INFO] [launch.py:139:main] 0 NCCL_HOME=/usr/local/tccl
+[2025-06-30 06:37:02,414] [INFO] [launch.py:146:main] WORLD INFO DICT: {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]}
+[2025-06-30 06:37:02,414] [INFO] [launch.py:152:main] nnodes=1, num_local_procs=8, node_rank=0
+[2025-06-30 06:37:02,414] [INFO] [launch.py:163:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]})
+[2025-06-30 06:37:02,414] [INFO] [launch.py:164:main] dist_world_size=8
+[2025-06-30 06:37:02,414] [INFO] [launch.py:168:main] Setting CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
+[2025-06-30 06:37:02,415] [INFO] [launch.py:256:main] process 856597 spawned with command: ['/root/miniconda3/envs/openrlhf/bin/python', '-u', '-m', 'openrlhf.cli.train_sft', '--local_rank=0', '--max_len', '4096', '--dataset', '/apdcephfs/share_300000800/user/yulaizhao/codes/rl_grounding/0304/make_reward_sft/reward_data_180k_augmented.json', '--input_key', 'query', '--output_key', 'output', '--apply_chat_template', '--train_batch_size', '128', '--micro_train_batch_size', '4', '--pretrain', '/apdcephfs/share_300000800/user/dyu/share/model/pretrain/Qwen/Qwen2.5-7B-Instruct', '--save_path', '/apdcephfs/share_300000800/user/yulaizhao/rlhf/checkpoints/RLVR_sft_augmented_data_180k', '--save_steps', '-1', '--logging_steps', '1', '--eval_steps', '-1', '--zero_stage', '3', '--max_epochs', '1', '--bf16', '--flash_attn', '--learning_rate', '5e-6', '--packing_samples', '--use_wandb', 'true', '--wandb_project', 'magic_spell', '--wandb_run_name', 'RLVR_sft_augmented_data_180k_20250630-0636']
+[2025-06-30 06:37:02,416] [INFO] [launch.py:256:main] process 856598 spawned with command: ['/root/miniconda3/envs/openrlhf/bin/python', '-u', '-m', 'openrlhf.cli.train_sft', '--local_rank=1', '--max_len', '4096', '--dataset', '/apdcephfs/share_300000800/user/yulaizhao/codes/rl_grounding/0304/make_reward_sft/reward_data_180k_augmented.json', '--input_key', 'query', '--output_key', 'output', '--apply_chat_template', '--train_batch_size', '128', '--micro_train_batch_size', '4', '--pretrain', '/apdcephfs/share_300000800/user/dyu/share/model/pretrain/Qwen/Qwen2.5-7B-Instruct', '--save_path', '/apdcephfs/share_300000800/user/yulaizhao/rlhf/checkpoints/RLVR_sft_augmented_data_180k', '--save_steps', '-1', '--logging_steps', '1', '--eval_steps', '-1', '--zero_stage', '3', '--max_epochs', '1', '--bf16', '--flash_attn', '--learning_rate', '5e-6', '--packing_samples', '--use_wandb', 'true', '--wandb_project', 'magic_spell', '--wandb_run_name', 'RLVR_sft_augmented_data_180k_20250630-0636']
+[2025-06-30 06:37:02,417] [INFO] [launch.py:256:main] process 856599 spawned with command: ['/root/miniconda3/envs/openrlhf/bin/python', '-u', '-m', 'openrlhf.cli.train_sft', '--local_rank=2', '--max_len', '4096', '--dataset', '/apdcephfs/share_300000800/user/yulaizhao/codes/rl_grounding/0304/make_reward_sft/reward_data_180k_augmented.json', '--input_key', 'query', '--output_key', 'output', '--apply_chat_template', '--train_batch_size', '128', '--micro_train_batch_size', '4', '--pretrain', '/apdcephfs/share_300000800/user/dyu/share/model/pretrain/Qwen/Qwen2.5-7B-Instruct', '--save_path', '/apdcephfs/share_300000800/user/yulaizhao/rlhf/checkpoints/RLVR_sft_augmented_data_180k', '--save_steps', '-1', '--logging_steps', '1', '--eval_steps', '-1', '--zero_stage', '3', '--max_epochs', '1', '--bf16', '--flash_attn', '--learning_rate', '5e-6', '--packing_samples', '--use_wandb', 'true', '--wandb_project', 'magic_spell', '--wandb_run_name', 'RLVR_sft_augmented_data_180k_20250630-0636']
+[2025-06-30 06:37:02,418] [INFO] [launch.py:256:main] process 856600 spawned with command: ['/root/miniconda3/envs/openrlhf/bin/python', '-u', '-m', 'openrlhf.cli.train_sft', '--local_rank=3', '--max_len', '4096', '--dataset', '/apdcephfs/share_300000800/user/yulaizhao/codes/rl_grounding/0304/make_reward_sft/reward_data_180k_augmented.json', '--input_key', 'query', '--output_key', 'output', '--apply_chat_template', '--train_batch_size', '128', '--micro_train_batch_size', '4', '--pretrain', '/apdcephfs/share_300000800/user/dyu/share/model/pretrain/Qwen/Qwen2.5-7B-Instruct', '--save_path', '/apdcephfs/share_300000800/user/yulaizhao/rlhf/checkpoints/RLVR_sft_augmented_data_180k', '--save_steps', '-1', '--logging_steps', '1', '--eval_steps', '-1', '--zero_stage', '3', '--max_epochs', '1', '--bf16', '--flash_attn', '--learning_rate', '5e-6', '--packing_samples', '--use_wandb', 'true', '--wandb_project', 'magic_spell', '--wandb_run_name', 'RLVR_sft_augmented_data_180k_20250630-0636']
+[2025-06-30 06:37:02,418] [INFO] [launch.py:256:main] process 856601 spawned with command: ['/root/miniconda3/envs/openrlhf/bin/python', '-u', '-m', 'openrlhf.cli.train_sft', '--local_rank=4', '--max_len', '4096', '--dataset', '/apdcephfs/share_300000800/user/yulaizhao/codes/rl_grounding/0304/make_reward_sft/reward_data_180k_augmented.json', '--input_key', 'query', '--output_key', 'output', '--apply_chat_template', '--train_batch_size', '128', '--micro_train_batch_size', '4', '--pretrain', '/apdcephfs/share_300000800/user/dyu/share/model/pretrain/Qwen/Qwen2.5-7B-Instruct', '--save_path', '/apdcephfs/share_300000800/user/yulaizhao/rlhf/checkpoints/RLVR_sft_augmented_data_180k', '--save_steps', '-1', '--logging_steps', '1', '--eval_steps', '-1', '--zero_stage', '3', '--max_epochs', '1', '--bf16', '--flash_attn', '--learning_rate', '5e-6', '--packing_samples', '--use_wandb', 'true', '--wandb_project', 'magic_spell', '--wandb_run_name', 'RLVR_sft_augmented_data_180k_20250630-0636']
+[2025-06-30 06:37:02,419] [INFO] [launch.py:256:main] process 856602 spawned with command: ['/root/miniconda3/envs/openrlhf/bin/python', '-u', '-m', 'openrlhf.cli.train_sft', '--local_rank=5', '--max_len', '4096', '--dataset', '/apdcephfs/share_300000800/user/yulaizhao/codes/rl_grounding/0304/make_reward_sft/reward_data_180k_augmented.json', '--input_key', 'query', '--output_key', 'output', '--apply_chat_template', '--train_batch_size', '128', '--micro_train_batch_size', '4', '--pretrain', '/apdcephfs/share_300000800/user/dyu/share/model/pretrain/Qwen/Qwen2.5-7B-Instruct', '--save_path', '/apdcephfs/share_300000800/user/yulaizhao/rlhf/checkpoints/RLVR_sft_augmented_data_180k', '--save_steps', '-1', '--logging_steps', '1', '--eval_steps', '-1', '--zero_stage', '3', '--max_epochs', '1', '--bf16', '--flash_attn', '--learning_rate', '5e-6', '--packing_samples', '--use_wandb', 'true', '--wandb_project', 'magic_spell', '--wandb_run_name', 'RLVR_sft_augmented_data_180k_20250630-0636']
+[2025-06-30 06:37:02,420] [INFO] [launch.py:256:main] process 856603 spawned with command: ['/root/miniconda3/envs/openrlhf/bin/python', '-u', '-m', 'openrlhf.cli.train_sft', '--local_rank=6', '--max_len', '4096', '--dataset', '/apdcephfs/share_300000800/user/yulaizhao/codes/rl_grounding/0304/make_reward_sft/reward_data_180k_augmented.json', '--input_key', 'query', '--output_key', 'output', '--apply_chat_template', '--train_batch_size', '128', '--micro_train_batch_size', '4', '--pretrain', '/apdcephfs/share_300000800/user/dyu/share/model/pretrain/Qwen/Qwen2.5-7B-Instruct', '--save_path', '/apdcephfs/share_300000800/user/yulaizhao/rlhf/checkpoints/RLVR_sft_augmented_data_180k', '--save_steps', '-1', '--logging_steps', '1', '--eval_steps', '-1', '--zero_stage', '3', '--max_epochs', '1', '--bf16', '--flash_attn', '--learning_rate', '5e-6', '--packing_samples', '--use_wandb', 'true', '--wandb_project', 'magic_spell', '--wandb_run_name', 'RLVR_sft_augmented_data_180k_20250630-0636']
+[2025-06-30 06:37:02,420] [INFO] [launch.py:256:main] process 856604 spawned with command: ['/root/miniconda3/envs/openrlhf/bin/python', '-u', '-m', 'openrlhf.cli.train_sft', '--local_rank=7', '--max_len', '4096', '--dataset', '/apdcephfs/share_300000800/user/yulaizhao/codes/rl_grounding/0304/make_reward_sft/reward_data_180k_augmented.json', '--input_key', 'query', '--output_key', 'output', '--apply_chat_template', '--train_batch_size', '128', '--micro_train_batch_size', '4', '--pretrain', '/apdcephfs/share_300000800/user/dyu/share/model/pretrain/Qwen/Qwen2.5-7B-Instruct', '--save_path', '/apdcephfs/share_300000800/user/yulaizhao/rlhf/checkpoints/RLVR_sft_augmented_data_180k', '--save_steps', '-1', '--logging_steps', '1', '--eval_steps', '-1', '--zero_stage', '3', '--max_epochs', '1', '--bf16', '--flash_attn', '--learning_rate', '5e-6', '--packing_samples', '--use_wandb', 'true', '--wandb_project', 'magic_spell', '--wandb_run_name', 'RLVR_sft_augmented_data_180k_20250630-0636']
+[2025-06-30 06:37:06,795] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-30 06:37:06,848] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-30 06:37:06,906] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-30 06:37:06,925] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-30 06:37:06,945] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-30 06:37:06,946] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-30 06:37:06,955] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-30 06:37:06,955] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
+[2025-06-30 06:37:08,074] [INFO] [comm.py:652:init_distributed] cdb=None
+[2025-06-30 06:37:08,207] [INFO] [comm.py:652:init_distributed] cdb=None
+[2025-06-30 06:37:08,207] [INFO] [comm.py:683:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
+[2025-06-30 06:37:08,452] [INFO] [comm.py:652:init_distributed] cdb=None
+[2025-06-30 06:37:08,489] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+[2025-06-30 06:37:08,692] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+[2025-06-30 06:37:08,751] [INFO] [comm.py:652:init_distributed] cdb=None
+[2025-06-30 06:37:08,753] [INFO] [comm.py:652:init_distributed] cdb=None
+[2025-06-30 06:37:08,754] [INFO] [comm.py:652:init_distributed] cdb=None
+[2025-06-30 06:37:08,754] [INFO] [comm.py:652:init_distributed] cdb=None
+[2025-06-30 06:37:08,755] [INFO] [comm.py:652:init_distributed] cdb=None
+[2025-06-30 06:37:08,791] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+[2025-06-30 06:37:08,794] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+[2025-06-30 06:37:08,795] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+[2025-06-30 06:37:08,795] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+[2025-06-30 06:37:08,796] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+[2025-06-30 06:37:08,796] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
+TENCENT64:856597:856597 [0] NCCL INFO Bootstrap : Using bond1:30.159.162.93<0>
+TENCENT64:856597:856597 [0] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
+TENCENT64:856597:856597 [0] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
+TENCENT64:856597:856597 [0] NCCL INFO NET/Plugin: Using internal network plugin.
+TENCENT64:856597:856597 [0] NCCL INFO cudaDriverVersion 12020
+NCCL version 2.21.5+cuda12.4
+TENCENT64:856601:856601 [4] NCCL INFO cudaDriverVersion 12020
+TENCENT64:856598:856598 [1] NCCL INFO cudaDriverVersion 12020
+TENCENT64:856604:856604 [7] NCCL INFO cudaDriverVersion 12020
+TENCENT64:856599:856599 [2] NCCL INFO cudaDriverVersion 12020
+TENCENT64:856602:856602 [5] NCCL INFO cudaDriverVersion 12020
+TENCENT64:856603:856603 [6] NCCL INFO cudaDriverVersion 12020
+TENCENT64:856600:856600 [3] NCCL INFO cudaDriverVersion 12020
+TENCENT64:856601:856601 [4] NCCL INFO Bootstrap : Using bond1:30.159.162.93<0>
+TENCENT64:856604:856604 [7] NCCL INFO Bootstrap : Using bond1:30.159.162.93<0>
+TENCENT64:856599:856599 [2] NCCL INFO Bootstrap : Using bond1:30.159.162.93<0>
+TENCENT64:856602:856602 [5] NCCL INFO Bootstrap : Using bond1:30.159.162.93<0>
+TENCENT64:856598:856598 [1] NCCL INFO Bootstrap : Using bond1:30.159.162.93<0>
+TENCENT64:856603:856603 [6] NCCL INFO Bootstrap : Using bond1:30.159.162.93<0>
+TENCENT64:856600:856600 [3] NCCL INFO Bootstrap : Using bond1:30.159.162.93<0>
+TENCENT64:856601:856601 [4] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
+TENCENT64:856601:856601 [4] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
+TENCENT64:856601:856601 [4] NCCL INFO NET/Plugin: Using internal network plugin.
+TENCENT64:856604:856604 [7] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
+TENCENT64:856604:856604 [7] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
+TENCENT64:856604:856604 [7] NCCL INFO NET/Plugin: Using internal network plugin.
+TENCENT64:856599:856599 [2] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
+TENCENT64:856599:856599 [2] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
+TENCENT64:856599:856599 [2] NCCL INFO NET/Plugin: Using internal network plugin.
+TENCENT64:856600:856600 [3] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
+TENCENT64:856600:856600 [3] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
+TENCENT64:856600:856600 [3] NCCL INFO NET/Plugin: Using internal network plugin.
+TENCENT64:856602:856602 [5] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
+TENCENT64:856602:856602 [5] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
+TENCENT64:856602:856602 [5] NCCL INFO NET/Plugin: Using internal network plugin.
+TENCENT64:856603:856603 [6] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
+TENCENT64:856603:856603 [6] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
+TENCENT64:856603:856603 [6] NCCL INFO NET/Plugin: Using internal network plugin.
+TENCENT64:856598:856598 [1] NCCL INFO NET/Plugin: No plugin found (libnccl-net.so)
+TENCENT64:856598:856598 [1] NCCL INFO NET/Plugin: Plugin load returned 2 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-net.so
+TENCENT64:856598:856598 [1] NCCL INFO NET/Plugin: Using internal network plugin.
+TENCENT64:856597:857871 [0] NCCL INFO NET/IB : Using [0]mlx5_bond_1:1/RoCE [1]mlx5_bond_2:1/RoCE [2]mlx5_bond_3:1/RoCE [3]mlx5_bond_4:1/RoCE [4]mlx5_bond_5:1/RoCE [5]mlx5_bond_6:1/RoCE [6]mlx5_bond_7:1/RoCE [7]mlx5_bond_8:1/RoCE [8]={[8] mlx5_0:1/RoCE, [9] mlx5_1:1/RoCE} [RO]; OOB bond1:30.159.162.93<0>
+TENCENT64:856597:857871 [0] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856597:857871 [0] NCCL INFO Using network IB
+TENCENT64:856604:857873 [7] NCCL INFO NET/IB : Using [0]mlx5_bond_1:1/RoCE [1]mlx5_bond_2:1/RoCE [2]mlx5_bond_3:1/RoCE [3]mlx5_bond_4:1/RoCE [4]mlx5_bond_5:1/RoCE [5]mlx5_bond_6:1/RoCE [6]mlx5_bond_7:1/RoCE [7]mlx5_bond_8:1/RoCE [8]={[8] mlx5_0:1/RoCE, [9] mlx5_1:1/RoCE} [RO]; OOB bond1:30.159.162.93<0>
+TENCENT64:856604:857873 [7] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856604:857873 [7] NCCL INFO Using network IB
+TENCENT64:856603:857878 [6] NCCL INFO NET/IB : Using [0]mlx5_bond_1:1/RoCE [1]mlx5_bond_2:1/RoCE [2]mlx5_bond_3:1/RoCE [3]mlx5_bond_4:1/RoCE [4]mlx5_bond_5:1/RoCE [5]mlx5_bond_6:1/RoCE [6]mlx5_bond_7:1/RoCE [7]mlx5_bond_8:1/RoCE [8]={[8] mlx5_0:1/RoCE, [9] mlx5_1:1/RoCE} [RO]; OOB bond1:30.159.162.93<0>
+TENCENT64:856603:857878 [6] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856603:857878 [6] NCCL INFO Using network IB
+TENCENT64:856602:857876 [5] NCCL INFO NET/IB : Using [0]mlx5_bond_1:1/RoCE [1]mlx5_bond_2:1/RoCE [2]mlx5_bond_3:1/RoCE [3]mlx5_bond_4:1/RoCE [4]mlx5_bond_5:1/RoCE [5]mlx5_bond_6:1/RoCE [6]mlx5_bond_7:1/RoCE [7]mlx5_bond_8:1/RoCE [8]={[8] mlx5_0:1/RoCE, [9] mlx5_1:1/RoCE} [RO]; OOB bond1:30.159.162.93<0>
+TENCENT64:856602:857876 [5] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856602:857876 [5] NCCL INFO Using network IB
+TENCENT64:856599:857874 [2] NCCL INFO NET/IB : Using [0]mlx5_bond_1:1/RoCE [1]mlx5_bond_2:1/RoCE [2]mlx5_bond_3:1/RoCE [3]mlx5_bond_4:1/RoCE [4]mlx5_bond_5:1/RoCE [5]mlx5_bond_6:1/RoCE [6]mlx5_bond_7:1/RoCE [7]mlx5_bond_8:1/RoCE [8]={[8] mlx5_0:1/RoCE, [9] mlx5_1:1/RoCE} [RO]; OOB bond1:30.159.162.93<0>
+TENCENT64:856601:857872 [4] NCCL INFO NET/IB : Using [0]mlx5_bond_1:1/RoCE [1]mlx5_bond_2:1/RoCE [2]mlx5_bond_3:1/RoCE [3]mlx5_bond_4:1/RoCE [4]mlx5_bond_5:1/RoCE [5]mlx5_bond_6:1/RoCE [6]mlx5_bond_7:1/RoCE [7]mlx5_bond_8:1/RoCE [8]={[8] mlx5_0:1/RoCE, [9] mlx5_1:1/RoCE} [RO]; OOB bond1:30.159.162.93<0>
+TENCENT64:856599:857874 [2] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856599:857874 [2] NCCL INFO Using network IB
+TENCENT64:856601:857872 [4] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856601:857872 [4] NCCL INFO Using network IB
+TENCENT64:856598:857877 [1] NCCL INFO NET/IB : Using [0]mlx5_bond_1:1/RoCE [1]mlx5_bond_2:1/RoCE [2]mlx5_bond_3:1/RoCE [3]mlx5_bond_4:1/RoCE [4]mlx5_bond_5:1/RoCE [5]mlx5_bond_6:1/RoCE [6]mlx5_bond_7:1/RoCE [7]mlx5_bond_8:1/RoCE [8]={[8] mlx5_0:1/RoCE, [9] mlx5_1:1/RoCE} [RO]; OOB bond1:30.159.162.93<0>
+TENCENT64:856600:857875 [3] NCCL INFO NET/IB : Using [0]mlx5_bond_1:1/RoCE [1]mlx5_bond_2:1/RoCE [2]mlx5_bond_3:1/RoCE [3]mlx5_bond_4:1/RoCE [4]mlx5_bond_5:1/RoCE [5]mlx5_bond_6:1/RoCE [6]mlx5_bond_7:1/RoCE [7]mlx5_bond_8:1/RoCE [8]={[8] mlx5_0:1/RoCE, [9] mlx5_1:1/RoCE} [RO]; OOB bond1:30.159.162.93<0>
+TENCENT64:856600:857875 [3] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856600:857875 [3] NCCL INFO Using network IB
+TENCENT64:856598:857877 [1] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856598:857877 [1] NCCL INFO Using network IB
+TENCENT64:856601:857872 [4] NCCL INFO ncclCommInitRank comm 0xbc11380 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId a2000 commId 0x4b26f6ae90d09515 - Init START
+TENCENT64:856602:857876 [5] NCCL INFO ncclCommInitRank comm 0xaf5f740 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId a7000 commId 0x4b26f6ae90d09515 - Init START
+TENCENT64:856600:857875 [3] NCCL INFO ncclCommInitRank comm 0xb785090 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 69000 commId 0x4b26f6ae90d09515 - Init START
+TENCENT64:856598:857877 [1] NCCL INFO ncclCommInitRank comm 0xac3d6d0 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 2b000 commId 0x4b26f6ae90d09515 - Init START
+TENCENT64:856604:857873 [7] NCCL INFO ncclCommInitRank comm 0xa8a21a0 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId e7000 commId 0x4b26f6ae90d09515 - Init START
+TENCENT64:856603:857878 [6] NCCL INFO ncclCommInitRank comm 0xada66d0 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId e1000 commId 0x4b26f6ae90d09515 - Init START
+TENCENT64:856599:857874 [2] NCCL INFO ncclCommInitRank comm 0xb7cd040 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 64000 commId 0x4b26f6ae90d09515 - Init START
+TENCENT64:856597:857871 [0] NCCL INFO ncclCommInitRank comm 0xa71bd30 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 25000 commId 0x4b26f6ae90d09515 - Init START
+TENCENT64:856597:857871 [0] NCCL INFO Setting affinity for GPU 0 to ff,00000000,00000000,00000000,000000ff,00000000
+TENCENT64:856597:857871 [0] NCCL INFO NVLS multicast support is not available on dev 0
+TENCENT64:856598:857877 [1] NCCL INFO Setting affinity for GPU 1 to ff,00000000,00000000,00000000,000000ff,00000000
+TENCENT64:856598:857877 [1] NCCL INFO NVLS multicast support is not available on dev 1
+TENCENT64:856604:857873 [7] NCCL INFO Setting affinity for GPU 7 to ff,00000000,00000000,00000000,000000ff,00000000,00000000
+TENCENT64:856604:857873 [7] NCCL INFO NVLS multicast support is not available on dev 7
+TENCENT64:856599:857874 [2] NCCL INFO Setting affinity for GPU 2 to ff,00000000,00000000,00000000,000000ff
+TENCENT64:856599:857874 [2] NCCL INFO NVLS multicast support is not available on dev 2
+TENCENT64:856600:857875 [3] NCCL INFO Setting affinity for GPU 3 to ff,00000000,00000000,00000000,000000ff
+TENCENT64:856600:857875 [3] NCCL INFO NVLS multicast support is not available on dev 3
+TENCENT64:856601:857872 [4] NCCL INFO Setting affinity for GPU 4 to ff,00000000,00000000,00000000,000000ff,00000000,00000000,00000000
+TENCENT64:856601:857872 [4] NCCL INFO NVLS multicast support is not available on dev 4
+TENCENT64:856603:857878 [6] NCCL INFO Setting affinity for GPU 6 to ff,00000000,00000000,00000000,000000ff,00000000,00000000
+TENCENT64:856603:857878 [6] NCCL INFO NVLS multicast support is not available on dev 6
+TENCENT64:856602:857876 [5] NCCL INFO Setting affinity for GPU 5 to ff,00000000,00000000,00000000,000000ff,00000000,00000000,00000000
+TENCENT64:856602:857876 [5] NCCL INFO NVLS multicast support is not available on dev 5
+TENCENT64:856598:857877 [1] NCCL INFO comm 0xac3d6d0 rank 1 nRanks 8 nNodes 1 localRanks 8 localRank 1 MNNVL 0
+TENCENT64:856599:857874 [2] NCCL INFO comm 0xb7cd040 rank 2 nRanks 8 nNodes 1 localRanks 8 localRank 2 MNNVL 0
+TENCENT64:856597:857871 [0] NCCL INFO comm 0xa71bd30 rank 0 nRanks 8 nNodes 1 localRanks 8 localRank 0 MNNVL 0
+TENCENT64:856602:857876 [5] NCCL INFO comm 0xaf5f740 rank 5 nRanks 8 nNodes 1 localRanks 8 localRank 5 MNNVL 0
+TENCENT64:856603:857878 [6] NCCL INFO comm 0xada66d0 rank 6 nRanks 8 nNodes 1 localRanks 8 localRank 6 MNNVL 0
+TENCENT64:856601:857872 [4] NCCL INFO comm 0xbc11380 rank 4 nRanks 8 nNodes 1 localRanks 8 localRank 4 MNNVL 0
+TENCENT64:856600:857875 [3] NCCL INFO comm 0xb785090 rank 3 nRanks 8 nNodes 1 localRanks 8 localRank 3 MNNVL 0
+TENCENT64:856604:857873 [7] NCCL INFO comm 0xa8a21a0 rank 7 nRanks 8 nNodes 1 localRanks 8 localRank 7 MNNVL 0
+TENCENT64:856597:857871 [0] NCCL INFO Channel 00/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:857871 [0] NCCL INFO Channel 01/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:857871 [0] NCCL INFO Channel 02/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:857871 [0] NCCL INFO Channel 03/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:857871 [0] NCCL INFO Channel 04/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:857871 [0] NCCL INFO Channel 05/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856598:857877 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0
+TENCENT64:856597:857871 [0] NCCL INFO Channel 06/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856602:857876 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4
+TENCENT64:856599:857874 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1
+TENCENT64:856603:857878 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5
+TENCENT64:856598:857877 [1] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856604:857873 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6
+TENCENT64:856597:857871 [0] NCCL INFO Channel 07/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856601:857872 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3
+TENCENT64:856600:857875 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2
+TENCENT64:856601:857872 [4] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856602:857876 [5] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856599:857874 [2] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856603:857878 [6] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856604:857873 [7] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856597:857871 [0] NCCL INFO Channel 08/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856600:857875 [3] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856597:857871 [0] NCCL INFO Channel 09/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:857871 [0] NCCL INFO Channel 10/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:857871 [0] NCCL INFO Channel 11/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:857871 [0] NCCL INFO Channel 12/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:857871 [0] NCCL INFO Channel 13/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:857871 [0] NCCL INFO Channel 14/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:857871 [0] NCCL INFO Channel 15/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:857871 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1
+TENCENT64:856597:857871 [0] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856598:857877 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 00/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 01/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 02/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 03/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 00/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 04/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 05/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 06/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 07/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 08/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 09/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 01/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 10/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 02/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 11/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 03/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 12/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 04/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 13/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 05/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 14/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 06/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 15/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 07/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 08/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 09/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 10/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 11/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 12/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 13/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 14/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 15/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Connected all rings
+TENCENT64:856597:857871 [0] NCCL INFO Connected all rings
+TENCENT64:856601:857872 [4] NCCL INFO Connected all rings
+TENCENT64:856599:857874 [2] NCCL INFO Connected all rings
+TENCENT64:856600:857875 [3] NCCL INFO Connected all rings
+TENCENT64:856598:857877 [1] NCCL INFO Channel 00/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 01/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 02/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 03/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 04/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Connected all rings
+TENCENT64:856604:857873 [7] NCCL INFO Channel 00/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Connected all rings
+TENCENT64:856603:857878 [6] NCCL INFO Connected all rings
+TENCENT64:856598:857877 [1] NCCL INFO Channel 05/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 01/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 06/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 02/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 07/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 03/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 08/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 04/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 09/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 05/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 10/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 06/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 11/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 07/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 12/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 08/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 13/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 09/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 00/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 00/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 14/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 00/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 10/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 01/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 01/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856598:857877 [1] NCCL INFO Channel 15/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 01/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 11/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 02/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 02/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 02/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 12/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 03/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 03/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 03/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 13/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 04/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 04/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 04/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 14/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 05/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 05/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:857873 [7] NCCL INFO Channel 15/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 06/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 06/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 07/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 07/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 08/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 08/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 09/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 09/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 05/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 10/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 06/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 10/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 00/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 07/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 00/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 11/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 01/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 08/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 01/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 12/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 02/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 09/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 02/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 13/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 03/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 10/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 03/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 14/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 04/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 11/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 04/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:857872 [4] NCCL INFO Channel 15/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 05/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 12/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 05/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 06/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 13/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 06/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 07/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 14/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 07/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 11/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 08/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 12/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856599:857874 [2] NCCL INFO Channel 15/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 09/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 13/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 10/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 14/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 08/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 11/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856600:857875 [3] NCCL INFO Channel 15/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 09/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 12/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 10/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 13/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 11/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 14/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 12/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:857878 [6] NCCL INFO Channel 15/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 13/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 14/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856602:857876 [5] NCCL INFO Channel 15/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:857871 [0] NCCL INFO Connected all trees
+TENCENT64:856597:857871 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856597:857871 [0] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856598:857877 [1] NCCL INFO Connected all trees
+TENCENT64:856598:857877 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856598:857877 [1] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856599:857874 [2] NCCL INFO Connected all trees
+TENCENT64:856599:857874 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856599:857874 [2] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856600:857875 [3] NCCL INFO Connected all trees
+TENCENT64:856600:857875 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856600:857875 [3] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856601:857872 [4] NCCL INFO Connected all trees
+TENCENT64:856601:857872 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856601:857872 [4] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856602:857876 [5] NCCL INFO Connected all trees
+TENCENT64:856602:857876 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856602:857876 [5] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856604:857873 [7] NCCL INFO Connected all trees
+TENCENT64:856604:857873 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856604:857873 [7] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856603:857878 [6] NCCL INFO Connected all trees
+TENCENT64:856603:857878 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856603:857878 [6] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856603:857878 [6] NCCL INFO TUNER/Plugin: Plugin load returned 11 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-tuner.so
+TENCENT64:856603:857878 [6] NCCL INFO TUNER/Plugin: Using internal tuner plugin.
+TENCENT64:856603:857878 [6] NCCL INFO ncclCommInitRank comm 0xada66d0 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId e1000 commId 0x4b26f6ae90d09515 - Init COMPLETE
+TENCENT64:856597:857871 [0] NCCL INFO TUNER/Plugin: Plugin load returned 11 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-tuner.so
+TENCENT64:856597:857871 [0] NCCL INFO TUNER/Plugin: Using internal tuner plugin.
+TENCENT64:856597:857871 [0] NCCL INFO ncclCommInitRank comm 0xa71bd30 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 25000 commId 0x4b26f6ae90d09515 - Init COMPLETE
+TENCENT64:856599:857874 [2] NCCL INFO TUNER/Plugin: Plugin load returned 11 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-tuner.so
+TENCENT64:856599:857874 [2] NCCL INFO TUNER/Plugin: Using internal tuner plugin.
+TENCENT64:856599:857874 [2] NCCL INFO ncclCommInitRank comm 0xb7cd040 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 64000 commId 0x4b26f6ae90d09515 - Init COMPLETE
+TENCENT64:856600:857875 [3] NCCL INFO TUNER/Plugin: Plugin load returned 11 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-tuner.so
+TENCENT64:856600:857875 [3] NCCL INFO TUNER/Plugin: Using internal tuner plugin.
+TENCENT64:856600:857875 [3] NCCL INFO ncclCommInitRank comm 0xb785090 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 69000 commId 0x4b26f6ae90d09515 - Init COMPLETE
+TENCENT64:856602:857876 [5] NCCL INFO TUNER/Plugin: Plugin load returned 11 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-tuner.so
+TENCENT64:856602:857876 [5] NCCL INFO TUNER/Plugin: Using internal tuner plugin.
+TENCENT64:856602:857876 [5] NCCL INFO ncclCommInitRank comm 0xaf5f740 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId a7000 commId 0x4b26f6ae90d09515 - Init COMPLETE
+TENCENT64:856601:857872 [4] NCCL INFO TUNER/Plugin: Plugin load returned 11 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-tuner.so
+TENCENT64:856601:857872 [4] NCCL INFO TUNER/Plugin: Using internal tuner plugin.
+TENCENT64:856601:857872 [4] NCCL INFO ncclCommInitRank comm 0xbc11380 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId a2000 commId 0x4b26f6ae90d09515 - Init COMPLETE
+TENCENT64:856604:857873 [7] NCCL INFO TUNER/Plugin: Plugin load returned 11 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-tuner.so
+TENCENT64:856604:857873 [7] NCCL INFO TUNER/Plugin: Using internal tuner plugin.
+TENCENT64:856604:857873 [7] NCCL INFO ncclCommInitRank comm 0xa8a21a0 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId e7000 commId 0x4b26f6ae90d09515 - Init COMPLETE
+TENCENT64:856598:857877 [1] NCCL INFO TUNER/Plugin: Plugin load returned 11 : libnccl-net.so: cannot open shared object file: No such file or directory : when loading libnccl-tuner.so
+TENCENT64:856598:857877 [1] NCCL INFO TUNER/Plugin: Using internal tuner plugin.
+TENCENT64:856598:857877 [1] NCCL INFO ncclCommInitRank comm 0xac3d6d0 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 2b000 commId 0x4b26f6ae90d09515 - Init COMPLETE
+[2025-06-30 06:37:11,243] [INFO] [partition_parameters.py:348:__exit__] finished initializing model - num_params = 339, num_elems = 7.62B
+
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:02,  1.45it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:02,  1.42it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:02,  1.41it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:02,  1.42it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:02,  1.40it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:02,  1.39it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:02,  1.37it/s]
Loading checkpoint shards:  25%|██▌       | 1/4 [00:00<00:02,  1.29it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:01<00:01,  1.23it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:01<00:01,  1.23it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:01<00:01,  1.22it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:01<00:01,  1.22it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:01<00:01,  1.22it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:01<00:01,  1.21it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:01<00:01,  1.16it/s]
Loading checkpoint shards:  50%|█████     | 2/4 [00:01<00:01,  1.10it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:02<00:00,  1.16it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:02<00:00,  1.15it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:02<00:00,  1.16it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:02<00:00,  1.15it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:02<00:00,  1.14it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:02<00:00,  1.13it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:02<00:00,  1.13it/s]
Loading checkpoint shards:  75%|███████▌  | 3/4 [00:02<00:00,  1.09it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.29it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.27it/s]
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.29it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.27it/s]
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.28it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.26it/s]
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.28it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.26it/s]
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.27it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.25it/s]
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.26it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.23it/s]
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.23it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.20it/s]
+
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.17it/s]
Loading checkpoint shards: 100%|██████████| 4/4 [00:03<00:00,  1.18it/s]
+Using /root/.cache/torch_extensions/py310_cu124 as PyTorch extensions root...
+Detected CUDA files, patching ldflags
+Emitting ninja build file /root/.cache/torch_extensions/py310_cu124/fused_adam/build.ninja...
+/root/miniconda3/envs/openrlhf/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1964: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
+If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
+  warnings.warn(
+Building extension module fused_adam...
+Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
+Using /root/.cache/torch_extensions/py310_cu124 as PyTorch extensions root...
+ninja: no work to do.
+Loading extension module fused_adam...
+Using /root/.cache/torch_extensions/py310_cu124 as PyTorch extensions root...
+Time to load fused_adam op: 0.027651548385620117 seconds
+Detected CUDA files, patching ldflags
+Emitting ninja build file /root/.cache/torch_extensions/py310_cu124/fused_adam/build.ninja...
+/root/miniconda3/envs/openrlhf/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1964: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
+If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
+  warnings.warn(
+Building extension module fused_adam...
+Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
+Using /root/.cache/torch_extensions/py310_cu124 as PyTorch extensions root...
+Using /root/.cache/torch_extensions/py310_cu124 as PyTorch extensions root...
+ninja: no work to do.
+Loading extension module fused_adam...
+Time to load fused_adam op: 0.03133273124694824 seconds
+Using /root/.cache/torch_extensions/py310_cu124 as PyTorch extensions root...
+Detected CUDA files, patching ldflags
+Emitting ninja build file /root/.cache/torch_extensions/py310_cu124/fused_adam/build.ninja...
+/root/miniconda3/envs/openrlhf/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1964: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
+If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
+  warnings.warn(
+Building extension module fused_adam...
+Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
+ninja: no work to do.
+Loading extension module fused_adam...
+Time to load fused_adam op: 0.030365467071533203 seconds
+Loading extension module fused_adam...
+Time to load fused_adam op: 0.10134530067443848 seconds
+Loading extension module fused_adam...
+Time to load fused_adam op: 0.10123777389526367 seconds
+Using /root/.cache/torch_extensions/py310_cu124 as PyTorch extensions root...
+Detected CUDA files, patching ldflags
+Emitting ninja build file /root/.cache/torch_extensions/py310_cu124/fused_adam/build.ninja...
+/root/miniconda3/envs/openrlhf/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1964: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
+If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
+  warnings.warn(
+Building extension module fused_adam...
+Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
+ninja: no work to do.
+Loading extension module fused_adam...
+Time to load fused_adam op: 0.03097367286682129 seconds
+Actor(
+  (model): Qwen2ForCausalLM(
+    (model): Qwen2Model(
+      (embed_tokens): Embedding(152064, 3584)
+      (layers): ModuleList(
+        (0-27): 28 x Qwen2DecoderLayer(
+          (self_attn): Qwen2Attention(
+            (q_proj): Linear(in_features=3584, out_features=3584, bias=True)
+            (k_proj): Linear(in_features=3584, out_features=512, bias=True)
+            (v_proj): Linear(in_features=3584, out_features=512, bias=True)
+            (o_proj): Linear(in_features=3584, out_features=3584, bias=False)
+          )
+          (mlp): Qwen2MLP(
+            (gate_proj): Linear(in_features=3584, out_features=18944, bias=False)
+            (up_proj): Linear(in_features=3584, out_features=18944, bias=False)
+            (down_proj): Linear(in_features=18944, out_features=3584, bias=False)
+            (act_fn): SiLU()
+          )
+          (input_layernorm): Qwen2RMSNorm((0,), eps=1e-06)
+          (post_attention_layernorm): Qwen2RMSNorm((0,), eps=1e-06)
+        )
+      )
+      (norm): Qwen2RMSNorm((0,), eps=1e-06)
+      (rotary_emb): Qwen2RotaryEmbedding()
+    )
+    (lm_head): Linear(in_features=3584, out_features=152064, bias=False)
+  )
+)
+Using /root/.cache/torch_extensions/py310_cu124 as PyTorch extensions root...
+Detected CUDA files, patching ldflags
+Emitting ninja build file /root/.cache/torch_extensions/py310_cu124/fused_adam/build.ninja...
+/root/miniconda3/envs/openrlhf/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1964: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
+If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
+  warnings.warn(
+Building extension module fused_adam...
+Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
+ninja: no work to do.
+Loading extension module fused_adam...
+Time to load fused_adam op: 0.34587979316711426 seconds
+dataset: /apdcephfs/share_300000800/user/yulaizhao/codes/rl_grounding/0304/make_reward_sft/reward_data_180k_augmented.json
+Loading extension module fused_adam...
+Time to load fused_adam op: 0.7020089626312256 seconds
+loaded /apdcephfs/share_300000800/user/yulaizhao/codes/rl_grounding/0304/make_reward_sft/reward_data_180k_augmented.json with data_files=/apdcephfs/share_300000800/user/yulaizhao/codes/rl_grounding/0304/make_reward_sft/reward_data_180k_augmented.json
+[Dataset({
+    features: ['query', 'output'],
+    num_rows: 179733
+})]
+
Map (num_proc=8):   0%|          | 0/179733 [00:00<?, ? examples/s]
Map (num_proc=8):   0%|          | 0/179733 [00:00<?, ? examples/s]
Map (num_proc=8):   0%|          | 0/179733 [00:00<?, ? examples/s]
Map (num_proc=8):   0%|          | 0/179733 [00:00<?, ? examples/s]
Map (num_proc=8):   0%|          | 0/179733 [00:00<?, ? examples/s]
Map (num_proc=8):   0%|          | 26/179733 [00:00<1:03:00, 47.54 examples/s]
Map (num_proc=8):   0%|          | 24/179733 [00:00<1:06:29, 45.04 examples/s]
Map (num_proc=8):   0%|          | 25/179733 [00:00<1:08:27, 43.75 examples/s]
Map (num_proc=8):   0%|          | 237/179733 [00:00<06:18, 474.50 examples/s]
Map (num_proc=8):   0%|          | 228/179733 [00:00<06:43, 444.86 examples/s]
Map (num_proc=8):   0%|          | 546/179733 [00:00<02:49, 1058.17 examples/s]
Map (num_proc=8):   0%|          | 293/179733 [00:00<05:25, 550.70 examples/s]
Map (num_proc=8):   0%|          | 478/179733 [00:00<03:18, 902.36 examples/s]
Map (num_proc=8):   1%|          | 924/179733 [00:00<01:44, 1713.13 examples/s]
Map (num_proc=8):   0%|          | 572/179733 [00:00<02:53, 1031.74 examples/s]
Map (num_proc=8):   0%|          | 24/179733 [00:00<1:14:52, 40.00 examples/s]
Map (num_proc=8):   0%|          | 885/179733 [00:00<01:47, 1656.63 examples/s]
Map (num_proc=8):   1%|          | 1426/179733 [00:00<01:09, 2561.48 examples/s]
Map (num_proc=8):   1%|          | 983/179733 [00:00<01:42, 1750.43 examples/s]
Map (num_proc=8):   0%|          | 252/179733 [00:00<06:27, 462.82 examples/s]
Map (num_proc=8):   1%|          | 1370/179733 [00:00<01:12, 2463.97 examples/s]
Map (num_proc=8):   1%|          | 2008/179733 [00:01<00:51, 3429.24 examples/s]
Map (num_proc=8):   1%|          | 1490/179733 [00:00<01:09, 2574.33 examples/s]
Map (num_proc=8):   0%|          | 524/179733 [00:00<03:10, 940.71 examples/s]
Map (num_proc=8):   0%|          | 24/179733 [00:00<1:10:08, 42.70 examples/s]
Map (num_proc=8):   1%|          | 1996/179733 [00:01<00:51, 3473.89 examples/s]
Map (num_proc=8):   2%|▏         | 2722/179733 [00:01<00:39, 4449.51 examples/s]
Map (num_proc=8):   1%|          | 2120/179733 [00:01<00:50, 3543.54 examples/s]
Map (num_proc=8):   1%|          | 942/179733 [00:00<01:45, 1688.84 examples/s]
Map (num_proc=8):   0%|          | 257/179733 [00:00<06:10, 484.40 examples/s]
Map (num_proc=8):   1%|▏         | 2687/179733 [00:01<00:40, 4388.96 examples/s]
Map (num_proc=8):   2%|▏         | 3418/179733 [00:01<00:34, 5145.56 examples/s]
Map (num_proc=8):   2%|▏         | 2821/179733 [00:01<00:39, 4468.67 examples/s]
Map (num_proc=8):   1%|          | 1427/179733 [00:01<01:12, 2464.17 examples/s]
Map (num_proc=8):   0%|          | 0/179733 [00:00<?, ? examples/s]
Map (num_proc=8):   0%|          | 529/179733 [00:00<03:05, 966.35 examples/s]
Map (num_proc=8):   2%|▏         | 3375/179733 [00:01<00:34, 5078.68 examples/s]
Map (num_proc=8):   2%|▏         | 4098/179733 [00:01<00:31, 5608.36 examples/s]
Map (num_proc=8):   2%|▏         | 3503/179733 [00:01<00:34, 5120.33 examples/s]
Map (num_proc=8):   1%|          | 2018/179733 [00:01<00:53, 3344.90 examples/s]
Map (num_proc=8):   1%|          | 978/179733 [00:00<01:39, 1795.48 examples/s]
Map (num_proc=8):   2%|▏         | 4064/179733 [00:01<00:31, 5579.48 examples/s]
Map (num_proc=8):   3%|▎         | 4813/179733 [00:01<00:29, 6030.30 examples/s]
Map (num_proc=8):   2%|▏         | 4192/179733 [00:01<00:31, 5594.01 examples/s]
Map (num_proc=8):   1%|▏         | 2683/179733 [00:01<00:41, 4224.70 examples/s]
Map (num_proc=8):   1%|          | 1490/179733 [00:00<01:07, 2625.20 examples/s]
Map (num_proc=8):   3%|▎         | 4745/179733 [00:01<00:29, 5926.66 examples/s]
Map (num_proc=8):   3%|▎         | 5506/179733 [00:01<00:27, 6275.91 examples/s]
Map (num_proc=8):   3%|▎         | 4876/179733 [00:01<00:29, 5911.23 examples/s]
Map (num_proc=8):   2%|▏         | 3361/179733 [00:01<00:35, 4922.10 examples/s]
Map (num_proc=8):   3%|▎         | 5436/179733 [00:01<00:28, 6194.67 examples/s]
Map (num_proc=8):   3%|▎         | 6200/179733 [00:01<00:26, 6457.34 examples/s]
Map (num_proc=8):   3%|▎         | 5567/179733 [00:01<00:28, 6176.11 examples/s]
Map (num_proc=8):   2%|▏         | 4030/179733 [00:01<00:32, 5408.42 examples/s]
Map (num_proc=8):   3%|▎         | 6130/179733 [00:01<00:27, 6385.83 examples/s]
Map (num_proc=8):   4%|▍         | 6903/179733 [00:01<00:26, 6590.68 examples/s]
Map (num_proc=8):   3%|▎         | 6255/179733 [00:01<00:27, 6362.51 examples/s]
Map (num_proc=8):   3%|▎         | 4726/179733 [00:01<00:29, 5842.61 examples/s]
Map (num_proc=8):   1%|          | 1858/179733 [00:01<01:40, 1772.81 examples/s]
Map (num_proc=8):   4%|▍         | 6803/179733 [00:01<00:26, 6453.19 examples/s]
Map (num_proc=8):   4%|▍         | 7582/179733 [00:01<00:25, 6624.74 examples/s]
Map (num_proc=8):   0%|          | 25/179733 [00:00<1:10:12, 42.66 examples/s]
Map (num_proc=8):   4%|▍         | 6923/179733 [00:01<00:26, 6428.88 examples/s]
Map (num_proc=8):   3%|▎         | 5406/179733 [00:01<00:28, 6105.43 examples/s]
Map (num_proc=8):   1%|▏         | 2512/179733 [00:01<01:06, 2660.93 examples/s]
Map (num_proc=8):   4%|▍         | 7466/179733 [00:01<00:26, 6492.86 examples/s]
Map (num_proc=8):   5%|▍         | 8287/179733 [00:01<00:25, 6705.83 examples/s]
Map (num_proc=8):   4%|▍         | 7616/179733 [00:01<00:26, 6431.84 examples/s]
Map (num_proc=8):   3%|▎         | 6089/179733 [00:01<00:27, 6273.72 examples/s]
Map (num_proc=8):   0%|          | 191/179733 [00:00<08:49, 339.18 examples/s]
Map (num_proc=8):   2%|▏         | 3204/179733 [00:01<00:49, 3551.89 examples/s]
Map (num_proc=8):   5%|▍         | 8149/179733 [00:01<00:26, 6549.06 examples/s]
Map (num_proc=8):   5%|▍         | 8986/179733 [00:02<00:25, 6738.69 examples/s]
Map (num_proc=8):   5%|▍         | 8299/179733 [00:02<00:26, 6511.08 examples/s]
Map (num_proc=8):   4%|▍         | 6758/179733 [00:01<00:27, 6373.78 examples/s]
Map (num_proc=8):   2%|▏         | 3876/179733 [00:01<00:41, 4271.93 examples/s]
Map (num_proc=8):   5%|▍         | 8816/179733 [00:02<00:26, 6528.23 examples/s]
Map (num_proc=8):   0%|          | 304/179733 [00:00<06:35, 453.51 examples/s]
Map (num_proc=8):   5%|▌         | 9680/179733 [00:02<00:25, 6766.11 examples/s]
Map (num_proc=8):   5%|▌         | 8997/179733 [00:02<00:25, 6614.42 examples/s]
Map (num_proc=8):   4%|▍         | 7417/179733 [00:01<00:26, 6408.45 examples/s]
Map (num_proc=8):   3%|▎         | 4578/179733 [00:01<00:35, 4943.81 examples/s]
Map (num_proc=8):   5%|▌         | 9508/179733 [00:02<00:25, 6625.50 examples/s]
Map (num_proc=8):   0%|          | 417/179733 [00:00<05:01, 595.00 examples/s]
Map (num_proc=8):   6%|▌         | 10380/179733 [00:02<00:24, 6796.98 examples/s]
Map (num_proc=8):   5%|▌         | 9666/179733 [00:02<00:25, 6614.44 examples/s]
Map (num_proc=8):   4%|▍         | 8080/179733 [00:02<00:26, 6435.30 examples/s]
Map (num_proc=8):   3%|▎         | 5277/179733 [00:01<00:31, 5475.33 examples/s]
Map (num_proc=8):   6%|▌         | 10204/179733 [00:02<00:25, 6659.24 examples/s]
Map (num_proc=8):   1%|          | 940/179733 [00:01<01:47, 1665.84 examples/s]
Map (num_proc=8):   6%|▌         | 11105/179733 [00:02<00:24, 6879.07 examples/s]
Map (num_proc=8):   6%|▌         | 10345/179733 [00:02<00:25, 6651.62 examples/s]
Map (num_proc=8):   5%|▍         | 8744/179733 [00:02<00:26, 6476.15 examples/s]
Map (num_proc=8):   3%|▎         | 5971/179733 [00:01<00:29, 5863.18 examples/s]
Map (num_proc=8):   6%|▌         | 10896/179733 [00:02<00:25, 6704.71 examples/s]
Map (num_proc=8):   1%|          | 1615/179733 [00:01<01:01, 2919.42 examples/s]
Map (num_proc=8):   7%|▋         | 11810/179733 [00:02<00:24, 6886.26 examples/s]
Map (num_proc=8):   6%|▌         | 11047/179733 [00:02<00:25, 6742.37 examples/s]
Map (num_proc=8):   5%|▌         | 9421/179733 [00:02<00:26, 6541.44 examples/s]
Map (num_proc=8):   4%|▎         | 6652/179733 [00:02<00:28, 6116.75 examples/s]
Map (num_proc=8):   6%|▋         | 11596/179733 [00:02<00:24, 6777.07 examples/s]
Map (num_proc=8):   1%|▏         | 2307/179733 [00:01<00:44, 3952.68 examples/s]
Map (num_proc=8):   7%|▋         | 12523/179733 [00:02<00:24, 6941.84 examples/s]
Map (num_proc=8):   7%|▋         | 11754/179733 [00:02<00:24, 6812.81 examples/s]
Map (num_proc=8):   6%|▌         | 10116/179733 [00:02<00:25, 6616.93 examples/s]
Map (num_proc=8):   4%|▍         | 7346/179733 [00:02<00:27, 6346.29 examples/s]
Map (num_proc=8):   7%|▋         | 12298/179733 [00:02<00:24, 6833.62 examples/s]
Map (num_proc=8):   2%|▏         | 2986/179733 [00:01<00:37, 4697.17 examples/s]
Map (num_proc=8):   7%|▋         | 13229/179733 [00:02<00:23, 6960.32 examples/s]
Map (num_proc=8):   7%|▋         | 12459/179733 [00:02<00:24, 6865.09 examples/s]
Map (num_proc=8):   6%|▌         | 10813/179733 [00:02<00:25, 6699.07 examples/s]
Map (num_proc=8):   0%|          | 0/179733 [00:00<?, ? examples/s]
Map (num_proc=8):   4%|▍         | 8046/179733 [00:02<00:26, 6505.09 examples/s]
Map (num_proc=8):   7%|▋         | 13005/179733 [00:02<00:24, 6870.11 examples/s]
Map (num_proc=8):   2%|▏         | 3679/179733 [00:01<00:33, 5307.54 examples/s]
Map (num_proc=8):   8%|▊         | 13931/179733 [00:02<00:23, 6945.69 examples/s]
Map (num_proc=8):   7%|▋         | 13148/179733 [00:02<00:24, 6864.79 examples/s]
Map (num_proc=8):   6%|▋         | 11527/179733 [00:02<00:24, 6768.24 examples/s]
Map (num_proc=8):   5%|▍         | 8749/179733 [00:02<00:25, 6612.30 examples/s]
Map (num_proc=8):   8%|▊         | 13694/179733 [00:02<00:24, 6844.42 examples/s]
Map (num_proc=8):   2%|▏         | 4388/179733 [00:01<00:30, 5797.68 examples/s]
Map (num_proc=8):   8%|▊         | 14662/179733 [00:02<00:23, 7011.87 examples/s]
Map (num_proc=8):   8%|▊         | 13843/179733 [00:02<00:24, 6870.78 examples/s]
Map (num_proc=8):   7%|▋         | 12222/179733 [00:02<00:24, 6808.79 examples/s]
Map (num_proc=8):   5%|▌         | 9428/179733 [00:02<00:25, 6641.81 examples/s]
Map (num_proc=8):   8%|▊         | 14391/179733 [00:02<00:24, 6833.13 examples/s]
Map (num_proc=8):   3%|▎         | 5100/179733 [00:01<00:28, 6174.37 examples/s]
Map (num_proc=8):   8%|▊         | 14576/179733 [00:02<00:24, 6872.14 examples/s]
Map (num_proc=8):   7%|▋         | 12916/179733 [00:02<00:24, 6808.71 examples/s]
Map (num_proc=8):   9%|▊         | 15395/179733 [00:03<00:24, 6670.90 examples/s]
Map (num_proc=8):   6%|▌         | 10140/179733 [00:02<00:25, 6753.74 examples/s]
Map (num_proc=8):   8%|▊         | 15083/179733 [00:02<00:24, 6841.58 examples/s]
Map (num_proc=8):   3%|▎         | 5764/179733 [00:01<00:27, 6281.41 examples/s]
Map (num_proc=8):   9%|▊         | 15294/179733 [00:03<00:23, 6932.16 examples/s]
Map (num_proc=8):   8%|▊         | 13639/179733 [00:02<00:24, 6789.93 examples/s]
Map (num_proc=8):   9%|▉         | 16113/179733 [00:03<00:24, 6560.20 examples/s]
Map (num_proc=8):   6%|▌         | 10827/179733 [00:02<00:24, 6776.74 examples/s]
Map (num_proc=8):   9%|▉         | 15797/179733 [00:03<00:23, 6869.22 examples/s]
Map (num_proc=8):   4%|▎         | 6461/179733 [00:01<00:26, 6463.20 examples/s]
Map (num_proc=8):   9%|▉         | 16006/179733 [00:03<00:23, 6954.71 examples/s]
Map (num_proc=8):   8%|▊         | 14360/179733 [00:02<00:24, 6782.95 examples/s]
Map (num_proc=8):   9%|▉         | 16790/179733 [00:03<00:24, 6587.31 examples/s]
Map (num_proc=8):   6%|▋         | 11523/179733 [00:02<00:24, 6824.74 examples/s]
Map (num_proc=8):   9%|▉         | 16510/179733 [00:03<00:23, 6886.12 examples/s]
Map (num_proc=8):   4%|▍         | 7152/179733 [00:01<00:26, 6587.16 examples/s]
Map (num_proc=8):   0%|          | 25/179733 [00:00<1:06:33, 45.00 examples/s]
Map (num_proc=8):   9%|▉         | 16731/179733 [00:03<00:24, 6712.40 examples/s]
Map (num_proc=8):  10%|▉         | 17475/179733 [00:03<00:24, 6640.12 examples/s]
Map (num_proc=8):   8%|▊         | 15067/179733 [00:03<00:24, 6661.20 examples/s]
Map (num_proc=8):   7%|▋         | 12230/179733 [00:02<00:24, 6857.58 examples/s]
Map (num_proc=8):  10%|▉         | 17227/179733 [00:03<00:23, 6941.95 examples/s]
Map (num_proc=8):   4%|▍         | 7838/179733 [00:02<00:25, 6638.11 examples/s]
Map (num_proc=8):  10%|▉         | 17405/179733 [00:03<00:24, 6706.14 examples/s]
Map (num_proc=8):  10%|█         | 18164/179733 [00:03<00:24, 6682.67 examples/s]
Map (num_proc=8):   9%|▉         | 15755/179733 [00:03<00:24, 6640.73 examples/s]
Map (num_proc=8):   7%|▋         | 12920/179733 [00:02<00:24, 6850.38 examples/s]
Map (num_proc=8):  10%|▉         | 17928/179733 [00:03<00:23, 6941.32 examples/s]
Map (num_proc=8):   5%|▍         | 8515/179733 [00:02<00:26, 6414.54 examples/s]
Map (num_proc=8):   0%|          | 182/179733 [00:00<09:48, 304.86 examples/s]
Map (num_proc=8):  10%|█         | 18119/179733 [00:03<00:23, 6794.69 examples/s]
Map (num_proc=8):  10%|█         | 18849/179733 [00:03<00:23, 6711.78 examples/s]
Map (num_proc=8):   0%|          | 0/179733 [00:00<?, ? examples/s]
Map (num_proc=8):   9%|▉         | 16431/179733 [00:03<00:24, 6665.29 examples/s]
Map (num_proc=8):   8%|▊         | 13609/179733 [00:03<00:24, 6842.18 examples/s]
Map (num_proc=8):  10%|█         | 18664/179733 [00:03<00:23, 6897.83 examples/s]
Map (num_proc=8):   5%|▌         | 9196/179733 [00:02<00:26, 6509.25 examples/s]
Map (num_proc=8):   0%|          | 627/179733 [00:00<02:42, 1105.50 examples/s]
Map (num_proc=8):  10%|█         | 18811/179733 [00:03<00:23, 6813.95 examples/s]
Map (num_proc=8):  11%|█         | 19552/179733 [00:03<00:23, 6764.97 examples/s]
Map (num_proc=8):  10%|▉         | 17117/179733 [00:03<00:24, 6715.18 examples/s]
Map (num_proc=8):   8%|▊         | 14306/179733 [00:03<00:24, 6855.75 examples/s]
Map (num_proc=8):  11%|█         | 19374/179733 [00:03<00:23, 6931.54 examples/s]
Map (num_proc=8):   6%|▌         | 9895/179733 [00:02<00:25, 6624.64 examples/s]
Map (num_proc=8):   1%|          | 1037/179733 [00:00<01:42, 1750.25 examples/s]
Map (num_proc=8):  11%|█         | 19509/179733 [00:03<00:23, 6841.33 examples/s]
Map (num_proc=8):  10%|▉         | 17809/179733 [00:03<00:24, 6738.55 examples/s]
Map (num_proc=8):  11%|█▏        | 20268/179733 [00:03<00:24, 6637.56 examples/s]
Map (num_proc=8):   8%|▊         | 15034/179733 [00:03<00:24, 6855.26 examples/s]
Map (num_proc=8):   6%|▌         | 10591/179733 [00:02<00:25, 6619.67 examples/s]
Map (num_proc=8):  11%|█         | 20111/179733 [00:03<00:23, 6719.06 examples/s]
Map (num_proc=8):   1%|          | 1508/179733 [00:01<01:12, 2441.92 examples/s]
Map (num_proc=8):  10%|█         | 18506/179733 [00:03<00:24, 6695.14 examples/s]
Map (num_proc=8):  12%|█▏        | 20940/179733 [00:03<00:23, 6646.48 examples/s]
Map (num_proc=8):  11%|█▏        | 20235/179733 [00:03<00:24, 6615.66 examples/s]
Map (num_proc=8):   9%|▉         | 15729/179733 [00:03<00:23, 6853.17 examples/s]
Map (num_proc=8):   6%|▋         | 11263/179733 [00:02<00:25, 6620.70 examples/s]
Map (num_proc=8):  12%|█▏        | 20799/179733 [00:03<00:23, 6742.33 examples/s]
Map (num_proc=8):   1%|          | 2089/179733 [00:01<00:54, 3281.96 examples/s]
Map (num_proc=8):  11%|█         | 19190/179733 [00:03<00:23, 6716.46 examples/s]
Map (num_proc=8):  12%|█▏        | 21630/179733 [00:03<00:23, 6698.34 examples/s]
Map (num_proc=8):  12%|█▏        | 20932/179733 [00:03<00:24, 6473.61 examples/s]
Map (num_proc=8):   9%|▉         | 16445/179733 [00:03<00:23, 6913.97 examples/s]
Map (num_proc=8):   7%|▋         | 11962/179733 [00:02<00:25, 6702.66 examples/s]
Map (num_proc=8):  12%|█▏        | 21493/179733 [00:03<00:23, 6783.54 examples/s]
Map (num_proc=8):   2%|▏         | 2747/179733 [00:01<00:42, 4136.84 examples/s]
Map (num_proc=8):  12%|█▏        | 22310/179733 [00:04<00:23, 6685.70 examples/s]
Map (num_proc=8):  11%|█         | 19889/179733 [00:03<00:24, 6561.88 examples/s]
Map (num_proc=8):  12%|█▏        | 21624/179733 [00:03<00:24, 6568.72 examples/s]
Map (num_proc=8):  10%|▉         | 17177/179733 [00:03<00:23, 6971.38 examples/s]
Map (num_proc=8):   7%|▋         | 12645/179733 [00:02<00:24, 6713.63 examples/s]
Map (num_proc=8):  12%|█▏        | 22198/179733 [00:04<00:23, 6825.56 examples/s]
Map (num_proc=8):   0%|          | 23/179733 [00:00<1:14:43, 40.08 examples/s]
Map (num_proc=8):   2%|▏         | 3413/179733 [00:01<00:36, 4802.09 examples/s]
Map (num_proc=8):  13%|█▎        | 22996/179733 [00:04<00:23, 6688.00 examples/s]
Map (num_proc=8):  12%|█▏        | 22314/179733 [00:04<00:23, 6656.32 examples/s]
Map (num_proc=8):  11%|█▏        | 20589/179733 [00:03<00:24, 6488.48 examples/s]
Map (num_proc=8):  10%|▉         | 17915/179733 [00:03<00:23, 7008.02 examples/s]
Map (num_proc=8):   7%|▋         | 13327/179733 [00:02<00:24, 6725.91 examples/s]
Map (num_proc=8):  13%|█▎        | 22882/179733 [00:04<00:23, 6811.22 examples/s]
Map (num_proc=8):   2%|▏         | 4060/179733 [00:01<00:33, 5249.55 examples/s]
Map (num_proc=8):  13%|█▎        | 23674/179733 [00:04<00:23, 6694.69 examples/s]
Map (num_proc=8):  13%|█▎        | 22995/179733 [00:04<00:23, 6667.69 examples/s]
Map (num_proc=8):  12%|█▏        | 21242/179733 [00:04<00:24, 6485.92 examples/s]
Map (num_proc=8):   8%|▊         | 14014/179733 [00:03<00:24, 6723.23 examples/s]
Map (num_proc=8):  13%|█▎        | 23570/179733 [00:04<00:22, 6804.51 examples/s]
Map (num_proc=8):   0%|          | 179/179733 [00:00<10:28, 285.63 examples/s]
Map (num_proc=8):   3%|▎         | 4738/179733 [00:01<00:30, 5662.19 examples/s]
Map (num_proc=8):  10%|█         | 18658/179733 [00:03<00:25, 6358.44 examples/s]
Map (num_proc=8):  14%|█▎        | 24355/179733 [00:04<00:23, 6726.14 examples/s]
Map (num_proc=8):  13%|█▎        | 23675/179733 [00:04<00:23, 6691.46 examples/s]
Map (num_proc=8):  12%|█▏        | 21919/179733 [00:04<00:24, 6536.92 examples/s]
Map (num_proc=8):   8%|▊         | 14703/179733 [00:03<00:24, 6760.88 examples/s]
Map (num_proc=8):   0%|          | 633/179733 [00:00<02:42, 1100.02 examples/s]
Map (num_proc=8):  14%|█▎        | 24282/179733 [00:04<00:22, 6800.54 examples/s]
Map (num_proc=8):   3%|▎         | 5428/179733 [00:01<00:29, 5997.92 examples/s]
Map (num_proc=8):  11%|█         | 19361/179733 [00:03<00:24, 6525.79 examples/s]
Map (num_proc=8):  14%|█▍        | 25035/179733 [00:04<00:22, 6736.11 examples/s]
Map (num_proc=8):  14%|█▎        | 24382/179733 [00:04<00:23, 6704.13 examples/s]
Map (num_proc=8):  13%|█▎        | 22595/179733 [00:04<00:23, 6578.13 examples/s]
Map (num_proc=8):   9%|▊         | 15410/179733 [00:03<00:24, 6833.01 examples/s]
Map (num_proc=8):   1%|          | 1062/179733 [00:00<01:40, 1774.08 examples/s]
Map (num_proc=8):  14%|█▍        | 24973/179733 [00:04<00:22, 6790.70 examples/s]
Map (num_proc=8):   3%|▎         | 6098/179733 [00:01<00:28, 6178.28 examples/s]
Map (num_proc=8):  11%|█         | 20062/179733 [00:04<00:24, 6651.39 examples/s]
Map (num_proc=8):  14%|█▍        | 25715/179733 [00:04<00:23, 6672.34 examples/s]
Map (num_proc=8):  13%|█▎        | 23260/179733 [00:04<00:23, 6578.45 examples/s]
Map (num_proc=8):  14%|█▍        | 25096/179733 [00:04<00:22, 6756.84 examples/s]
Map (num_proc=8):   9%|▉         | 16095/179733 [00:03<00:24, 6808.60 examples/s]
Map (num_proc=8):  14%|█▍        | 25654/179733 [00:04<00:22, 6787.07 examples/s]
Map (num_proc=8):  12%|█▏        | 20769/179733 [00:04<00:23, 6762.87 examples/s]
Map (num_proc=8):   4%|▍         | 6778/179733 [00:01<00:27, 6231.02 examples/s]
Map (num_proc=8):  15%|█▍        | 26409/179733 [00:04<00:22, 6695.23 examples/s]
Map (num_proc=8):  13%|█▎        | 23926/179733 [00:04<00:23, 6567.13 examples/s]
Map (num_proc=8):  14%|█▍        | 25787/179733 [00:04<00:22, 6759.97 examples/s]
Map (num_proc=8):   9%|▉         | 16806/179733 [00:03<00:24, 6739.38 examples/s]
Map (num_proc=8):   4%|▍         | 7449/179733 [00:01<00:27, 6326.09 examples/s]
Map (num_proc=8):  12%|█▏        | 21494/179733 [00:04<00:23, 6826.49 examples/s]
Map (num_proc=8):  15%|█▍        | 26340/179733 [00:04<00:23, 6426.34 examples/s]
Map (num_proc=8):   1%|          | 1392/179733 [00:01<01:50, 1612.51 examples/s]
Map (num_proc=8):  15%|█▌        | 27093/179733 [00:04<00:22, 6704.32 examples/s]
Map (num_proc=8):  14%|█▎        | 24606/179733 [00:04<00:23, 6613.92 examples/s]
Map (num_proc=8):  15%|█▍        | 26506/179733 [00:04<00:22, 6830.33 examples/s]
Map (num_proc=8):  10%|▉         | 17505/179733 [00:03<00:23, 6776.56 examples/s]
Map (num_proc=8):   5%|▍         | 8129/179733 [00:02<00:26, 6438.88 examples/s]
Map (num_proc=8):  12%|█▏        | 22224/179733 [00:04<00:22, 6915.27 examples/s]
Map (num_proc=8):  15%|█▌        | 27022/179733 [00:04<00:23, 6533.39 examples/s]
Map (num_proc=8):   1%|          | 1911/179733 [00:01<01:16, 2326.49 examples/s]
Map (num_proc=8):  15%|█▌        | 27788/179733 [00:04<00:22, 6764.70 examples/s]
Map (num_proc=8):  15%|█▌        | 27219/179733 [00:04<00:22, 6879.62 examples/s]
Map (num_proc=8):  14%|█▍        | 25303/179733 [00:04<00:23, 6556.19 examples/s]
Map (num_proc=8):  10%|█         | 18195/179733 [00:03<00:23, 6773.81 examples/s]
Map (num_proc=8):   5%|▍         | 8815/179733 [00:02<00:26, 6537.10 examples/s]
Map (num_proc=8):  13%|█▎        | 22955/179733 [00:04<00:22, 6980.23 examples/s]
Map (num_proc=8):  15%|█▌        | 27720/179733 [00:04<00:22, 6655.54 examples/s]
Map (num_proc=8):   1%|▏         | 2595/179733 [00:01<00:53, 3328.08 examples/s]
Map (num_proc=8):  16%|█▌        | 28476/179733 [00:04<00:22, 6752.82 examples/s]
Map (num_proc=8):  14%|█▍        | 25971/179733 [00:04<00:23, 6539.16 examples/s]
Map (num_proc=8):  16%|█▌        | 27941/179733 [00:04<00:22, 6751.69 examples/s]
Map (num_proc=8):  11%|█         | 18910/179733 [00:03<00:23, 6851.45 examples/s]
Map (num_proc=8):   5%|▌         | 9475/179733 [00:02<00:26, 6528.97 examples/s]
Map (num_proc=8):  13%|█▎        | 23671/179733 [00:04<00:22, 6993.52 examples/s]
Map (num_proc=8):  16%|█▌        | 28401/179733 [00:04<00:22, 6680.40 examples/s]
Map (num_proc=8):   2%|▏         | 3265/179733 [00:01<00:42, 4120.56 examples/s]
Map (num_proc=8):  16%|█▌        | 29183/179733 [00:05<00:22, 6802.47 examples/s]
Map (num_proc=8):  16%|█▌        | 28625/179733 [00:05<00:22, 6737.92 examples/s]
Map (num_proc=8):  15%|█▍        | 26664/179733 [00:04<00:23, 6513.01 examples/s]
Map (num_proc=8):  14%|█▎        | 24387/179733 [00:04<00:22, 7001.37 examples/s]
Map (num_proc=8):   6%|▌         | 10155/179733 [00:02<00:26, 6493.57 examples/s]
Map (num_proc=8):  16%|█▌        | 29093/179733 [00:05<00:22, 6736.62 examples/s]
Map (num_proc=8):  11%|█         | 19633/179733 [00:03<00:24, 6548.38 examples/s]
Map (num_proc=8):   2%|▏         | 3914/179733 [00:01<00:37, 4715.47 examples/s]
Map (num_proc=8):  17%|█▋        | 29865/179733 [00:05<00:22, 6776.39 examples/s]
Map (num_proc=8):  15%|█▌        | 27364/179733 [00:04<00:22, 6636.55 examples/s]
Map (num_proc=8):  16%|█▋        | 29328/179733 [00:05<00:22, 6746.87 examples/s]
Map (num_proc=8):   6%|▌         | 10808/179733 [00:02<00:26, 6487.66 examples/s]
Map (num_proc=8):  14%|█▍        | 25110/179733 [00:04<00:21, 7034.42 examples/s]
Map (num_proc=8):  17%|█▋        | 29774/179733 [00:05<00:22, 6715.72 examples/s]
Map (num_proc=8):  11%|█▏        | 20318/179733 [00:03<00:24, 6601.39 examples/s]
Map (num_proc=8):   3%|▎         | 4596/179733 [00:01<00:33, 5256.47 examples/s]
Map (num_proc=8):  17%|█▋        | 30585/179733 [00:05<00:21, 6832.23 examples/s]
Map (num_proc=8):  16%|█▌        | 28043/179733 [00:05<00:22, 6665.26 examples/s]
Map (num_proc=8):  17%|█▋        | 30042/179733 [00:05<00:22, 6540.16 examples/s]
Map (num_proc=8):   6%|▋         | 11475/179733 [00:02<00:25, 6519.16 examples/s]
Map (num_proc=8):  17%|█▋        | 30456/179733 [00:05<00:22, 6723.58 examples/s]
Map (num_proc=8):  12%|█▏        | 21030/179733 [00:04<00:23, 6728.84 examples/s]
Map (num_proc=8):   3%|▎         | 5294/179733 [00:01<00:30, 5720.69 examples/s]
Map (num_proc=8):  17%|█▋        | 31277/179733 [00:05<00:21, 6840.80 examples/s]
Map (num_proc=8):  14%|█▍        | 25855/179733 [00:04<00:24, 6324.30 examples/s]
Map (num_proc=8):  16%|█▌        | 28751/179733 [00:05<00:22, 6669.40 examples/s]
Map (num_proc=8):  17%|█▋        | 30732/179733 [00:05<00:22, 6618.72 examples/s]
Map (num_proc=8):   7%|▋         | 12168/179733 [00:02<00:25, 6613.13 examples/s]
Map (num_proc=8):  17%|█▋        | 31153/179733 [00:05<00:21, 6763.52 examples/s]
Map (num_proc=8):  12%|█▏        | 21728/179733 [00:04<00:23, 6781.38 examples/s]
Map (num_proc=8):   3%|▎         | 5993/179733 [00:01<00:28, 6051.50 examples/s]
Map (num_proc=8):  18%|█▊        | 31968/179733 [00:05<00:21, 6827.85 examples/s]
Map (num_proc=8):  15%|█▍        | 26525/179733 [00:04<00:23, 6399.66 examples/s]
Map (num_proc=8):  16%|█▋        | 29432/179733 [00:05<00:22, 6701.67 examples/s]
Map (num_proc=8):  17%|█▋        | 31411/179733 [00:05<00:22, 6616.17 examples/s]
Map (num_proc=8):   7%|▋         | 12847/179733 [00:02<00:25, 6624.26 examples/s]
Map (num_proc=8):  18%|█▊        | 31846/179733 [00:05<00:21, 6791.90 examples/s]
Map (num_proc=8):  12%|█▏        | 22409/179733 [00:04<00:23, 6777.87 examples/s]
Map (num_proc=8):   4%|▎         | 6665/179733 [00:02<00:27, 6242.51 examples/s]
Map (num_proc=8):  18%|█▊        | 32689/179733 [00:05<00:21, 6893.91 examples/s]
Map (num_proc=8):  15%|█▌        | 27216/179733 [00:05<00:23, 6512.56 examples/s]
Map (num_proc=8):  17%|█▋        | 30130/179733 [00:05<00:22, 6763.63 examples/s]
Map (num_proc=8):  18%|█▊        | 32088/179733 [00:05<00:22, 6633.88 examples/s]
Map (num_proc=8):   8%|▊         | 13519/179733 [00:02<00:25, 6638.37 examples/s]
Map (num_proc=8):  18%|█▊        | 32526/179733 [00:05<00:21, 6772.26 examples/s]
Map (num_proc=8):  13%|█▎        | 23106/179733 [00:04<00:22, 6825.46 examples/s]
Map (num_proc=8):   4%|▍         | 7335/179733 [00:02<00:27, 6340.99 examples/s]
Map (num_proc=8):  16%|█▌        | 27916/179733 [00:05<00:22, 6605.91 examples/s]
Map (num_proc=8):  19%|█▊        | 33397/179733 [00:05<00:22, 6488.06 examples/s]
Map (num_proc=8):  17%|█▋        | 30835/179733 [00:05<00:22, 6694.04 examples/s]
Map (num_proc=8):  18%|█▊        | 32753/179733 [00:05<00:22, 6604.75 examples/s]
Map (num_proc=8):   8%|▊         | 14199/179733 [00:02<00:24, 6673.70 examples/s]
Map (num_proc=8):  18%|█▊        | 33218/179733 [00:05<00:21, 6804.64 examples/s]
Map (num_proc=8):  13%|█▎        | 23793/179733 [00:04<00:22, 6828.39 examples/s]
Map (num_proc=8):   4%|▍         | 8017/179733 [00:02<00:26, 6469.25 examples/s]
Map (num_proc=8):  16%|█▌        | 28623/179733 [00:05<00:22, 6733.63 examples/s]
Map (num_proc=8):  19%|█▉        | 34072/179733 [00:05<00:22, 6547.39 examples/s]
Map (num_proc=8):  18%|█▊        | 31520/179733 [00:05<00:22, 6713.29 examples/s]
Map (num_proc=8):  19%|█▊        | 33420/179733 [00:05<00:22, 6606.20 examples/s]
Map (num_proc=8):   8%|▊         | 14900/179733 [00:03<00:24, 6698.52 examples/s]
Map (num_proc=8):  19%|█▉        | 33922/179733 [00:05<00:21, 6717.76 examples/s]
Map (num_proc=8):  14%|█▎        | 24495/179733 [00:04<00:23, 6702.64 examples/s]
Map (num_proc=8):   5%|▍         | 8706/179733 [00:02<00:26, 6538.50 examples/s]
Map (num_proc=8):  16%|█▋        | 29322/179733 [00:05<00:22, 6709.78 examples/s]
Map (num_proc=8):  19%|█▉        | 34763/179733 [00:05<00:21, 6621.53 examples/s]
Map (num_proc=8):  19%|█▉        | 34093/179733 [00:05<00:22, 6608.64 examples/s]
Map (num_proc=8):  18%|█▊        | 32224/179733 [00:05<00:22, 6448.68 examples/s]
Map (num_proc=8):   9%|▊         | 15577/179733 [00:03<00:24, 6660.73 examples/s]
Map (num_proc=8):  19%|█▉        | 34627/179733 [00:05<00:21, 6743.75 examples/s]
Map (num_proc=8):   5%|▌         | 9400/179733 [00:02<00:25, 6644.96 examples/s]
Map (num_proc=8):  14%|█▍        | 25178/179733 [00:04<00:23, 6548.59 examples/s]
Map (num_proc=8):  17%|█▋        | 30042/179733 [00:05<00:21, 6840.29 examples/s]
Map (num_proc=8):  20%|█▉        | 35432/179733 [00:05<00:21, 6596.04 examples/s]
Map (num_proc=8):  19%|█▉        | 34783/179733 [00:05<00:21, 6671.64 examples/s]
Map (num_proc=8):  18%|█▊        | 32884/179733 [00:05<00:22, 6471.27 examples/s]
Map (num_proc=8):  20%|█▉        | 35311/179733 [00:05<00:21, 6753.83 examples/s]
Map (num_proc=8):   9%|▉         | 16272/179733 [00:03<00:25, 6466.29 examples/s]
Map (num_proc=8):  14%|█▍        | 25836/179733 [00:04<00:23, 6538.02 examples/s]
Map (num_proc=8):   6%|▌         | 10113/179733 [00:02<00:25, 6719.98 examples/s]
Map (num_proc=8):  17%|█▋        | 30736/179733 [00:05<00:21, 6847.69 examples/s]
Map (num_proc=8):  20%|██        | 36116/179733 [00:06<00:21, 6654.28 examples/s]
Map (num_proc=8):  20%|█▉        | 35469/179733 [00:06<00:21, 6691.16 examples/s]
Map (num_proc=8):  19%|█▊        | 33558/179733 [00:05<00:22, 6479.42 examples/s]
Map (num_proc=8):  20%|██        | 36023/179733 [00:06<00:21, 6822.86 examples/s]
Map (num_proc=8):   9%|▉         | 16931/179733 [00:03<00:25, 6450.41 examples/s]
Map (num_proc=8):  15%|█▍        | 26530/179733 [00:04<00:23, 6634.73 examples/s]
Map (num_proc=8):   6%|▌         | 10808/179733 [00:02<00:24, 6773.49 examples/s]
Map (num_proc=8):  18%|█▊        | 31454/179733 [00:05<00:21, 6908.08 examples/s]
Map (num_proc=8):  20%|██        | 36798/179733 [00:06<00:21, 6656.88 examples/s]
Map (num_proc=8):  20%|██        | 36149/179733 [00:06<00:21, 6701.61 examples/s]
Map (num_proc=8):  19%|█▉        | 34218/179733 [00:05<00:22, 6484.85 examples/s]
Map (num_proc=8):  20%|██        | 36741/179733 [00:06<00:20, 6870.47 examples/s]
Map (num_proc=8):  10%|▉         | 17601/179733 [00:03<00:24, 6501.44 examples/s]
Map (num_proc=8):   6%|▋         | 11500/179733 [00:02<00:24, 6781.32 examples/s]
Map (num_proc=8):  15%|█▌        | 27238/179733 [00:04<00:22, 6703.78 examples/s]
Map (num_proc=8):  18%|█▊        | 32175/179733 [00:05<00:21, 6956.11 examples/s]
Map (num_proc=8):  21%|██        | 37480/179733 [00:06<00:21, 6668.90 examples/s]
Map (num_proc=8):  20%|██        | 36838/179733 [00:06<00:21, 6535.73 examples/s]
Map (num_proc=8):  19%|█▉        | 34883/179733 [00:06<00:22, 6500.58 examples/s]
Map (num_proc=8):  21%|██        | 37473/179733 [00:06<00:20, 6960.34 examples/s]
Map (num_proc=8):  10%|█         | 18279/179733 [00:03<00:24, 6558.50 examples/s]
Map (num_proc=8):   7%|▋         | 12215/179733 [00:02<00:24, 6873.35 examples/s]
Map (num_proc=8):  16%|█▌        | 27931/179733 [00:05<00:22, 6730.94 examples/s]
Map (num_proc=8):  18%|█▊        | 32893/179733 [00:05<00:21, 6918.21 examples/s]
Map (num_proc=8):  21%|██        | 38169/179733 [00:06<00:21, 6701.38 examples/s]
Map (num_proc=8):  21%|██        | 37499/179733 [00:06<00:21, 6531.81 examples/s]
Map (num_proc=8):  20%|█▉        | 35544/179733 [00:06<00:22, 6497.39 examples/s]
Map (num_proc=8):  11%|█         | 18959/179733 [00:03<00:24, 6599.28 examples/s]
Map (num_proc=8):   7%|▋         | 12911/179733 [00:02<00:24, 6862.74 examples/s]
Map (num_proc=8):  16%|█▌        | 28615/179733 [00:05<00:22, 6729.29 examples/s]
Map (num_proc=8):  21%|██        | 38191/179733 [00:06<00:21, 6600.10 examples/s]
Map (num_proc=8):  19%|█▊        | 33614/179733 [00:05<00:21, 6952.25 examples/s]
Map (num_proc=8):  22%|██▏       | 38848/179733 [00:06<00:20, 6709.96 examples/s]
Map (num_proc=8):  21%|██        | 38174/179733 [00:06<00:21, 6575.45 examples/s]
Map (num_proc=8):  20%|██        | 36240/179733 [00:06<00:21, 6598.21 examples/s]
Map (num_proc=8):  11%|█         | 19630/179733 [00:03<00:24, 6606.70 examples/s]
Map (num_proc=8):   8%|▊         | 13606/179733 [00:03<00:24, 6876.54 examples/s]
Map (num_proc=8):  16%|█▋        | 29320/179733 [00:05<00:22, 6788.82 examples/s]
Map (num_proc=8):  22%|██▏       | 38876/179733 [00:06<00:21, 6638.02 examples/s]
Map (num_proc=8):  19%|█▉        | 34323/179733 [00:06<00:20, 6941.75 examples/s]
Map (num_proc=8):  22%|██▏       | 39555/179733 [00:06<00:21, 6479.05 examples/s]
Map (num_proc=8):  22%|██▏       | 38872/179733 [00:06<00:21, 6646.67 examples/s]
Map (num_proc=8):  21%|██        | 36911/179733 [00:06<00:21, 6587.17 examples/s]
Map (num_proc=8):  11%|█▏        | 20292/179733 [00:03<00:24, 6604.21 examples/s]
Map (num_proc=8):   8%|▊         | 14309/179733 [00:03<00:23, 6909.41 examples/s]
Map (num_proc=8):  17%|█▋        | 30026/179733 [00:05<00:22, 6636.90 examples/s]
Map (num_proc=8):  22%|██▏       | 39543/179733 [00:06<00:21, 6614.67 examples/s]
Map (num_proc=8):  19%|█▉        | 35039/179733 [00:06<00:20, 6971.69 examples/s]
Map (num_proc=8):  22%|██▏       | 40212/179733 [00:06<00:21, 6490.86 examples/s]
Map (num_proc=8):  21%|██        | 37581/179733 [00:06<00:21, 6606.05 examples/s]
Map (num_proc=8):  22%|██▏       | 39562/179733 [00:06<00:21, 6564.40 examples/s]
Map (num_proc=8):  12%|█▏        | 20966/179733 [00:04<00:23, 6626.78 examples/s]
Map (num_proc=8):   8%|▊         | 15026/179733 [00:03<00:23, 6926.07 examples/s]
Map (num_proc=8):  22%|██▏       | 40211/179733 [00:06<00:21, 6609.15 examples/s]
Map (num_proc=8):  17%|█▋        | 30723/179733 [00:05<00:22, 6659.94 examples/s]
Map (num_proc=8):  23%|██▎       | 40889/179733 [00:06<00:21, 6559.46 examples/s]
Map (num_proc=8):  21%|██▏       | 38255/179733 [00:06<00:21, 6608.67 examples/s]
Map (num_proc=8):  20%|█▉        | 35773/179733 [00:06<00:22, 6339.74 examples/s]
Map (num_proc=8):  22%|██▏       | 40244/179733 [00:06<00:21, 6621.35 examples/s]
Map (num_proc=8):   9%|▉         | 15738/179733 [00:03<00:23, 6935.53 examples/s]
Map (num_proc=8):  12%|█▏        | 21670/179733 [00:04<00:23, 6618.90 examples/s]
Map (num_proc=8):  23%|██▎       | 40892/179733 [00:06<00:20, 6658.56 examples/s]
Map (num_proc=8):  17%|█▋        | 31414/179733 [00:05<00:22, 6699.02 examples/s]
Map (num_proc=8):  23%|██▎       | 41552/179733 [00:06<00:21, 6555.74 examples/s]
Map (num_proc=8):  22%|██▏       | 38939/179733 [00:06<00:21, 6647.29 examples/s]
Map (num_proc=8):  20%|██        | 36439/179733 [00:06<00:22, 6380.88 examples/s]
Map (num_proc=8):  23%|██▎       | 40928/179733 [00:06<00:20, 6656.55 examples/s]
Map (num_proc=8):   9%|▉         | 16449/179733 [00:03<00:23, 6964.75 examples/s]
Map (num_proc=8):  12%|█▏        | 22335/179733 [00:04<00:23, 6616.64 examples/s]
Map (num_proc=8):  23%|██▎       | 41588/179733 [00:06<00:20, 6670.66 examples/s]
Map (num_proc=8):  18%|█▊        | 32119/179733 [00:05<00:22, 6639.97 examples/s]
Map (num_proc=8):  23%|██▎       | 42235/179733 [00:07<00:21, 6545.25 examples/s]
Map (num_proc=8):  21%|██        | 37108/179733 [00:06<00:22, 6458.36 examples/s]
Map (num_proc=8):  23%|██▎       | 41624/179733 [00:06<00:20, 6666.76 examples/s]
Map (num_proc=8):  22%|██▏       | 39616/179733 [00:06<00:21, 6394.60 examples/s]
Map (num_proc=8):  13%|█▎        | 23008/179733 [00:04<00:23, 6640.29 examples/s]
Map (num_proc=8):  10%|▉         | 17158/179733 [00:03<00:23, 6832.79 examples/s]
Map (num_proc=8):  18%|█▊        | 32796/179733 [00:05<00:22, 6626.08 examples/s]
Map (num_proc=8):  24%|██▎       | 42285/179733 [00:07<00:21, 6364.34 examples/s]
Map (num_proc=8):  24%|██▍       | 42910/179733 [00:07<00:20, 6602.25 examples/s]
Map (num_proc=8):  21%|██        | 37802/179733 [00:06<00:21, 6564.82 examples/s]
Map (num_proc=8):  24%|██▎       | 42305/179733 [00:07<00:20, 6588.34 examples/s]
Map (num_proc=8):  22%|██▏       | 40293/179733 [00:06<00:21, 6478.87 examples/s]
Map (num_proc=8):  13%|█▎        | 23683/179733 [00:04<00:23, 6658.72 examples/s]
Map (num_proc=8):  19%|█▊        | 33500/179733 [00:05<00:22, 6547.91 examples/s]
Map (num_proc=8):  24%|██▍       | 42963/179733 [00:07<00:21, 6477.63 examples/s]
Map (num_proc=8):  10%|▉         | 17883/179733 [00:03<00:25, 6269.27 examples/s]
Map (num_proc=8):  24%|██▍       | 43609/179733 [00:07<00:20, 6691.36 examples/s]
Map (num_proc=8):  21%|██▏       | 38498/179733 [00:06<00:21, 6658.62 examples/s]
Map (num_proc=8):  24%|██▍       | 42967/179733 [00:07<00:20, 6557.71 examples/s]
Map (num_proc=8):  23%|██▎       | 40945/179733 [00:06<00:21, 6475.06 examples/s]
Map (num_proc=8):  14%|█▎        | 24381/179733 [00:04<00:23, 6656.12 examples/s]
Map (num_proc=8):  19%|█▉        | 34176/179733 [00:06<00:22, 6594.57 examples/s]
Map (num_proc=8):  24%|██▍       | 43653/179733 [00:07<00:20, 6499.23 examples/s]
Map (num_proc=8):  10%|█         | 18593/179733 [00:03<00:24, 6468.00 examples/s]
Map (num_proc=8):  25%|██▍       | 44311/179733 [00:07<00:20, 6747.63 examples/s]
Map (num_proc=8):  22%|██▏       | 39170/179733 [00:06<00:21, 6658.71 examples/s]
Map (num_proc=8):  23%|██▎       | 41612/179733 [00:07<00:21, 6483.78 examples/s]
Map (num_proc=8):  24%|██▍       | 43669/179733 [00:07<00:20, 6618.17 examples/s]
Map (num_proc=8):  14%|█▍        | 25052/179733 [00:04<00:23, 6636.61 examples/s]
Map (num_proc=8):  19%|█▉        | 34842/179733 [00:06<00:22, 6581.61 examples/s]
Map (num_proc=8):  25%|██▍       | 44319/179733 [00:07<00:20, 6534.58 examples/s]
Map (num_proc=8):  11%|█         | 19276/179733 [00:03<00:24, 6566.98 examples/s]
Map (num_proc=8):  25%|██▌       | 45027/179733 [00:07<00:20, 6718.71 examples/s]
Map (num_proc=8):  22%|██▏       | 39867/179733 [00:06<00:20, 6722.41 examples/s]
Map (num_proc=8):  24%|██▎       | 42265/179733 [00:07<00:21, 6471.56 examples/s]
Map (num_proc=8):  25%|██▍       | 44361/179733 [00:07<00:20, 6667.21 examples/s]
Map (num_proc=8):  14%|█▍        | 25743/179733 [00:04<00:23, 6421.20 examples/s]
Map (num_proc=8):  20%|█▉        | 35524/179733 [00:06<00:21, 6623.16 examples/s]
Map (num_proc=8):  25%|██▌       | 44983/179733 [00:07<00:20, 6536.84 examples/s]
Map (num_proc=8):  11%|█         | 19973/179733 [00:03<00:24, 6654.48 examples/s]
Map (num_proc=8):  25%|██▌       | 45718/179733 [00:07<00:19, 6751.20 examples/s]
Map (num_proc=8):  23%|██▎       | 40569/179733 [00:07<00:20, 6732.90 examples/s]
Map (num_proc=8):  25%|██▌       | 45028/179733 [00:07<00:20, 6649.23 examples/s]
Map (num_proc=8):  24%|██▍       | 42953/179733 [00:07<00:20, 6534.52 examples/s]
Map (num_proc=8):  15%|█▍        | 26404/179733 [00:04<00:24, 6320.05 examples/s]
Map (num_proc=8):  20%|██        | 36193/179733 [00:06<00:21, 6609.07 examples/s]
Map (num_proc=8):  25%|██▌       | 45680/179733 [00:07<00:20, 6630.40 examples/s]
Map (num_proc=8):  11%|█▏        | 20651/179733 [00:04<00:23, 6666.25 examples/s]
Map (num_proc=8):  26%|██▌       | 46437/179733 [00:07<00:19, 6754.41 examples/s]
Map (num_proc=8):  23%|██▎       | 41282/179733 [00:07<00:20, 6816.38 examples/s]
Map (num_proc=8):  25%|██▌       | 45720/179733 [00:07<00:19, 6709.06 examples/s]
Map (num_proc=8):  24%|██▍       | 43630/179733 [00:07<00:20, 6553.52 examples/s]
Map (num_proc=8):  15%|█▌        | 27066/179733 [00:04<00:23, 6378.17 examples/s]
Map (num_proc=8):  26%|██▌       | 46366/179733 [00:07<00:19, 6684.24 examples/s]
Map (num_proc=8):  21%|██        | 36897/179733 [00:06<00:21, 6627.00 examples/s]
Map (num_proc=8):  12%|█▏        | 21334/179733 [00:04<00:23, 6679.47 examples/s]
Map (num_proc=8):  26%|██▌       | 47132/179733 [00:07<00:19, 6785.82 examples/s]
Map (num_proc=8):  23%|██▎       | 41983/179733 [00:07<00:20, 6758.05 examples/s]
Map (num_proc=8):  26%|██▌       | 46394/179733 [00:07<00:19, 6702.83 examples/s]
Map (num_proc=8):  25%|██▍       | 44291/179733 [00:07<00:20, 6526.87 examples/s]
Map (num_proc=8):  15%|█▌        | 27763/179733 [00:05<00:23, 6497.34 examples/s]
Map (num_proc=8):  26%|██▌       | 47037/179733 [00:07<00:19, 6660.44 examples/s]
Map (num_proc=8):  21%|██        | 37595/179733 [00:06<00:21, 6677.59 examples/s]
Map (num_proc=8):  12%|█▏        | 22022/179733 [00:04<00:23, 6703.62 examples/s]
Map (num_proc=8):  27%|██▋       | 47814/179733 [00:07<00:19, 6790.54 examples/s]
Map (num_proc=8):  24%|██▎       | 42671/179733 [00:07<00:20, 6755.77 examples/s]
Map (num_proc=8):  25%|██▌       | 44968/179733 [00:07<00:20, 6577.69 examples/s]
Map (num_proc=8):  26%|██▌       | 47101/179733 [00:07<00:21, 6275.69 examples/s]
Map (num_proc=8):  16%|█▌        | 28444/179733 [00:05<00:23, 6568.85 examples/s]
Map (num_proc=8):  27%|██▋       | 47725/179733 [00:07<00:19, 6695.29 examples/s]
Map (num_proc=8):  13%|█▎        | 22704/179733 [00:04<00:23, 6723.95 examples/s]
Map (num_proc=8):  21%|██▏       | 38289/179733 [00:06<00:21, 6595.09 examples/s]
Map (num_proc=8):  24%|██▍       | 43385/179733 [00:07<00:20, 6803.81 examples/s]
Map (num_proc=8):  27%|██▋       | 48529/179733 [00:07<00:20, 6529.48 examples/s]
Map (num_proc=8):  25%|██▌       | 45642/179733 [00:07<00:20, 6598.61 examples/s]
Map (num_proc=8):  27%|██▋       | 47747/179733 [00:07<00:20, 6303.79 examples/s]
Map (num_proc=8):  16%|█▌        | 29130/179733 [00:05<00:22, 6615.61 examples/s]
Map (num_proc=8):  27%|██▋       | 48433/179733 [00:07<00:19, 6751.57 examples/s]
Map (num_proc=8):  13%|█▎        | 23388/179733 [00:04<00:23, 6719.81 examples/s]
Map (num_proc=8):  22%|██▏       | 38960/179733 [00:06<00:21, 6602.33 examples/s]
Map (num_proc=8):  25%|██▍       | 44108/179733 [00:07<00:19, 6876.08 examples/s]
Map (num_proc=8):  27%|██▋       | 49210/179733 [00:08<00:19, 6576.30 examples/s]
Map (num_proc=8):  26%|██▌       | 46328/179733 [00:07<00:20, 6419.13 examples/s]
Map (num_proc=8):  27%|██▋       | 48414/179733 [00:08<00:20, 6391.72 examples/s]
Map (num_proc=8):  17%|█▋        | 29796/179733 [00:05<00:22, 6598.87 examples/s]
Map (num_proc=8):  27%|██▋       | 49141/179733 [00:08<00:19, 6789.21 examples/s]
Map (num_proc=8):  13%|█▎        | 24068/179733 [00:04<00:23, 6719.59 examples/s]
Map (num_proc=8):  22%|██▏       | 39623/179733 [00:06<00:21, 6594.71 examples/s]
Map (num_proc=8):  28%|██▊       | 49894/179733 [00:08<00:19, 6613.75 examples/s]
Map (num_proc=8):  25%|██▍       | 44804/179733 [00:07<00:19, 6813.32 examples/s]
Map (num_proc=8):  26%|██▌       | 46997/179733 [00:07<00:20, 6469.86 examples/s]
Map (num_proc=8):  27%|██▋       | 49101/179733 [00:08<00:20, 6516.91 examples/s]
Map (num_proc=8):  17%|█▋        | 30468/179733 [00:05<00:22, 6625.90 examples/s]
Map (num_proc=8):  28%|██▊       | 49855/179733 [00:08<00:18, 6843.72 examples/s]
Map (num_proc=8):  14%|█▍        | 24782/179733 [00:04<00:22, 6767.92 examples/s]
Map (num_proc=8):  22%|██▏       | 40330/179733 [00:06<00:20, 6668.04 examples/s]
Map (num_proc=8):  28%|██▊       | 50589/179733 [00:08<00:19, 6670.57 examples/s]
Map (num_proc=8):  25%|██▌       | 45497/179733 [00:07<00:19, 6828.31 examples/s]
Map (num_proc=8):  27%|██▋       | 47652/179733 [00:08<00:20, 6472.81 examples/s]
Map (num_proc=8):  28%|██▊       | 49801/179733 [00:08<00:20, 6496.58 examples/s]
Map (num_proc=8):  17%|█▋        | 31133/179733 [00:05<00:22, 6607.62 examples/s]
Map (num_proc=8):  28%|██▊       | 50563/179733 [00:08<00:18, 6862.41 examples/s]
Map (num_proc=8):  14%|█▍        | 25492/179733 [00:04<00:22, 6838.83 examples/s]
Map (num_proc=8):  23%|██▎       | 41015/179733 [00:07<00:21, 6507.83 examples/s]
Map (num_proc=8):  29%|██▊       | 51283/179733 [00:08<00:19, 6540.49 examples/s]
Map (num_proc=8):  27%|██▋       | 48328/179733 [00:08<00:20, 6540.71 examples/s]
Map (num_proc=8):  26%|██▌       | 46197/179733 [00:07<00:21, 6180.93 examples/s]
Map (num_proc=8):  28%|██▊       | 50488/179733 [00:08<00:19, 6468.82 examples/s]
Map (num_proc=8):  18%|█▊        | 31838/179733 [00:05<00:22, 6715.42 examples/s]
Map (num_proc=8):  15%|█▍        | 26189/179733 [00:04<00:22, 6801.84 examples/s]
Map (num_proc=8):  23%|██▎       | 41674/179733 [00:07<00:21, 6509.67 examples/s]
Map (num_proc=8):  29%|██▊       | 51272/179733 [00:08<00:19, 6526.15 examples/s]
Map (num_proc=8):  29%|██▉       | 51950/179733 [00:08<00:19, 6550.91 examples/s]
Map (num_proc=8):  27%|██▋       | 49006/179733 [00:08<00:19, 6589.91 examples/s]
Map (num_proc=8):  26%|██▌       | 46887/179733 [00:08<00:20, 6363.37 examples/s]
Map (num_proc=8):  28%|██▊       | 51176/179733 [00:08<00:19, 6565.58 examples/s]
Map (num_proc=8):  18%|█▊        | 32545/179733 [00:05<00:21, 6703.36 examples/s]
Map (num_proc=8):  15%|█▍        | 26895/179733 [00:05<00:22, 6794.60 examples/s]
Map (num_proc=8):  24%|██▎       | 42333/179733 [00:07<00:21, 6503.85 examples/s]
Map (num_proc=8):  29%|██▉       | 51946/179733 [00:08<00:19, 6567.33 examples/s]
Map (num_proc=8):  29%|██▉       | 52635/179733 [00:08<00:19, 6620.72 examples/s]
Map (num_proc=8):  28%|██▊       | 49707/179733 [00:08<00:20, 6283.75 examples/s]
Map (num_proc=8):  26%|██▋       | 47586/179733 [00:08<00:20, 6504.94 examples/s]
Map (num_proc=8):  29%|██▉       | 51838/179733 [00:08<00:19, 6551.84 examples/s]
Map (num_proc=8):  18%|█▊        | 33221/179733 [00:05<00:21, 6697.31 examples/s]
Map (num_proc=8):  15%|█▌        | 27591/179733 [00:05<00:22, 6823.56 examples/s]
Map (num_proc=8):  24%|██▍       | 43017/179733 [00:07<00:20, 6579.22 examples/s]
Map (num_proc=8):  29%|██▉       | 52630/179733 [00:08<00:19, 6603.49 examples/s]
Map (num_proc=8):  30%|██▉       | 53315/179733 [00:08<00:18, 6658.48 examples/s]
Map (num_proc=8):  28%|██▊       | 50347/179733 [00:08<00:20, 6292.52 examples/s]
Map (num_proc=8):  29%|██▉       | 52534/179733 [00:08<00:19, 6648.49 examples/s]
Map (num_proc=8):  27%|██▋       | 48280/179733 [00:08<00:20, 6550.11 examples/s]
Map (num_proc=8):  19%|█▉        | 33895/179733 [00:06<00:23, 6328.58 examples/s]
Map (num_proc=8):  16%|█▌        | 28288/179733 [00:05<00:22, 6806.48 examples/s]
Map (num_proc=8):  24%|██▍       | 43687/179733 [00:07<00:20, 6598.71 examples/s]
Map (num_proc=8):  30%|██▉       | 53317/179733 [00:08<00:19, 6645.46 examples/s]
Map (num_proc=8):  30%|███       | 54010/179733 [00:08<00:18, 6713.04 examples/s]
Map (num_proc=8):  28%|██▊       | 51011/179733 [00:08<00:20, 6386.69 examples/s]
Map (num_proc=8):  30%|██▉       | 53201/179733 [00:08<00:19, 6628.52 examples/s]
Map (num_proc=8):  27%|██▋       | 48952/179733 [00:08<00:19, 6580.15 examples/s]
Map (num_proc=8):  19%|█▉        | 34553/179733 [00:06<00:22, 6394.44 examples/s]
Map (num_proc=8):  16%|█▌        | 29004/179733 [00:05<00:21, 6875.72 examples/s]
Map (num_proc=8):  25%|██▍       | 44363/179733 [00:07<00:20, 6630.67 examples/s]
Map (num_proc=8):  30%|███       | 54013/179733 [00:08<00:18, 6728.37 examples/s]
Map (num_proc=8):  30%|███       | 54722/179733 [00:08<00:18, 6628.74 examples/s]
Map (num_proc=8):  29%|██▉       | 51676/179733 [00:08<00:19, 6429.07 examples/s]
Map (num_proc=8):  28%|██▊       | 49634/179733 [00:08<00:19, 6617.53 examples/s]
Map (num_proc=8):  30%|██▉       | 53882/179733 [00:08<00:18, 6635.94 examples/s]
Map (num_proc=8):  20%|█▉        | 35223/179733 [00:06<00:22, 6475.89 examples/s]
Map (num_proc=8):  17%|█▋        | 29721/179733 [00:05<00:21, 6909.49 examples/s]
Map (num_proc=8):  25%|██▌       | 45052/179733 [00:07<00:20, 6675.21 examples/s]
Map (num_proc=8):  30%|███       | 54715/179733 [00:08<00:18, 6730.89 examples/s]
Map (num_proc=8):  31%|███       | 55405/179733 [00:09<00:18, 6670.43 examples/s]
Map (num_proc=8):  29%|██▉       | 52346/179733 [00:08<00:19, 6457.00 examples/s]
Map (num_proc=8):  28%|██▊       | 50318/179733 [00:08<00:19, 6671.12 examples/s]
Map (num_proc=8):  30%|███       | 54577/179733 [00:08<00:19, 6566.04 examples/s]
Map (num_proc=8):  20%|█▉        | 35879/179733 [00:06<00:22, 6441.04 examples/s]
Map (num_proc=8):  17%|█▋        | 30417/179733 [00:05<00:21, 6885.07 examples/s]
Map (num_proc=8):  25%|██▌       | 45751/179733 [00:07<00:20, 6642.72 examples/s]
Map (num_proc=8):  31%|███       | 55401/179733 [00:08<00:18, 6745.54 examples/s]
Map (num_proc=8):  31%|███       | 56098/179733 [00:09<00:18, 6707.93 examples/s]
Map (num_proc=8):  29%|██▉       | 53014/179733 [00:08<00:19, 6503.03 examples/s]
Map (num_proc=8):  28%|██▊       | 50988/179733 [00:08<00:19, 6655.28 examples/s]
Map (num_proc=8):  31%|███       | 55242/179733 [00:09<00:18, 6589.74 examples/s]
Map (num_proc=8):  20%|██        | 36548/179733 [00:06<00:22, 6499.84 examples/s]
Map (num_proc=8):  17%|█▋        | 31134/179733 [00:05<00:21, 6936.15 examples/s]
Map (num_proc=8):  26%|██▌       | 46423/179733 [00:07<00:20, 6643.71 examples/s]
Map (num_proc=8):  31%|███       | 56078/179733 [00:09<00:18, 6742.65 examples/s]
Map (num_proc=8):  32%|███▏      | 56802/179733 [00:09<00:18, 6715.72 examples/s]
Map (num_proc=8):  30%|██▉       | 53682/179733 [00:08<00:19, 6533.45 examples/s]
Map (num_proc=8):  29%|██▊       | 51672/179733 [00:08<00:19, 6663.20 examples/s]
Map (num_proc=8):  31%|███       | 55919/179733 [00:09<00:18, 6556.94 examples/s]
Map (num_proc=8):  21%|██        | 37224/179733 [00:06<00:21, 6537.97 examples/s]
Map (num_proc=8):  26%|██▌       | 47091/179733 [00:07<00:20, 6631.65 examples/s]
Map (num_proc=8):  32%|███▏      | 56795/179733 [00:09<00:19, 6454.26 examples/s]
Map (num_proc=8):  18%|█▊        | 31837/179733 [00:05<00:23, 6314.62 examples/s]
Map (num_proc=8):  32%|███▏      | 57515/179733 [00:09<00:17, 6812.68 examples/s]
Map (num_proc=8):  30%|███       | 54364/179733 [00:09<00:19, 6582.26 examples/s]
Map (num_proc=8):  29%|██▉       | 52359/179733 [00:08<00:18, 6710.70 examples/s]
Map (num_proc=8):  31%|███▏      | 56606/179733 [00:09<00:18, 6596.24 examples/s]
Map (num_proc=8):  21%|██        | 37884/179733 [00:06<00:21, 6521.55 examples/s]
Map (num_proc=8):  27%|██▋       | 47760/179733 [00:08<00:19, 6617.99 examples/s]
Map (num_proc=8):  32%|███▏      | 57460/179733 [00:09<00:18, 6494.00 examples/s]
Map (num_proc=8):  18%|█▊        | 32511/179733 [00:05<00:23, 6377.73 examples/s]
Map (num_proc=8):  32%|███▏      | 58197/179733 [00:09<00:17, 6796.48 examples/s]
Map (num_proc=8):  31%|███       | 55030/179733 [00:09<00:18, 6567.86 examples/s]
Map (num_proc=8):  30%|██▉       | 53039/179733 [00:08<00:18, 6718.41 examples/s]
Map (num_proc=8):  32%|███▏      | 57290/179733 [00:09<00:18, 6657.67 examples/s]
Map (num_proc=8):  21%|██▏       | 38552/179733 [00:06<00:21, 6542.17 examples/s]
Map (num_proc=8):  27%|██▋       | 48422/179733 [00:08<00:19, 6601.86 examples/s]
Map (num_proc=8):  32%|███▏      | 58130/179733 [00:09<00:18, 6526.55 examples/s]
Map (num_proc=8):  18%|█▊        | 33203/179733 [00:05<00:22, 6522.99 examples/s]
Map (num_proc=8):  33%|███▎      | 58901/179733 [00:09<00:17, 6750.10 examples/s]
Map (num_proc=8):  31%|███       | 55704/179733 [00:09<00:18, 6577.09 examples/s]
Map (num_proc=8):  30%|██▉       | 53733/179733 [00:09<00:18, 6760.18 examples/s]
Map (num_proc=8):  32%|███▏      | 57998/179733 [00:09<00:18, 6683.15 examples/s]
Map (num_proc=8):  22%|██▏       | 39222/179733 [00:06<00:21, 6566.87 examples/s]
Map (num_proc=8):  27%|██▋       | 49117/179733 [00:08<00:19, 6653.01 examples/s]
Map (num_proc=8):  33%|███▎      | 58806/179733 [00:09<00:18, 6557.88 examples/s]
Map (num_proc=8):  19%|█▉        | 33891/179733 [00:06<00:22, 6432.23 examples/s]
Map (num_proc=8):  33%|███▎      | 59588/179733 [00:09<00:17, 6757.58 examples/s]
Map (num_proc=8):  31%|███▏      | 56366/179733 [00:09<00:18, 6572.55 examples/s]
Map (num_proc=8):  30%|███       | 54417/179733 [00:09<00:18, 6774.55 examples/s]
Map (num_proc=8):  22%|██▏       | 39905/179733 [00:06<00:21, 6615.80 examples/s]
Map (num_proc=8):  28%|██▊       | 49789/179733 [00:08<00:19, 6642.51 examples/s]
Map (num_proc=8):  33%|███▎      | 58697/179733 [00:09<00:19, 6261.38 examples/s]
Map (num_proc=8):  33%|███▎      | 59486/179733 [00:09<00:18, 6592.90 examples/s]
Map (num_proc=8):  19%|█▉        | 34559/179733 [00:06<00:22, 6496.93 examples/s]
Map (num_proc=8):  34%|███▎      | 60303/179733 [00:09<00:17, 6764.16 examples/s]
Map (num_proc=8):  31%|███       | 55104/179733 [00:09<00:18, 6772.32 examples/s]
Map (num_proc=8):  32%|███▏      | 57028/179733 [00:09<00:18, 6529.60 examples/s]
Map (num_proc=8):  23%|██▎       | 40598/179733 [00:07<00:21, 6514.92 examples/s]
Map (num_proc=8):  28%|██▊       | 50454/179733 [00:08<00:19, 6625.05 examples/s]
Map (num_proc=8):  33%|███▎      | 59346/179733 [00:09<00:19, 6316.85 examples/s]
Map (num_proc=8):  33%|███▎      | 60148/179733 [00:09<00:18, 6568.76 examples/s]
Map (num_proc=8):  20%|█▉        | 35234/179733 [00:06<00:22, 6536.20 examples/s]
Map (num_proc=8):  31%|███       | 55785/179733 [00:09<00:18, 6760.03 examples/s]
Map (num_proc=8):  32%|███▏      | 57710/179733 [00:09<00:18, 6582.80 examples/s]
Map (num_proc=8):  34%|███▍      | 61016/179733 [00:09<00:18, 6540.35 examples/s]
Map (num_proc=8):  23%|██▎       | 41266/179733 [00:07<00:21, 6545.44 examples/s]
Map (num_proc=8):  28%|██▊       | 51120/179733 [00:08<00:19, 6592.02 examples/s]
Map (num_proc=8):  33%|███▎      | 60024/179733 [00:09<00:18, 6362.29 examples/s]
Map (num_proc=8):  34%|███▍      | 60838/179733 [00:09<00:17, 6648.91 examples/s]
Map (num_proc=8):  20%|█▉        | 35917/179733 [00:06<00:21, 6599.70 examples/s]
Map (num_proc=8):  31%|███▏      | 56464/179733 [00:09<00:18, 6731.99 examples/s]
Map (num_proc=8):  32%|███▏      | 58410/179733 [00:09<00:18, 6670.57 examples/s]
Map (num_proc=8):  34%|███▍      | 61715/179733 [00:09<00:17, 6656.07 examples/s]
Map (num_proc=8):  29%|██▉       | 51799/179733 [00:08<00:19, 6632.22 examples/s]
Map (num_proc=8):  34%|███▍      | 60711/179733 [00:09<00:18, 6479.28 examples/s]
Map (num_proc=8):  23%|██▎       | 41950/179733 [00:07<00:21, 6332.64 examples/s]
Map (num_proc=8):  34%|███▍      | 61540/179733 [00:09<00:17, 6704.61 examples/s]
Map (num_proc=8):  20%|██        | 36615/179733 [00:06<00:21, 6663.45 examples/s]
Map (num_proc=8):  32%|███▏      | 57151/179733 [00:09<00:18, 6754.65 examples/s]
Map (num_proc=8):  33%|███▎      | 59079/179733 [00:09<00:18, 6651.26 examples/s]
Map (num_proc=8):  35%|███▍      | 62385/179733 [00:10<00:17, 6650.03 examples/s]
Map (num_proc=8):  29%|██▉       | 52482/179733 [00:08<00:19, 6662.58 examples/s]
Map (num_proc=8):  34%|███▍      | 61367/179733 [00:10<00:18, 6479.80 examples/s]
Map (num_proc=8):  24%|██▎       | 42602/179733 [00:07<00:21, 6350.92 examples/s]
Map (num_proc=8):  35%|███▍      | 62215/179733 [00:10<00:17, 6702.01 examples/s]
Map (num_proc=8):  21%|██        | 37291/179733 [00:06<00:21, 6686.19 examples/s]
Map (num_proc=8):  32%|███▏      | 57836/179733 [00:09<00:18, 6755.44 examples/s]
Map (num_proc=8):  35%|███▌      | 63074/179733 [00:10<00:17, 6692.61 examples/s]
Map (num_proc=8):  33%|███▎      | 59772/179733 [00:09<00:18, 6543.00 examples/s]
Map (num_proc=8):  35%|███▍      | 62046/179733 [00:10<00:18, 6535.62 examples/s]
Map (num_proc=8):  30%|██▉       | 53183/179733 [00:08<00:19, 6544.57 examples/s]
Map (num_proc=8):  24%|██▍       | 43254/179733 [00:07<00:21, 6398.10 examples/s]
Map (num_proc=8):  35%|███▍      | 62886/179733 [00:10<00:17, 6679.20 examples/s]
Map (num_proc=8):  21%|██        | 38003/179733 [00:06<00:21, 6744.64 examples/s]
Map (num_proc=8):  33%|███▎      | 58551/179733 [00:09<00:17, 6820.76 examples/s]
Map (num_proc=8):  34%|███▎      | 60429/179733 [00:09<00:18, 6529.12 examples/s]
Map (num_proc=8):  35%|███▌      | 63774/179733 [00:10<00:17, 6570.94 examples/s]
Map (num_proc=8):  35%|███▍      | 62711/179733 [00:10<00:17, 6554.83 examples/s]
Map (num_proc=8):  24%|██▍       | 43909/179733 [00:07<00:21, 6434.34 examples/s]
Map (num_proc=8):  30%|██▉       | 53853/179733 [00:09<00:19, 6554.66 examples/s]
Map (num_proc=8):  35%|███▌      | 63569/179733 [00:10<00:17, 6703.90 examples/s]
Map (num_proc=8):  22%|██▏       | 38695/179733 [00:06<00:20, 6753.97 examples/s]
Map (num_proc=8):  33%|███▎      | 59236/179733 [00:09<00:17, 6804.66 examples/s]
Map (num_proc=8):  34%|███▍      | 61111/179733 [00:10<00:17, 6599.45 examples/s]
Map (num_proc=8):  36%|███▌      | 64435/179733 [00:10<00:17, 6558.19 examples/s]
Map (num_proc=8):  35%|███▌      | 63371/179733 [00:10<00:17, 6507.84 examples/s]
Map (num_proc=8):  25%|██▍       | 44558/179733 [00:07<00:20, 6445.94 examples/s]
Map (num_proc=8):  30%|███       | 54545/179733 [00:09<00:19, 6533.80 examples/s]
Map (num_proc=8):  36%|███▌      | 64246/179733 [00:10<00:17, 6700.95 examples/s]
Map (num_proc=8):  22%|██▏       | 39401/179733 [00:06<00:20, 6790.39 examples/s]
Map (num_proc=8):  33%|███▎      | 59946/179733 [00:09<00:17, 6751.74 examples/s]
Map (num_proc=8):  34%|███▍      | 61798/179733 [00:10<00:17, 6639.95 examples/s]
Map (num_proc=8):  36%|███▌      | 65113/179733 [00:10<00:17, 6595.11 examples/s]
Map (num_proc=8):  36%|███▌      | 64033/179733 [00:10<00:17, 6518.36 examples/s]
Map (num_proc=8):  25%|██▌       | 45225/179733 [00:07<00:20, 6475.63 examples/s]
Map (num_proc=8):  31%|███       | 55230/179733 [00:09<00:18, 6592.16 examples/s]
Map (num_proc=8):  36%|███▌      | 64919/179733 [00:10<00:17, 6622.56 examples/s]
Map (num_proc=8):  22%|██▏       | 40119/179733 [00:06<00:20, 6851.62 examples/s]
Map (num_proc=8):  34%|███▎      | 60634/179733 [00:10<00:17, 6774.23 examples/s]
Map (num_proc=8):  35%|███▍      | 62488/179733 [00:10<00:17, 6690.12 examples/s]
Map (num_proc=8):  37%|███▋      | 65776/179733 [00:10<00:17, 6592.22 examples/s]
Map (num_proc=8):  36%|███▌      | 64711/179733 [00:10<00:17, 6572.16 examples/s]
Map (num_proc=8):  26%|██▌       | 45905/179733 [00:07<00:20, 6546.89 examples/s]
Map (num_proc=8):  31%|███       | 55921/179733 [00:09<00:18, 6544.30 examples/s]
Map (num_proc=8):  37%|███▋      | 65617/179733 [00:10<00:17, 6661.10 examples/s]
Map (num_proc=8):  23%|██▎       | 40836/179733 [00:07<00:20, 6860.47 examples/s]
Map (num_proc=8):  34%|███▍      | 61321/179733 [00:10<00:17, 6774.92 examples/s]
Map (num_proc=8):  35%|███▌      | 63199/179733 [00:10<00:17, 6690.00 examples/s]
Map (num_proc=8):  37%|███▋      | 66445/179733 [00:10<00:17, 6587.89 examples/s]
Map (num_proc=8):  36%|███▋      | 65378/179733 [00:10<00:17, 6593.25 examples/s]
Map (num_proc=8):  26%|██▌       | 46575/179733 [00:07<00:20, 6569.54 examples/s]
Map (num_proc=8):  31%|███▏      | 56614/179733 [00:09<00:18, 6585.76 examples/s]
Map (num_proc=8):  37%|███▋      | 66286/179733 [00:10<00:17, 6620.57 examples/s]
Map (num_proc=8):  23%|██▎       | 41556/179733 [00:07<00:20, 6888.76 examples/s]
Map (num_proc=8):  35%|███▍      | 62042/179733 [00:10<00:17, 6809.52 examples/s]
Map (num_proc=8):  37%|███▋      | 67135/179733 [00:10<00:16, 6674.76 examples/s]
Map (num_proc=8):  36%|███▌      | 63909/179733 [00:10<00:17, 6779.53 examples/s]
Map (num_proc=8):  26%|██▋       | 47240/179733 [00:08<00:20, 6574.68 examples/s]
Map (num_proc=8):  37%|███▋      | 66075/179733 [00:10<00:17, 6521.45 examples/s]
Map (num_proc=8):  37%|███▋      | 66991/179733 [00:10<00:16, 6682.67 examples/s]
Map (num_proc=8):  32%|███▏      | 57314/179733 [00:09<00:18, 6541.73 examples/s]
Map (num_proc=8):  35%|███▍      | 62748/179733 [00:10<00:17, 6861.95 examples/s]
Map (num_proc=8):  24%|██▎       | 42279/179733 [00:07<00:21, 6423.21 examples/s]
Map (num_proc=8):  38%|███▊      | 67817/179733 [00:10<00:16, 6695.98 examples/s]
Map (num_proc=8):  36%|███▌      | 64588/179733 [00:10<00:17, 6758.49 examples/s]
Map (num_proc=8):  27%|██▋       | 47900/179733 [00:08<00:20, 6564.27 examples/s]
Map (num_proc=8):  37%|███▋      | 66741/179733 [00:10<00:17, 6524.54 examples/s]
Map (num_proc=8):  38%|███▊      | 67673/179733 [00:10<00:16, 6708.01 examples/s]
Map (num_proc=8):  32%|███▏      | 57992/179733 [00:09<00:19, 6332.74 examples/s]
Map (num_proc=8):  35%|███▌      | 63441/179733 [00:10<00:16, 6855.50 examples/s]
Map (num_proc=8):  24%|██▍       | 42946/179733 [00:07<00:21, 6467.24 examples/s]
Map (num_proc=8):  38%|███▊      | 68501/179733 [00:10<00:16, 6725.09 examples/s]
Map (num_proc=8):  36%|███▋      | 65302/179733 [00:10<00:17, 6608.86 examples/s]
Map (num_proc=8):  27%|██▋       | 48592/179733 [00:08<00:19, 6629.82 examples/s]
Map (num_proc=8):  38%|███▊      | 67402/179733 [00:10<00:17, 6507.73 examples/s]
Map (num_proc=8):  33%|███▎      | 58658/179733 [00:09<00:18, 6417.09 examples/s]
Map (num_proc=8):  38%|███▊      | 68380/179733 [00:10<00:17, 6378.49 examples/s]
Map (num_proc=8):  36%|███▌      | 64169/179733 [00:10<00:16, 6951.57 examples/s]
Map (num_proc=8):  24%|██▍       | 43599/179733 [00:07<00:21, 6482.40 examples/s]
Map (num_proc=8):  39%|███▊      | 69215/179733 [00:11<00:16, 6805.76 examples/s]
Map (num_proc=8):  37%|███▋      | 65978/179733 [00:10<00:17, 6607.08 examples/s]
Map (num_proc=8):  27%|██▋       | 49257/179733 [00:08<00:19, 6612.59 examples/s]
Map (num_proc=8):  38%|███▊      | 68069/179733 [00:11<00:17, 6531.81 examples/s]
Map (num_proc=8):  33%|███▎      | 59337/179733 [00:09<00:18, 6494.21 examples/s]
Map (num_proc=8):  38%|███▊      | 69059/179733 [00:11<00:17, 6463.67 examples/s]
Map (num_proc=8):  25%|██▍       | 44263/179733 [00:07<00:20, 6511.48 examples/s]
Map (num_proc=8):  39%|███▉      | 69916/179733 [00:11<00:16, 6849.82 examples/s]
Map (num_proc=8):  37%|███▋      | 66666/179733 [00:10<00:16, 6658.15 examples/s]
Map (num_proc=8):  36%|███▌      | 64898/179733 [00:10<00:18, 6359.83 examples/s]
Map (num_proc=8):  28%|██▊       | 49923/179733 [00:08<00:19, 6611.77 examples/s]
Map (num_proc=8):  38%|███▊      | 68740/179733 [00:11<00:16, 6550.12 examples/s]
Map (num_proc=8):  33%|███▎      | 59999/179733 [00:09<00:18, 6515.41 examples/s]
Map (num_proc=8):  39%|███▉      | 69723/179733 [00:11<00:16, 6500.26 examples/s]
Map (num_proc=8):  25%|██▌       | 44945/179733 [00:07<00:21, 6283.86 examples/s]
Map (num_proc=8):  39%|███▉      | 70606/179733 [00:11<00:16, 6584.79 examples/s]
Map (num_proc=8):  37%|███▋      | 67335/179733 [00:11<00:16, 6648.36 examples/s]
Map (num_proc=8):  36%|███▋      | 65569/179733 [00:10<00:17, 6437.90 examples/s]
Map (num_proc=8):  28%|██▊       | 50607/179733 [00:08<00:19, 6550.48 examples/s]
Map (num_proc=8):  39%|███▊      | 69430/179733 [00:11<00:16, 6610.58 examples/s]
Map (num_proc=8):  34%|███▍      | 60685/179733 [00:10<00:18, 6596.06 examples/s]
Map (num_proc=8):  39%|███▉      | 70400/179733 [00:11<00:16, 6551.57 examples/s]
Map (num_proc=8):  25%|██▌       | 45635/179733 [00:07<00:20, 6441.47 examples/s]
Map (num_proc=8):  40%|███▉      | 71298/179733 [00:11<00:16, 6648.17 examples/s]
Map (num_proc=8):  38%|███▊      | 68002/179733 [00:11<00:16, 6634.03 examples/s]
Map (num_proc=8):  37%|███▋      | 66250/179733 [00:10<00:17, 6518.18 examples/s]
Map (num_proc=8):  29%|██▊       | 51270/179733 [00:08<00:19, 6544.26 examples/s]
Map (num_proc=8):  39%|███▉      | 70114/179733 [00:11<00:16, 6656.24 examples/s]
Map (num_proc=8):  34%|███▍      | 61368/179733 [00:10<00:17, 6628.84 examples/s]
Map (num_proc=8):  40%|███▉      | 71084/179733 [00:11<00:16, 6610.48 examples/s]
Map (num_proc=8):  26%|██▌       | 46296/179733 [00:07<00:20, 6462.44 examples/s]
Map (num_proc=8):  38%|███▊      | 68701/179733 [00:11<00:16, 6718.13 examples/s]
Map (num_proc=8):  40%|████      | 72001/179733 [00:11<00:16, 6671.19 examples/s]
Map (num_proc=8):  37%|███▋      | 66945/179733 [00:10<00:17, 6614.56 examples/s]
Map (num_proc=8):  29%|██▉       | 51953/179733 [00:08<00:19, 6565.07 examples/s]
Map (num_proc=8):  39%|███▉      | 70795/179733 [00:11<00:16, 6665.17 examples/s]
Map (num_proc=8):  35%|███▍      | 62046/179733 [00:10<00:17, 6657.74 examples/s]
Map (num_proc=8):  40%|███▉      | 71763/179733 [00:11<00:16, 6637.57 examples/s]
Map (num_proc=8):  26%|██▌       | 46987/179733 [00:08<00:20, 6565.95 examples/s]
Map (num_proc=8):  39%|███▊      | 69381/179733 [00:11<00:16, 6722.10 examples/s]
Map (num_proc=8):  40%|████      | 72685/179733 [00:11<00:16, 6689.07 examples/s]
Map (num_proc=8):  38%|███▊      | 67617/179733 [00:11<00:16, 6636.01 examples/s]
Map (num_proc=8):  29%|██▉       | 52648/179733 [00:08<00:19, 6660.92 examples/s]
Map (num_proc=8):  40%|███▉      | 71506/179733 [00:11<00:16, 6746.07 examples/s]
Map (num_proc=8):  35%|███▍      | 62721/179733 [00:10<00:17, 6674.33 examples/s]
Map (num_proc=8):  40%|████      | 72431/179733 [00:11<00:16, 6613.69 examples/s]
Map (num_proc=8):  27%|██▋       | 47664/179733 [00:08<00:20, 6601.68 examples/s]
Map (num_proc=8):  39%|███▉      | 70055/179733 [00:11<00:16, 6698.91 examples/s]
Map (num_proc=8):  41%|████      | 73364/179733 [00:11<00:15, 6665.94 examples/s]
Map (num_proc=8):  38%|███▊      | 68296/179733 [00:11<00:16, 6645.88 examples/s]
Map (num_proc=8):  40%|████      | 72212/179733 [00:11<00:15, 6784.14 examples/s]
Map (num_proc=8):  30%|██▉       | 53344/179733 [00:08<00:19, 6530.21 examples/s]
Map (num_proc=8):  35%|███▌      | 63414/179733 [00:10<00:17, 6733.49 examples/s]
Map (num_proc=8):  41%|████      | 73100/179733 [00:11<00:16, 6618.11 examples/s]
Map (num_proc=8):  39%|███▉      | 70726/179733 [00:11<00:16, 6690.50 examples/s]
Map (num_proc=8):  27%|██▋       | 48367/179733 [00:08<00:19, 6677.99 examples/s]
Map (num_proc=8):  41%|████      | 74039/179733 [00:11<00:15, 6675.41 examples/s]
Map (num_proc=8):  38%|███▊      | 68989/179733 [00:11<00:16, 6718.67 examples/s]
Map (num_proc=8):  41%|████      | 72929/179733 [00:11<00:15, 6850.74 examples/s]
Map (num_proc=8):  30%|███       | 54031/179733 [00:09<00:19, 6604.71 examples/s]
Map (num_proc=8):  36%|███▌      | 64097/179733 [00:10<00:17, 6721.18 examples/s]
Map (num_proc=8):  41%|████      | 73775/179733 [00:11<00:15, 6635.71 examples/s]
Map (num_proc=8):  27%|██▋       | 49047/179733 [00:08<00:19, 6697.09 examples/s]
Map (num_proc=8):  42%|████▏     | 74731/179733 [00:11<00:15, 6742.57 examples/s]
Map (num_proc=8):  40%|███▉      | 71404/179733 [00:11<00:16, 6611.36 examples/s]
Map (num_proc=8):  39%|███▉      | 69685/179733 [00:11<00:16, 6779.45 examples/s]
Map (num_proc=8):  41%|████      | 73637/179733 [00:11<00:15, 6836.39 examples/s]
Map (num_proc=8):  30%|███       | 54697/179733 [00:09<00:18, 6617.93 examples/s]
Map (num_proc=8):  36%|███▌      | 64777/179733 [00:10<00:17, 6743.74 examples/s]
Map (num_proc=8):  41%|████▏     | 74455/179733 [00:11<00:15, 6680.80 examples/s]
Map (num_proc=8):  28%|██▊       | 49724/179733 [00:08<00:19, 6697.02 examples/s]
Map (num_proc=8):  40%|████      | 72077/179733 [00:11<00:16, 6617.74 examples/s]
Map (num_proc=8):  42%|████▏     | 75443/179733 [00:11<00:15, 6748.82 examples/s]
Map (num_proc=8):  39%|███▉      | 70389/179733 [00:11<00:16, 6804.47 examples/s]
Map (num_proc=8):  31%|███       | 55368/179733 [00:09<00:18, 6634.65 examples/s]
Map (num_proc=8):  36%|███▋      | 65453/179733 [00:10<00:17, 6711.75 examples/s]
Map (num_proc=8):  42%|████▏     | 75141/179733 [00:11<00:15, 6730.15 examples/s]
Map (num_proc=8):  41%|████▏     | 74339/179733 [00:11<00:16, 6362.63 examples/s]
Map (num_proc=8):  28%|██▊       | 50436/179733 [00:08<00:19, 6775.32 examples/s]
Map (num_proc=8):  40%|████      | 72773/179733 [00:11<00:15, 6701.65 examples/s]
Map (num_proc=8):  42%|████▏     | 76144/179733 [00:12<00:15, 6789.32 examples/s]
Map (num_proc=8):  40%|███▉      | 71109/179733 [00:11<00:15, 6884.12 examples/s]
Map (num_proc=8):  31%|███       | 56073/179733 [00:09<00:18, 6516.73 examples/s]
Map (num_proc=8):  37%|███▋      | 66136/179733 [00:10<00:16, 6693.15 examples/s]
Map (num_proc=8):  42%|████▏     | 75007/179733 [00:12<00:16, 6429.00 examples/s]
Map (num_proc=8):  42%|████▏     | 75842/179733 [00:12<00:15, 6697.25 examples/s]
Map (num_proc=8):  41%|████      | 73468/179733 [00:11<00:15, 6760.50 examples/s]
Map (num_proc=8):  28%|██▊       | 51138/179733 [00:08<00:19, 6733.00 examples/s]
Map (num_proc=8):  43%|████▎     | 76849/179733 [00:12<00:15, 6759.32 examples/s]
Map (num_proc=8):  40%|███▉      | 71832/179733 [00:11<00:15, 6920.73 examples/s]
Map (num_proc=8):  32%|███▏      | 56737/179733 [00:09<00:18, 6490.67 examples/s]
Map (num_proc=8):  37%|███���      | 66807/179733 [00:10<00:16, 6653.32 examples/s]
Map (num_proc=8):  42%|████▏     | 75679/179733 [00:12<00:16, 6477.64 examples/s]
Map (num_proc=8):  43%|████▎     | 76540/179733 [00:12<00:15, 6544.31 examples/s]
Map (num_proc=8):  29%|██▉       | 51824/179733 [00:08<00:19, 6731.84 examples/s]
Map (num_proc=8):  41%|████▏     | 74177/179733 [00:12<00:16, 6474.94 examples/s]
Map (num_proc=8):  40%|████      | 72565/179733 [00:11<00:15, 6893.44 examples/s]
Map (num_proc=8):  43%|████▎     | 77561/179733 [00:12<00:15, 6410.77 examples/s]
Map (num_proc=8):  32%|███▏      | 57409/179733 [00:09<00:18, 6536.80 examples/s]
Map (num_proc=8):  38%|███▊      | 67486/179733 [00:11<00:16, 6683.55 examples/s]
Map (num_proc=8):  42%|████▏     | 76333/179733 [00:12<00:15, 6479.63 examples/s]
Map (num_proc=8):  43%|████▎     | 77222/179733 [00:12<00:15, 6555.51 examples/s]
Map (num_proc=8):  29%|██▉       | 52534/179733 [00:08<00:18, 6793.44 examples/s]
Map (num_proc=8):  42%|████▏     | 74847/179733 [00:12<00:16, 6490.99 examples/s]
Map (num_proc=8):  41%|████      | 73289/179733 [00:11<00:15, 6959.70 examples/s]
Map (num_proc=8):  44%|████▎     | 78255/179733 [00:12<00:15, 6540.29 examples/s]
Map (num_proc=8):  32%|███▏      | 58073/179733 [00:09<00:18, 6536.20 examples/s]
Map (num_proc=8):  38%|███▊      | 68196/179733 [00:11<00:16, 6705.97 examples/s]
Map (num_proc=8):  43%|████▎     | 77017/179733 [00:12<00:15, 6425.74 examples/s]
Map (num_proc=8):  43%|████▎     | 77890/179733 [00:12<00:15, 6563.21 examples/s]
Map (num_proc=8):  30%|██▉       | 53225/179733 [00:08<00:18, 6818.56 examples/s]
Map (num_proc=8):  42%|████▏     | 75520/179733 [00:12<00:15, 6530.54 examples/s]
Map (num_proc=8):  41%|████      | 74022/179733 [00:12<00:15, 6903.59 examples/s]
Map (num_proc=8):  44%|████▍     | 78946/179733 [00:12<00:15, 6606.82 examples/s]
Map (num_proc=8):  33%|███▎      | 58736/179733 [00:09<00:18, 6503.10 examples/s]
Map (num_proc=8):  38%|███▊      | 68888/179733 [00:11<00:16, 6745.12 examples/s]
Map (num_proc=8):  43%|████▎     | 77676/179733 [00:12<00:15, 6459.88 examples/s]
Map (num_proc=8):  44%|████▎     | 78577/179733 [00:12<00:15, 6642.26 examples/s]
Map (num_proc=8):  30%|██▉       | 53919/179733 [00:09<00:18, 6821.87 examples/s]
Map (num_proc=8):  42%|████▏     | 76208/179733 [00:12<00:15, 6628.36 examples/s]
Map (num_proc=8):  42%|████▏     | 74757/179733 [00:12<00:14, 7000.08 examples/s]
Map (num_proc=8):  44%|████▍     | 79618/179733 [00:12<00:15, 6616.10 examples/s]
Map (num_proc=8):  33%|███▎      | 59394/179733 [00:09<00:18, 6515.01 examples/s]
Map (num_proc=8):  39%|███▊      | 69590/179733 [00:11<00:16, 6758.37 examples/s]
Map (num_proc=8):  44%|████▎     | 78347/179733 [00:12<00:15, 6521.86 examples/s]
Map (num_proc=8):  44%|████▍     | 79261/179733 [00:12<00:15, 6682.65 examples/s]
Map (num_proc=8):  30%|███       | 54642/179733 [00:09<00:18, 6880.70 examples/s]
Map (num_proc=8):  43%|████▎     | 76912/179733 [00:12<00:15, 6696.93 examples/s]
Map (num_proc=8):  42%|████▏     | 75490/179733 [00:12<00:14, 7052.11 examples/s]
Map (num_proc=8):  45%|████▍     | 80310/179733 [00:12<00:14, 6683.23 examples/s]
Map (num_proc=8):  33%|███▎      | 60069/179733 [00:10<00:18, 6557.45 examples/s]
Map (num_proc=8):  39%|███▉      | 70281/179733 [00:11<00:16, 6767.98 examples/s]
Map (num_proc=8):  44%|████▍     | 79033/179733 [00:12<00:15, 6611.07 examples/s]
Map (num_proc=8):  44%|████▍     | 79951/179733 [00:12<00:14, 6701.18 examples/s]
Map (num_proc=8):  31%|███       | 55366/179733 [00:09<00:17, 6933.26 examples/s]
Map (num_proc=8):  43%|████▎     | 77603/179733 [00:12<00:15, 6727.33 examples/s]
Map (num_proc=8):  45%|████▌     | 81017/179733 [00:12<00:14, 6726.94 examples/s]
Map (num_proc=8):  34%|███▍      | 60727/179733 [00:10<00:18, 6544.02 examples/s]
Map (num_proc=8):  42%|████▏     | 76200/179733 [00:12<00:16, 6446.97 examples/s]
Map (num_proc=8):  44%|████▍     | 79734/179733 [00:12<00:15, 6555.09 examples/s]
Map (num_proc=8):  40%|███▉      | 70995/179733 [00:11<00:16, 6599.92 examples/s]
Map (num_proc=8):  45%|████▍     | 80661/179733 [00:12<00:14, 6781.82 examples/s]
Map (num_proc=8):  44%|████▎     | 78288/179733 [00:12<00:15, 6750.12 examples/s]
Map (num_proc=8):  31%|███       | 56095/179733 [00:09<00:18, 6610.32 examples/s]
Map (num_proc=8):  45%|████▌     | 81720/179733 [00:12<00:14, 6767.03 examples/s]
Map (num_proc=8):  34%|███▍      | 61423/179733 [00:10<00:17, 6636.15 examples/s]
Map (num_proc=8):  43%|████▎     | 76874/179733 [00:12<00:15, 6522.47 examples/s]
Map (num_proc=8):  45%|████▍     | 80395/179733 [00:12<00:15, 6547.86 examples/s]
Map (num_proc=8):  40%|███▉      | 71667/179733 [00:11<00:16, 6606.02 examples/s]
Map (num_proc=8):  45%|████▌     | 81355/179733 [00:12<00:14, 6603.26 examples/s]
Map (num_proc=8):  44%|████▍     | 78989/179733 [00:12<00:14, 6822.76 examples/s]
Map (num_proc=8):  46%|████▌     | 82404/179733 [00:13<00:14, 6729.06 examples/s]
Map (num_proc=8):  32%|███▏      | 56798/179733 [00:09<00:19, 6436.48 examples/s]
Map (num_proc=8):  35%|███▍      | 62098/179733 [00:10<00:17, 6658.35 examples/s]
Map (num_proc=8):  43%|████▎     | 77544/179733 [00:12<00:15, 6570.16 examples/s]
Map (num_proc=8):  40%|████      | 72348/179733 [00:11<00:16, 6662.24 examples/s]
Map (num_proc=8):  45%|████▌     | 81052/179733 [00:13<00:15, 6531.18 examples/s]
Map (num_proc=8):  46%|████▌     | 82031/179733 [00:13<00:14, 6628.75 examples/s]
Map (num_proc=8):  44%|████▍     | 79713/179733 [00:12<00:14, 6889.49 examples/s]
Map (num_proc=8):  46%|████▌     | 83089/179733 [00:13<00:14, 6738.53 examples/s]
Map (num_proc=8):  32%|███▏      | 57479/179733 [00:09<00:18, 6500.24 examples/s]
Map (num_proc=8):  35%|███▍      | 62803/179733 [00:10<00:17, 6638.85 examples/s]
Map (num_proc=8):  44%|████▎     | 78240/179733 [00:12<00:15, 6665.74 examples/s]
Map (num_proc=8):  45%|████▌     | 81726/179733 [00:13<00:14, 6578.01 examples/s]
Map (num_proc=8):  41%|████      | 73029/179733 [00:11<00:15, 6682.39 examples/s]
Map (num_proc=8):  46%|████▌     | 82712/179733 [00:13<00:14, 6662.32 examples/s]
Map (num_proc=8):  45%|████▍     | 80438/179733 [00:12<00:14, 6874.88 examples/s]
Map (num_proc=8):  47%|████▋     | 83770/179733 [00:13<00:14, 6737.66 examples/s]
Map (num_proc=8):  32%|███▏      | 58150/179733 [00:09<00:18, 6545.32 examples/s]
Map (num_proc=8):  35%|███▌      | 63483/179733 [00:10<00:17, 6654.21 examples/s]
Map (num_proc=8):  44%|████▍     | 78944/179733 [00:12<00:14, 6771.98 examples/s]
Map (num_proc=8):  46%|████▌     | 82400/179733 [00:13<00:14, 6612.35 examples/s]
Map (num_proc=8):  41%|████      | 73741/179733 [00:12<00:16, 6620.22 examples/s]
Map (num_proc=8):  46%|████▋     | 83385/179733 [00:13<00:14, 6672.42 examples/s]
Map (num_proc=8):  45%|████▌     | 81126/179733 [00:13<00:14, 6856.80 examples/s]
Map (num_proc=8):  33%|███▎      | 58829/179733 [00:09<00:18, 6589.03 examples/s]
Map (num_proc=8):  47%|████▋     | 84457/179733 [00:13<00:14, 6604.95 examples/s]
Map (num_proc=8):  44%|████▍     | 79635/179733 [00:12<00:14, 6803.43 examples/s]
Map (num_proc=8):  36%|███▌      | 64171/179733 [00:10<00:17, 6488.75 examples/s]
Map (num_proc=8):  46%|████▌     | 83079/179733 [00:13<00:14, 6652.80 examples/s]
Map (num_proc=8):  41%|████▏     | 74420/179733 [00:12<00:15, 6642.38 examples/s]
Map (num_proc=8):  47%|████▋     | 84058/179733 [00:13<00:14, 6665.82 examples/s]
Map (num_proc=8):  46%|████▌     | 81819/179733 [00:13<00:14, 6833.14 examples/s]
Map (num_proc=8):  33%|███▎      | 59510/179733 [00:09<00:18, 6623.88 examples/s]
Map (num_proc=8):  47%|████▋     | 85148/179733 [00:13<00:14, 6675.40 examples/s]
Map (num_proc=8):  45%|████▍     | 80325/179733 [00:12<00:14, 6816.92 examples/s]
Map (num_proc=8):  36%|███▌      | 64840/179733 [00:10<00:17, 6519.87 examples/s]
Map (num_proc=8):  47%|████▋     | 83763/179733 [00:13<00:14, 6518.51 examples/s]
Map (num_proc=8):  42%|████▏     | 75119/179733 [00:12<00:15, 6693.43 examples/s]
Map (num_proc=8):  47%|████▋     | 84726/179733 [00:13<00:14, 6645.96 examples/s]
Map (num_proc=8):  46%|████▌     | 82527/179733 [00:13<00:14, 6873.88 examples/s]
Map (num_proc=8):  33%|███▎      | 60184/179733 [00:10<00:18, 6639.30 examples/s]
Map (num_proc=8):  48%|████▊     | 85832/179733 [00:13<00:13, 6715.04 examples/s]
Map (num_proc=8):  36%|███▋      | 65502/179733 [00:10<00:17, 6532.26 examples/s]
Map (num_proc=8):  45%|████▌     | 81048/179733 [00:13<00:14, 6756.88 examples/s]
Map (num_proc=8):  47%|████▋     | 84442/179733 [00:13<00:14, 6556.46 examples/s]
Map (num_proc=8):  42%|████▏     | 75790/179733 [00:12<00:15, 6665.16 examples/s]
Map (num_proc=8):  48%|████▊     | 85420/179733 [00:13<00:14, 6714.69 examples/s]
Map (num_proc=8):  46%|████▋     | 83215/179733 [00:13<00:14, 6840.48 examples/s]
Map (num_proc=8):  48%|████▊     | 86507/179733 [00:13<00:13, 6699.17 examples/s]
Map (num_proc=8):  34%|███▍      | 60878/179733 [00:10<00:17, 6668.47 examples/s]
Map (num_proc=8):  37%|███▋      | 66181/179733 [00:10<00:17, 6554.87 examples/s]
Map (num_proc=8):  45%|████▌     | 81755/179733 [00:13<00:14, 6785.48 examples/s]
Map (num_proc=8):  47%|████▋     | 85113/179733 [00:13<00:14, 6570.27 examples/s]
Map (num_proc=8):  43%|████▎     | 76466/179733 [00:12<00:15, 6670.31 examples/s]
Map (num_proc=8):  48%|████▊     | 86100/179733 [00:13<00:13, 6729.06 examples/s]
Map (num_proc=8):  47%|████▋     | 83906/179733 [00:13<00:14, 6839.89 examples/s]
Map (num_proc=8):  34%|███▍      | 61573/179733 [00:10<00:17, 6725.86 examples/s]
Map (num_proc=8):  49%|████▊     | 87220/179733 [00:13<00:13, 6608.46 examples/s]
Map (num_proc=8):  37%|███▋      | 66864/179733 [00:11<00:17, 6615.21 examples/s]
Map (num_proc=8):  46%|████▌     | 82444/179733 [00:13<00:14, 6784.73 examples/s]
Map (num_proc=8):  48%|████▊     | 85794/179733 [00:13<00:14, 6605.43 examples/s]
Map (num_proc=8):  43%|████▎     | 77140/179733 [00:12<00:15, 6668.34 examples/s]
Map (num_proc=8):  48%|████▊     | 86790/179733 [00:13<00:13, 6720.12 examples/s]
Map (num_proc=8):  47%|████▋     | 84595/179733 [00:13<00:13, 6822.86 examples/s]
Map (num_proc=8):  35%|███▍      | 62255/179733 [00:10<00:17, 6726.84 examples/s]
Map (num_proc=8):  49%|████▉     | 87897/179733 [00:13<00:13, 6591.93 examples/s]
Map (num_proc=8):  38%|███▊      | 67538/179733 [00:11<00:16, 6625.38 examples/s]
Map (num_proc=8):  46%|████▋     | 83157/179733 [00:13<00:14, 6757.00 examples/s]
Map (num_proc=8):  43%|████▎     | 77813/179733 [00:12<00:15, 6675.02 examples/s]
Map (num_proc=8):  48%|████▊     | 86474/179733 [00:13<00:14, 6614.79 examples/s]
Map (num_proc=8):  49%|████▊     | 87489/179733 [00:13<00:14, 6547.80 examples/s]
Map (num_proc=8):  47%|████▋     | 85310/179733 [00:13<00:13, 6851.25 examples/s]
Map (num_proc=8):  35%|███▌      | 62945/179733 [00:10<00:17, 6756.85 examples/s]
Map (num_proc=8):  49%|████▉     | 88572/179733 [00:13<00:13, 6627.42 examples/s]
Map (num_proc=8):  38%|███▊      | 68211/179733 [00:11<00:16, 6648.11 examples/s]
Map (num_proc=8):  47%|████▋     | 83837/179733 [00:13<00:14, 6735.06 examples/s]
Map (num_proc=8):  44%|████▎     | 78482/179733 [00:12<00:15, 6665.29 examples/s]
Map (num_proc=8):  48%|████▊     | 87147/179733 [00:13<00:13, 6622.93 examples/s]
Map (num_proc=8):  49%|████▉     | 88155/179733 [00:13<00:13, 6566.88 examples/s]
Map (num_proc=8):  48%|████▊     | 86004/179733 [00:13<00:13, 6845.52 examples/s]
Map (num_proc=8):  35%|███▌      | 63649/179733 [00:10<00:17, 6786.42 examples/s]
Map (num_proc=8):  50%|████▉     | 89259/179733 [00:14<00:13, 6669.28 examples/s]
Map (num_proc=8):  38%|███▊      | 68880/179733 [00:11<00:16, 6621.95 examples/s]
Map (num_proc=8):  47%|████▋     | 84528/179733 [00:13<00:14, 6767.69 examples/s]
Map (num_proc=8):  44%|████▍     | 79149/179733 [00:12<00:15, 6634.34 examples/s]
Map (num_proc=8):  49%|████▉     | 87827/179733 [00:14<00:13, 6645.69 examples/s]
Map (num_proc=8):  49%|████▉     | 88832/179733 [00:14<00:13, 6613.48 examples/s]
Map (num_proc=8):  48%|████▊     | 86718/179733 [00:13<00:13, 6729.93 examples/s]
Map (num_proc=8):  36%|███▌      | 64329/179733 [00:10<00:17, 6762.55 examples/s]
Map (num_proc=8):  50%|█████     | 89952/179733 [00:14<00:13, 6732.61 examples/s]
Map (num_proc=8):  39%|███▊      | 69568/179733 [00:11<00:16, 6675.51 examples/s]
Map (num_proc=8):  47%|████▋     | 85250/179733 [00:13<00:13, 6807.54 examples/s]
Map (num_proc=8):  44%|████▍     | 79839/179733 [00:12<00:14, 6669.35 examples/s]
Map (num_proc=8):  49%|████▉     | 88512/179733 [00:14<00:13, 6676.86 examples/s]
Map (num_proc=8):  50%|████▉     | 89522/179733 [00:14<00:13, 6676.76 examples/s]
Map (num_proc=8):  36%|███▌      | 65032/179733 [00:10<00:16, 6785.10 examples/s]
Map (num_proc=8):  49%|████▊     | 87433/179733 [00:14<00:14, 6564.18 examples/s]
Map (num_proc=8):  50%|█████     | 90635/179733 [00:14<00:13, 6726.65 examples/s]
Map (num_proc=8):  39%|███▉      | 70270/179733 [00:11<00:16, 6603.86 examples/s]
Map (num_proc=8):  48%|████▊     | 85956/179733 [00:13<00:13, 6828.40 examples/s]
Map (num_proc=8):  45%|████▍     | 80530/179733 [00:13<00:14, 6706.11 examples/s]
Map (num_proc=8):  50%|████▉     | 89191/179733 [00:14<00:13, 6678.36 examples/s]
Map (num_proc=8):  50%|█████     | 90219/179733 [00:14<00:13, 6482.90 examples/s]
Map (num_proc=8):  37%|███▋      | 65722/179733 [00:10<00:16, 6810.65 examples/s]
Map (num_proc=8):  49%|████▉     | 88106/179733 [00:14<00:13, 6572.12 examples/s]
Map (num_proc=8):  51%|█████     | 91317/179733 [00:14<00:13, 6732.88 examples/s]
Map (num_proc=8):  39%|███▉      | 70942/179733 [00:11<00:16, 6636.00 examples/s]
Map (num_proc=8):  48%|████▊     | 86675/179733 [00:13<00:13, 6868.85 examples/s]
Map (num_proc=8):  50%|████▉     | 89861/179733 [00:14<00:13, 6655.09 examples/s]
Map (num_proc=8):  45%|████▌     | 81243/179733 [00:13<00:14, 6671.67 examples/s]
Map (num_proc=8):  51%|█████     | 90902/179733 [00:14<00:13, 6564.43 examples/s]
Map (num_proc=8):  37%|███▋      | 66412/179733 [00:10<00:16, 6801.40 examples/s]
Map (num_proc=8):  51%|█████     | 91993/179733 [00:14<00:13, 6733.90 examples/s]
Map (num_proc=8):  49%|████▉     | 88801/179733 [00:14<00:13, 6630.94 examples/s]
Map (num_proc=8):  40%|███▉      | 71614/179733 [00:11<00:16, 6619.90 examples/s]
Map (num_proc=8):  49%|████▊     | 87393/179733 [00:13<00:13, 6826.22 examples/s]
Map (num_proc=8):  46%|████▌     | 81918/179733 [00:13<00:14, 6673.25 examples/s]
Map (num_proc=8):  51%|█████     | 91598/179733 [00:14<00:13, 6646.06 examples/s]
Map (num_proc=8):  50%|█████     | 90558/179733 [00:14<00:14, 6125.01 examples/s]
Map (num_proc=8):  37%|███▋      | 67123/179733 [00:11<00:16, 6741.35 examples/s]
Map (num_proc=8):  50%|████▉     | 89469/179733 [00:14<00:13, 6636.14 examples/s]
Map (num_proc=8):  52%|█████▏    | 92693/179733 [00:14<00:12, 6726.49 examples/s]
Map (num_proc=8):  40%|████      | 72294/179733 [00:11<00:16, 6517.61 examples/s]
Map (num_proc=8):  46%|████▌     | 82612/179733 [00:13<00:14, 6735.16 examples/s]
Map (num_proc=8):  49%|████▉     | 88118/179733 [00:14<00:13, 6550.55 examples/s]
Map (num_proc=8):  51%|█████     | 91216/179733 [00:14<00:14, 6231.80 examples/s]
Map (num_proc=8):  51%|█████▏    | 92267/179733 [00:14<00:13, 6469.87 examples/s]
Map (num_proc=8):  38%|███▊      | 67837/179733 [00:11<00:16, 6785.14 examples/s]
Map (num_proc=8):  52%|█████▏    | 93385/179733 [00:14<00:12, 6763.87 examples/s]
Map (num_proc=8):  50%|█████     | 90174/179733 [00:14<00:13, 6423.77 examples/s]
Map (num_proc=8):  41%|████      | 72974/179733 [00:11<00:16, 6552.07 examples/s]
Map (num_proc=8):  46%|████▋     | 83316/179733 [00:13<00:14, 6532.84 examples/s]
Map (num_proc=8):  49%|████▉     | 88811/179733 [00:14<00:13, 6632.88 examples/s]
Map (num_proc=8):  51%|█████     | 91850/179733 [00:14<00:14, 6246.23 examples/s]
Map (num_proc=8):  52%|█████▏    | 92938/179733 [00:14<00:13, 6491.48 examples/s]
Map (num_proc=8):  38%|███▊      | 68549/179733 [00:11<00:16, 6835.07 examples/s]
Map (num_proc=8):  52%|█████▏    | 94091/179733 [00:14<00:12, 6798.61 examples/s]
Map (num_proc=8):  51%|█████     | 90852/179733 [00:14<00:13, 6511.57 examples/s]
Map (num_proc=8):  41%|████      | 73649/179733 [00:12<00:16, 6600.04 examples/s]
Map (num_proc=8):  47%|████▋     | 83976/179733 [00:13<00:14, 6539.90 examples/s]
Map (num_proc=8):  50%|████▉     | 89499/179733 [00:14<00:13, 6681.51 examples/s]
Map (num_proc=8):  51%|█████▏    | 92508/179733 [00:14<00:13, 6331.63 examples/s]
Map (num_proc=8):  52%|█████▏    | 93618/179733 [00:14<00:13, 6545.89 examples/s]
Map (num_proc=8):  39%|███▊      | 69248/179733 [00:11<00:16, 6846.26 examples/s]
Map (num_proc=8):  53%|█████▎    | 94780/179733 [00:14<00:12, 6769.25 examples/s]
Map (num_proc=8):  51%|█████     | 91531/179733 [00:14<00:13, 6577.61 examples/s]
Map (num_proc=8):  41%|████▏     | 74340/179733 [00:12<00:15, 6655.84 examples/s]
Map (num_proc=8):  47%|████▋     | 84656/179733 [00:13<00:14, 6556.61 examples/s]
Map (num_proc=8):  50%|█████     | 90188/179733 [00:14<00:13, 6721.33 examples/s]
Map (num_proc=8):  52%|█████▏    | 93185/179733 [00:14<00:13, 6373.38 examples/s]
Map (num_proc=8):  52%|█████▏    | 94285/179733 [00:14<00:13, 6555.58 examples/s]
Map (num_proc=8):  39%|███▉      | 69975/179733 [00:11<00:15, 6918.80 examples/s]
Map (num_proc=8):  51%|█████▏    | 92211/179733 [00:14<00:13, 6620.84 examples/s]
Map (num_proc=8):  53%|█████▎    | 95463/179733 [00:15<00:13, 6421.46 examples/s]
Map (num_proc=8):  42%|████▏     | 75044/179733 [00:12<00:16, 6527.60 examples/s]
Map (num_proc=8):  47%|████▋     | 85323/179733 [00:13<00:14, 6580.13 examples/s]
Map (num_proc=8):  51%|█████     | 90871/179733 [00:14<00:13, 6742.27 examples/s]
Map (num_proc=8):  53%|█████▎    | 94965/179733 [00:14<00:12, 6590.39 examples/s]
Map (num_proc=8):  52%|█████▏    | 93858/179733 [00:14<00:13, 6356.48 examples/s]
Map (num_proc=8):  52%|█████▏    | 92896/179733 [00:14<00:13, 6627.89 examples/s]
Map (num_proc=8):  39%|███▉      | 70690/179733 [00:11<00:16, 6626.96 examples/s]
Map (num_proc=8):  53%|█████▎    | 96111/179733 [00:15<00:13, 6426.26 examples/s]
Map (num_proc=8):  42%|████▏     | 75732/179733 [00:12<00:15, 6568.55 examples/s]
Map (num_proc=8):  51%|█████     | 91551/179733 [00:14<00:13, 6724.94 examples/s]
Map (num_proc=8):  48%|████▊     | 86014/179733 [00:13<00:14, 6508.17 examples/s]
Map (num_proc=8):  53%|█████▎    | 95630/179733 [00:15<00:12, 6507.56 examples/s]
Map (num_proc=8):  53%|█████▎    | 94516/179733 [00:15<00:13, 6253.08 examples/s]
Map (num_proc=8):  52%|█████▏    | 93563/179733 [00:14<00:12, 6633.89 examples/s]
Map (num_proc=8):  40%|███▉      | 71356/179733 [00:11<00:16, 6596.20 examples/s]
Map (num_proc=8):  54%|█████▍    | 96782/179733 [00:15<00:12, 6482.64 examples/s]
Map (num_proc=8):  43%|████▎     | 76431/179733 [00:12<00:15, 6635.92 examples/s]
Map (num_proc=8):  51%|█████▏    | 92238/179733 [00:14<00:12, 6753.86 examples/s]
Map (num_proc=8):  48%|████▊     | 86712/179733 [00:13<00:14, 6614.03 examples/s]
Map (num_proc=8):  54%|█████▎    | 96317/179733 [00:15<00:12, 6546.67 examples/s]
Map (num_proc=8):  53%|█████▎    | 95166/179733 [00:15<00:13, 6307.01 examples/s]
Map (num_proc=8):  52%|█████▏    | 94256/179733 [00:15<00:13, 6571.53 examples/s]
Map (num_proc=8):  54%|█████▍    | 97459/179733 [00:15<00:12, 6333.35 examples/s]
Map (num_proc=8):  40%|████      | 72029/179733 [00:11<00:17, 6258.09 examples/s]
Map (num_proc=8):  43%|████▎     | 77109/179733 [00:12<00:15, 6452.95 examples/s]
Map (num_proc=8):  49%|████▊     | 87398/179733 [00:14<00:13, 6637.91 examples/s]
Map (num_proc=8):  52%|█████▏    | 92937/179733 [00:14<00:13, 6632.81 examples/s]
Map (num_proc=8):  53%|█████▎    | 95817/179733 [00:15<00:13, 6333.15 examples/s]
Map (num_proc=8):  54%|█████▍    | 97007/179733 [00:15<00:12, 6540.86 examples/s]
Map (num_proc=8):  53%|█████▎    | 94920/179733 [00:15<00:12, 6571.56 examples/s]
Map (num_proc=8):  55%|█████▍    | 98108/179733 [00:15<00:12, 6353.62 examples/s]
Map (num_proc=8):  40%|████      | 72708/179733 [00:11<00:16, 6396.91 examples/s]
Map (num_proc=8):  43%|████▎     | 77775/179733 [00:12<00:15, 6478.70 examples/s]
Map (num_proc=8):  49%|████▉     | 88077/179733 [00:14<00:13, 6676.69 examples/s]
Map (num_proc=8):  52%|█████▏    | 93627/179733 [00:14<00:12, 6674.76 examples/s]
Map (num_proc=8):  54%|█████▎    | 96456/179733 [00:15<00:13, 6319.27 examples/s]
Map (num_proc=8):  54%|█████▍    | 97663/179733 [00:15<00:12, 6520.60 examples/s]
Map (num_proc=8):  53%|█████▎    | 95584/179733 [00:15<00:12, 6565.04 examples/s]
Map (num_proc=8):  55%|█████▍    | 98751/179733 [00:15<00:12, 6369.87 examples/s]
Map (num_proc=8):  41%|████      | 73393/179733 [00:12<00:16, 6502.53 examples/s]
Map (num_proc=8):  44%|████▎     | 78464/179733 [00:12<00:15, 6569.79 examples/s]
Map (num_proc=8):  52%|█████▏    | 94325/179733 [00:15<00:12, 6674.21 examples/s]
Map (num_proc=8):  49%|████▉     | 88786/179733 [00:14<00:13, 6515.48 examples/s]
Map (num_proc=8):  54%|█████▍    | 97108/179733 [00:15<00:12, 6365.28 examples/s]
Map (num_proc=8):  55%|█████▍    | 98359/179733 [00:15<00:12, 6541.88 examples/s]
Map (num_proc=8):  54%|█████▎    | 96250/179733 [00:15<00:12, 6571.55 examples/s]
Map (num_proc=8):  55%|█████▌    | 99410/179733 [00:15<00:12, 6326.06 examples/s]
Map (num_proc=8):  41%|████      | 74081/179733 [00:12<00:16, 6579.61 examples/s]
Map (num_proc=8):  44%|████▍     | 79129/179733 [00:12<00:15, 6555.16 examples/s]
Map (num_proc=8):  53%|█████▎    | 95022/179733 [00:15<00:12, 6610.56 examples/s]
Map (num_proc=8):  50%|████▉     | 89467/179733 [00:14<00:13, 6577.72 examples/s]
Map (num_proc=8):  54%|█████▍    | 97780/179733 [00:15<00:12, 6431.81 examples/s]
Map (num_proc=8):  55%|█████▌    | 99047/179733 [00:15<00:12, 6278.82 examples/s]
Map (num_proc=8):  54%|█████▍    | 96909/179733 [00:15<00:12, 6550.96 examples/s]
Map (num_proc=8):  56%|█████▌    | 100050/179733 [00:15<00:12, 6333.52 examples/s]
Map (num_proc=8):  42%|████▏     | 74753/179733 [00:12<00:15, 6604.20 examples/s]
Map (num_proc=8):  44%|████▍     | 79796/179733 [00:12<00:15, 6567.36 examples/s]
Map (num_proc=8):  50%|█████     | 90159/179733 [00:14<00:13, 6654.57 examples/s]
Map (num_proc=8):  53%|█████▎    | 95713/179733 [00:15<00:12, 6624.00 examples/s]
Map (num_proc=8):  55%|█████▍    | 98455/179733 [00:15<00:12, 6462.00 examples/s]
Map (num_proc=8):  55%|█████▌    | 99712/179733 [00:15<00:12, 6195.10 examples/s]
Map (num_proc=8):  54%|█████▍    | 97596/179733 [00:15<00:12, 6597.79 examples/s]
Map (num_proc=8):  56%|█████▌    | 100714/179733 [00:15<00:12, 6404.73 examples/s]
Map (num_proc=8):  42%|████▏     | 75446/179733 [00:12<00:15, 6595.88 examples/s]
Map (num_proc=8):  45%|████▍     | 80468/179733 [00:13<00:15, 6577.71 examples/s]
Map (num_proc=8):  51%|█████     | 90860/179733 [00:14<00:13, 6668.10 examples/s]
Map (num_proc=8):  54%|█████▎    | 96420/179733 [00:15<00:12, 6643.85 examples/s]
Map (num_proc=8):  55%|█████▌    | 99118/179733 [00:15<00:12, 6488.73 examples/s]
Map (num_proc=8):  56%|█████▌    | 100363/179733 [00:15<00:12, 6260.40 examples/s]
Map (num_proc=8):  55%|█████▍    | 98279/179733 [00:15<00:12, 6615.99 examples/s]
Map (num_proc=8):  42%|████▏     | 76120/179733 [00:12<00:15, 6627.57 examples/s]
Map (num_proc=8):  56%|█████▋    | 101397/179733 [00:15<00:12, 6329.28 examples/s]
Map (num_proc=8):  45%|████▌     | 81159/179733 [00:13<00:15, 6559.97 examples/s]
Map (num_proc=8):  54%|█████▍    | 97096/179733 [00:15<00:12, 6653.26 examples/s]
Map (num_proc=8):  56%|█████▌    | 99772/179733 [00:15<00:12, 6454.40 examples/s]
Map (num_proc=8):  51%|█████     | 91557/179733 [00:14<00:13, 6575.48 examples/s]
Map (num_proc=8):  56%|█████▌    | 101006/179733 [00:15<00:12, 6301.91 examples/s]
Map (num_proc=8):  55%|█████▌    | 98975/179733 [00:15<00:12, 6500.48 examples/s]
Map (num_proc=8):  43%|████▎     | 76815/179733 [00:12<00:15, 6683.98 examples/s]
Map (num_proc=8):  57%|█████▋    | 102058/179733 [00:16<00:12, 6387.84 examples/s]
Map (num_proc=8):  46%|████▌     | 81831/179733 [00:13<00:15, 6459.69 examples/s]
Map (num_proc=8):  51%|█████▏    | 92250/179733 [00:14<00:13, 6647.11 examples/s]
Map (num_proc=8):  54%|█████▍    | 97787/179733 [00:15<00:12, 6539.40 examples/s]
Map (num_proc=8):  56%|█████▌    | 100435/179733 [00:16<00:12, 6407.06 examples/s]
Map (num_proc=8):  57%|█████▋    | 101671/179733 [00:16<00:12, 6230.65 examples/s]
Map (num_proc=8):  55%|█████▌    | 99626/179733 [00:15<00:12, 6489.07 examples/s]
Map (num_proc=8):  43%|████▎     | 77495/179733 [00:12<00:15, 6688.39 examples/s]
Map (num_proc=8):  57%|█████▋    | 102715/179733 [00:16<00:11, 6422.75 examples/s]
Map (num_proc=8):  46%|████▌     | 82502/179733 [00:13<00:14, 6521.52 examples/s]
Map (num_proc=8):  52%|█████▏    | 92916/179733 [00:14<00:13, 6630.51 examples/s]
Map (num_proc=8):  55%|█████▍    | 98456/179733 [00:15<00:12, 6557.36 examples/s]
Map (num_proc=8):  56%|█████▋    | 101109/179733 [00:16<00:12, 6092.00 examples/s]
Map (num_proc=8):  57%|█████▋    | 102307/179733 [00:16<00:12, 6244.50 examples/s]
Map (num_proc=8):  56%|█████▌    | 100292/179733 [00:15<00:12, 6516.79 examples/s]
Map (num_proc=8):  43%|████▎     | 78176/179733 [00:12<00:15, 6700.89 examples/s]
Map (num_proc=8):  58%|█████▊    | 103365/179733 [00:16<00:11, 6435.23 examples/s]
Map (num_proc=8):  46%|████▋     | 83170/179733 [00:13<00:14, 6549.04 examples/s]
Map (num_proc=8):  52%|█████▏    | 93588/179733 [00:14<00:13, 6605.93 examples/s]
Map (num_proc=8):  55%|█████▌    | 99130/179733 [00:15<00:12, 6571.36 examples/s]
Map (num_proc=8):  57%|█████▋    | 101754/179733 [00:16<00:12, 6142.64 examples/s]
Map (num_proc=8):  57%|█████▋    | 102960/179733 [00:16<00:12, 6309.70 examples/s]
Map (num_proc=8):  56%|█████▌    | 100960/179733 [00:16<00:12, 6529.98 examples/s]
Map (num_proc=8):  58%|█████▊    | 104022/179733 [00:16<00:11, 6456.67 examples/s]
Map (num_proc=8):  44%|████▍     | 78862/179733 [00:12<00:15, 6716.43 examples/s]
Map (num_proc=8):  47%|████▋     | 83837/179733 [00:13<00:14, 6551.90 examples/s]
Map (num_proc=8):  52%|█████▏    | 94266/179733 [00:15<00:13, 6350.96 examples/s]
Map (num_proc=8):  56%|█████▌    | 99819/179733 [00:15<00:12, 6352.69 examples/s]
Map (num_proc=8):  57%|█████▋    | 102393/179733 [00:16<00:12, 6179.34 examples/s]
Map (num_proc=8):  58%|█████▊    | 103631/179733 [00:16<00:11, 6420.96 examples/s]
Map (num_proc=8):  57%|█████▋    | 101620/179733 [00:16<00:11, 6535.48 examples/s]
Map (num_proc=8):  58%|█████▊    | 104673/179733 [00:16<00:11, 6442.50 examples/s]
Map (num_proc=8):  44%|████▍     | 79565/179733 [00:12<00:14, 6702.96 examples/s]
Map (num_proc=8):  47%|████▋     | 84528/179733 [00:13<00:14, 6462.12 examples/s]
Map (num_proc=8):  56%|█████▌    | 100461/179733 [00:15<00:12, 6346.60 examples/s]
Map (num_proc=8):  53%|█████▎    | 94925/179733 [00:15<00:13, 6258.58 examples/s]
Map (num_proc=8):  57%|█████▋    | 103027/179733 [00:16<00:12, 6209.54 examples/s]
Map (num_proc=8):  58%|█████▊    | 104289/179733 [00:16<00:11, 6447.33 examples/s]
Map (num_proc=8):  59%|█████▊    | 105329/179733 [00:16<00:11, 6444.81 examples/s]
Map (num_proc=8):  57%|█████▋    | 102298/179733 [00:16<00:12, 6260.27 examples/s]
Map (num_proc=8):  45%|████▍     | 80262/179733 [00:13<00:14, 6726.09 examples/s]
Map (num_proc=8):  47%|████▋     | 85203/179733 [00:13<00:14, 6528.30 examples/s]
Map (num_proc=8):  56%|█████▋    | 101132/179733 [00:16<00:12, 6389.76 examples/s]
Map (num_proc=8):  53%|█████▎    | 95579/179733 [00:15<00:13, 6334.89 examples/s]
Map (num_proc=8):  58%|█████▊    | 103678/179733 [00:16<00:12, 6106.23 examples/s]
Map (num_proc=8):  58%|█████▊    | 104973/179733 [00:16<00:11, 6425.15 examples/s]
Map (num_proc=8):  59%|█████▉    | 105978/179733 [00:16<00:11, 6417.68 examples/s]
Map (num_proc=8):  57%|█████▋    | 102973/179733 [00:16<00:12, 6382.52 examples/s]
Map (num_proc=8):  45%|████▌     | 80976/179733 [00:13<00:14, 6764.60 examples/s]
Map (num_proc=8):  48%|████▊     | 85862/179733 [00:13<00:14, 6515.63 examples/s]
Map (num_proc=8):  57%|█████▋    | 101779/179733 [00:16<00:12, 6396.86 examples/s]
Map (num_proc=8):  54%|█████▎    | 96240/179733 [00:15<00:13, 6369.38 examples/s]
Map (num_proc=8):  58%|█████▊    | 104325/179733 [00:16<00:12, 6168.55 examples/s]
Map (num_proc=8):  59%|█████▉    | 105621/179733 [00:16<00:11, 6408.98 examples/s]
Map (num_proc=8):  59%|█████▉    | 106638/179733 [00:16<00:11, 6436.24 examples/s]
Map (num_proc=8):  58%|█████▊    | 103640/179733 [00:16<00:11, 6447.33 examples/s]
Map (num_proc=8):  45%|████▌     | 81692/179733 [00:13<00:14, 6778.12 examples/s]
Map (num_proc=8):  48%|████▊     | 86530/179733 [00:14<00:14, 6540.71 examples/s]
Map (num_proc=8):  57%|█████▋    | 102433/179733 [00:16<00:12, 6422.05 examples/s]
Map (num_proc=8):  54%|█████▍    | 96883/179733 [00:15<00:13, 6367.49 examples/s]
Map (num_proc=8):  58%|█████▊    | 104955/179733 [00:16<00:12, 6190.49 examples/s]
Map (num_proc=8):  59%|█████▉    | 106285/179733 [00:16<00:11, 6445.45 examples/s]
Map (num_proc=8):  58%|█████▊    | 104290/179733 [00:16<00:11, 6439.33 examples/s]
Map (num_proc=8):  60%|█████▉    | 107311/179733 [00:16<00:11, 6440.10 examples/s]
Map (num_proc=8):  46%|████▌     | 82415/179733 [00:13<00:14, 6636.82 examples/s]
Map (num_proc=8):  49%|████▊     | 87204/179733 [00:14<00:14, 6561.91 examples/s]
Map (num_proc=8):  57%|█████▋    | 103110/179733 [00:16<00:11, 6500.85 examples/s]
Map (num_proc=8):  54%|█████▍    | 97526/179733 [00:15<00:12, 6374.82 examples/s]
Map (num_proc=8):  59%|█████▊    | 105586/179733 [00:16<00:11, 6209.04 examples/s]
Map (num_proc=8):  60%|█████▉    | 106964/179733 [00:16<00:11, 6414.89 examples/s]
Map (num_proc=8):  60%|██████    | 107967/179733 [00:16<00:11, 6462.48 examples/s]
Map (num_proc=8):  46%|████▌     | 83088/179733 [00:13<00:14, 6649.82 examples/s]
Map (num_proc=8):  58%|█████▊    | 104959/179733 [00:16<00:12, 6122.86 examples/s]
Map (num_proc=8):  49%|████▉     | 87874/179733 [00:14<00:13, 6586.66 examples/s]
Map (num_proc=8):  58%|█████▊    | 103785/179733 [00:16<00:11, 6557.82 examples/s]
Map (num_proc=8):  55%|█████▍    | 98201/179733 [00:15<00:13, 6239.63 examples/s]
Map (num_proc=8):  59%|█████▉    | 106222/179733 [00:16<00:11, 6226.13 examples/s]
Map (num_proc=8):  60%|█████▉    | 107606/179733 [00:16<00:11, 6397.34 examples/s]
Map (num_proc=8):  60%|██████    | 108616/179733 [00:17<00:11, 6412.85 examples/s]
Map (num_proc=8):  47%|████▋     | 83765/179733 [00:13<00:14, 6659.33 examples/s]
Map (num_proc=8):  59%|█████▉    | 105620/179733 [00:16<00:11, 6244.69 examples/s]
Map (num_proc=8):  49%|████▉     | 88544/179733 [00:14<00:13, 6555.94 examples/s]
Map (num_proc=8):  58%|█████▊    | 104471/179733 [00:16<00:11, 6389.48 examples/s]
Map (num_proc=8):  59%|█████▉    | 106853/179733 [00:17<00:11, 6243.83 examples/s]
Map (num_proc=8):  55%|█████▌    | 98863/179733 [00:15<00:12, 6311.09 examples/s]
Map (num_proc=8):  60%|██████    | 108280/179733 [00:17<00:11, 6406.90 examples/s]
Map (num_proc=8):  61%|██████    | 109270/179733 [00:17<00:11, 6398.42 examples/s]
Map (num_proc=8):  59%|█████▉    | 106254/179733 [00:16<00:11, 6246.75 examples/s]
Map (num_proc=8):  47%|████▋     | 84442/179733 [00:13<00:14, 6659.26 examples/s]
Map (num_proc=8):  50%|████▉     | 89208/179733 [00:14<00:13, 6546.57 examples/s]
Map (num_proc=8):  58%|█████▊    | 105140/179733 [00:16<00:11, 6443.83 examples/s]
Map (num_proc=8):  60%|█████▉    | 107493/179733 [00:17<00:11, 6251.29 examples/s]
Map (num_proc=8):  55%|█████▌    | 99526/179733 [00:15<00:12, 6361.50 examples/s]
Map (num_proc=8):  61%|██████    | 108963/179733 [00:17<00:10, 6462.81 examples/s]
Map (num_proc=8):  61%|██████    | 109929/179733 [00:17<00:10, 6436.71 examples/s]
Map (num_proc=8):  47%|████▋     | 85113/179733 [00:13<00:14, 6644.21 examples/s]
Map (num_proc=8):  59%|█████▉    | 106911/179733 [00:17<00:11, 6303.76 examples/s]
Map (num_proc=8):  50%|█████     | 89904/179733 [00:14<00:13, 6632.05 examples/s]
Map (num_proc=8):  59%|█████▉    | 105790/179733 [00:16<00:11, 6420.48 examples/s]
Map (num_proc=8):  56%|█████▌    | 100166/179733 [00:16<00:12, 6358.97 examples/s]
Map (num_proc=8):  60%|██████    | 108139/179733 [00:17<00:11, 6285.29 examples/s]
Map (num_proc=8):  61%|██████    | 109648/179733 [00:17<00:11, 6255.81 examples/s]
Map (num_proc=8):  62%|██████▏   | 110595/179733 [00:17<00:10, 6467.28 examples/s]
Map (num_proc=8):  48%|████▊     | 85787/179733 [00:13<00:14, 6654.25 examples/s]
Map (num_proc=8):  60%|█████▉    | 107571/179733 [00:17<00:11, 6332.09 examples/s]
Map (num_proc=8):  50%|█████     | 90577/179733 [00:14<00:13, 6639.49 examples/s]
Map (num_proc=8):  59%|█████▉    | 106471/179733 [00:16<00:11, 6484.15 examples/s]
Map (num_proc=8):  61%|██████    | 108779/179733 [00:17<00:11, 6295.79 examples/s]
Map (num_proc=8):  56%|█████▌    | 100811/179733 [00:16<00:12, 6348.13 examples/s]
Map (num_proc=8):  61%|██████▏   | 110327/179733 [00:17<00:10, 6390.10 examples/s]
Map (num_proc=8):  62%|██████▏   | 111245/179733 [00:17<00:10, 6455.65 examples/s]
Map (num_proc=8):  60%|██████    | 108235/179733 [00:17<00:11, 6391.01 examples/s]
Map (num_proc=8):  51%|█████     | 91242/179733 [00:14<00:13, 6620.01 examples/s]
Map (num_proc=8):  48%|████▊     | 86495/179733 [00:13<00:14, 6406.84 examples/s]
Map (num_proc=8):  60%|█████▉    | 107157/179733 [00:17<00:11, 6545.43 examples/s]
Map (num_proc=8):  56%|█████▋    | 101463/179733 [00:16<00:12, 6358.12 examples/s]
Map (num_proc=8):  61%|██████    | 109436/179733 [00:17<00:11, 6214.40 examples/s]
Map (num_proc=8):  62%|██████▏   | 110978/179733 [00:17<00:10, 6395.34 examples/s]
Map (num_proc=8):  62%|██████▏   | 111908/179733 [00:17<00:10, 6474.03 examples/s]
Map (num_proc=8):  51%|█████     | 91912/179733 [00:14<00:13, 6632.44 examples/s]
Map (num_proc=8):  61%|██████    | 108919/179733 [00:17<00:10, 6495.42 examples/s]
Map (num_proc=8):  48%|████▊     | 87143/179733 [00:14<00:14, 6369.65 examples/s]
Map (num_proc=8):  57%|█████▋    | 102119/179733 [00:16<00:12, 6397.92 examples/s]
Map (num_proc=8):  60%|██████    | 107845/179733 [00:17<00:11, 6487.59 examples/s]
Map (num_proc=8):  61%|██████    | 110084/179733 [00:17<00:11, 6274.68 examples/s]
Map (num_proc=8):  62%|██████▏   | 111629/179733 [00:17<00:10, 6413.00 examples/s]
Map (num_proc=8):  63%|██████▎   | 112558/179733 [00:17<00:10, 6470.14 examples/s]
Map (num_proc=8):  61%|██████    | 109605/179733 [00:17<00:10, 6554.68 examples/s]
Map (num_proc=8):  52%|█████▏    | 92600/179733 [00:14<00:13, 6510.71 examples/s]
Map (num_proc=8):  49%|████▉     | 87805/179733 [00:14<00:14, 6393.73 examples/s]
Map (num_proc=8):  57%|█████▋    | 102774/179733 [00:16<00:12, 6412.71 examples/s]
Map (num_proc=8):  60%|██████    | 108496/179733 [00:17<00:11, 6472.96 examples/s]
Map (num_proc=8):  62%|██████▏   | 110733/179733 [00:17<00:10, 6300.82 examples/s]
Map (num_proc=8):  62%|██████▏   | 112276/179733 [00:17<00:10, 6396.02 examples/s]
Map (num_proc=8):  63%|██████▎   | 113225/179733 [00:17<00:10, 6503.55 examples/s]
Map (num_proc=8):  52%|█████▏    | 93264/179733 [00:15<00:13, 6529.42 examples/s]
Map (num_proc=8):  61%|██████▏   | 110283/179733 [00:17<00:10, 6452.65 examples/s]
Map (num_proc=8):  49%|████▉     | 88488/179733 [00:14<00:14, 6507.62 examples/s]
Map (num_proc=8):  58%|█████▊    | 103445/179733 [00:16<00:11, 6487.92 examples/s]
Map (num_proc=8):  61%|██████    | 109151/179733 [00:17<00:10, 6493.76 examples/s]
Map (num_proc=8):  62%|██████▏   | 111390/179733 [00:17<00:10, 6360.55 examples/s]
Map (num_proc=8):  63%|██████▎   | 112936/179733 [00:17<00:10, 6432.43 examples/s]
Map (num_proc=8):  63%|██████▎   | 113881/179733 [00:17<00:10, 6493.65 examples/s]
Map (num_proc=8):  52%|█████▏    | 93934/179733 [00:15<00:13, 6565.08 examples/s]
Map (num_proc=8):  50%|████▉     | 89168/179733 [00:14<00:13, 6567.06 examples/s]
Map (num_proc=8):  62%|██████▏   | 110964/179733 [00:17<00:11, 6183.76 examples/s]
Map (num_proc=8):  58%|█████▊    | 104131/179733 [00:16<00:11, 6508.72 examples/s]
Map (num_proc=8):  61%|██████    | 109804/179733 [00:17<00:10, 6474.89 examples/s]
Map (num_proc=8):  62%|██████▏   | 112030/179733 [00:17<00:10, 6337.92 examples/s]
Map (num_proc=8):  63%|██████▎   | 113595/179733 [00:17<00:10, 6472.28 examples/s]
Map (num_proc=8):  64%|██████▎   | 114536/179733 [00:17<00:10, 6474.19 examples/s]
Map (num_proc=8):  53%|█████▎    | 94624/179733 [00:15<00:12, 6617.38 examples/s]
Map (num_proc=8):  50%|████▉     | 89863/179733 [00:14<00:13, 6622.74 examples/s]
Map (num_proc=8):  62%|██████▏   | 111600/179733 [00:17<00:11, 6146.88 examples/s]
Map (num_proc=8):  61%|██████▏   | 110482/179733 [00:17<00:10, 6550.70 examples/s]
Map (num_proc=8):  63%|██████▎   | 112689/179733 [00:17<00:10, 6337.40 examples/s]
Map (num_proc=8):  58%|█████▊    | 104795/179733 [00:16<00:11, 6313.68 examples/s]
Map (num_proc=8):  64%|██████▍   | 115215/179733 [00:18<00:09, 6544.27 examples/s]
Map (num_proc=8):  64%|██████▎   | 114287/179733 [00:18<00:10, 6276.98 examples/s]
Map (num_proc=8):  53%|█████▎    | 95312/179733 [00:15<00:13, 6456.92 examples/s]
Map (num_proc=8):  50%|█████     | 90545/179733 [00:14<00:13, 6509.26 examples/s]
Map (num_proc=8):  62%|██████▏   | 112238/179733 [00:17<00:10, 6178.17 examples/s]
Map (num_proc=8):  62%|██████▏   | 111185/179733 [00:17<00:10, 6605.29 examples/s]
Map (num_proc=8):  63%|██████▎   | 113346/179733 [00:18<00:10, 6388.88 examples/s]
Map (num_proc=8):  59%|█████▊    | 105435/179733 [00:16<00:11, 6305.13 examples/s]
Map (num_proc=8):  64%|██████▍   | 115877/179733 [00:18<00:09, 6546.94 examples/s]
Map (num_proc=8):  64%|██████▍   | 114939/179733 [00:18<00:10, 6322.36 examples/s]
Map (num_proc=8):  53%|█████▎    | 95983/179733 [00:15<00:13, 6377.91 examples/s]
Map (num_proc=8):  51%|█████     | 91241/179733 [00:14<00:13, 6600.79 examples/s]
Map (num_proc=8):  63%|██████▎   | 112870/179733 [00:17<00:10, 6202.54 examples/s]
Map (num_proc=8):  62%|██████▏   | 111846/179733 [00:17<00:10, 6543.69 examples/s]
Map (num_proc=8):  63%|██████▎   | 113988/179733 [00:18<00:10, 6362.23 examples/s]
Map (num_proc=8):  59%|█████▉    | 106092/179733 [00:16<00:11, 6357.15 examples/s]
Map (num_proc=8):  65%|██████▍   | 116569/179733 [00:18<00:09, 6536.92 examples/s]
Map (num_proc=8):  64%|██████▍   | 115602/179733 [00:18<00:10, 6379.95 examples/s]
Map (num_proc=8):  54%|█████▍    | 96626/179733 [00:15<00:13, 6369.80 examples/s]
Map (num_proc=8):  51%|█████     | 91907/179733 [00:14<00:13, 6596.71 examples/s]
Map (num_proc=8):  63%|██████▎   | 113508/179733 [00:18<00:10, 6230.51 examples/s]
Map (num_proc=8):  63%|██████▎   | 112532/179733 [00:17<00:10, 6607.19 examples/s]
Map (num_proc=8):  64%|██████▍   | 114625/179733 [00:18<00:10, 6351.98 examples/s]
Map (num_proc=8):  59%|█████▉    | 106734/179733 [00:17<00:11, 6364.71 examples/s]
Map (num_proc=8):  65%|██████▌   | 117225/179733 [00:18<00:09, 6521.26 examples/s]
Map (num_proc=8):  65%|██████▍   | 116282/179733 [00:18<00:09, 6421.65 examples/s]
Map (num_proc=8):  54%|█████▍    | 97291/179733 [00:15<00:12, 6426.95 examples/s]
Map (num_proc=8):  52%|█████▏    | 92577/179733 [00:14<00:13, 6599.30 examples/s]
Map (num_proc=8):  64%|██████▎   | 114160/179733 [00:18<00:10, 6282.89 examples/s]
Map (num_proc=8):  64%|██████▍   | 115294/179733 [00:18<00:10, 6420.95 examples/s]
Map (num_proc=8):  63%|██████▎   | 113222/179733 [00:17<00:10, 6546.65 examples/s]
Map (num_proc=8):  60%|█████▉    | 107398/179733 [00:17<00:11, 6357.88 examples/s]
Map (num_proc=8):  66%|██████▌   | 117903/179733 [00:18<00:09, 6426.37 examples/s]
Map (num_proc=8):  65%|██████▌   | 116961/179733 [00:18<00:09, 6497.67 examples/s]
Map (num_proc=8):  52%|█████▏    | 93251/179733 [00:15<00:13, 6612.02 examples/s]
Map (num_proc=8):  55%|█████▍    | 97967/179733 [00:15<00:12, 6420.66 examples/s]
Map (num_proc=8):  64%|██████▍   | 114822/179733 [00:18<00:10, 6340.51 examples/s]
Map (num_proc=8):  65%|██████▍   | 115945/179733 [00:18<00:10, 6361.01 examples/s]
Map (num_proc=8):  63%|██████▎   | 113908/179733 [00:18<00:09, 6617.99 examples/s]
Map (num_proc=8):  60%|██████    | 108068/179733 [00:17<00:11, 6399.71 examples/s]
Map (num_proc=8):  65%|██████▌   | 117625/179733 [00:18<00:09, 6481.98 examples/s]
Map (num_proc=8):  66%|██████▌   | 118582/179733 [00:18<00:09, 6339.01 examples/s]
Map (num_proc=8):  55%|█████▍    | 98625/179733 [00:15<00:12, 6455.63 examples/s]
Map (num_proc=8):  52%|█████▏    | 93941/179733 [00:15<00:12, 6611.10 examples/s]
Map (num_proc=8):  64%|██████▍   | 115473/179733 [00:18<00:10, 6363.80 examples/s]
Map (num_proc=8):  65%|██████▍   | 116599/179733 [00:18<00:09, 6396.43 examples/s]
Map (num_proc=8):  64%|██████▍   | 114597/179733 [00:18<00:09, 6590.66 examples/s]
Map (num_proc=8):  60%|██████    | 108718/179733 [00:17<00:11, 6391.45 examples/s]
Map (num_proc=8):  66%|██████▌   | 118285/179733 [00:18<00:09, 6489.78 examples/s]
Map (num_proc=8):  66%|██████▋   | 119244/179733 [00:18<00:09, 6403.56 examples/s]
Map (num_proc=8):  53%|█████▎    | 94626/179733 [00:15<00:12, 6660.47 examples/s]
Map (num_proc=8):  55%|█████▌    | 99279/179733 [00:15<00:12, 6264.55 examples/s]
Map (num_proc=8):  65%|██████▍   | 116139/179733 [00:18<00:10, 6145.27 examples/s]
Map (num_proc=8):  65%|██████▌   | 117267/179733 [00:18<00:09, 6390.36 examples/s]
Map (num_proc=8):  64%|██████▍   | 115257/179733 [00:18<00:09, 6560.59 examples/s]
Map (num_proc=8):  61%|██████    | 109379/179733 [00:17<00:10, 6420.82 examples/s]
Map (num_proc=8):  67%|██████▋   | 119890/179733 [00:18<00:09, 6395.14 examples/s]
Map (num_proc=8):  66%|██████▌   | 118967/179733 [00:18<00:09, 6406.04 examples/s]
Map (num_proc=8):  53%|█████▎    | 95311/179733 [00:15<00:12, 6677.45 examples/s]
Map (num_proc=8):  56%|█████▌    | 99923/179733 [00:16<00:12, 6305.90 examples/s]
Map (num_proc=8):  65%|██████▍   | 116762/179733 [00:18<00:10, 6158.81 examples/s]
Map (num_proc=8):  66%|██████▌   | 117927/179733 [00:18<00:09, 6421.65 examples/s]
Map (num_proc=8):  64%|██████▍   | 115926/179733 [00:18<00:09, 6572.69 examples/s]
Map (num_proc=8):  61%|██████    | 110027/179733 [00:17<00:10, 6398.43 examples/s]
Map (num_proc=8):  67%|██████▋   | 120552/179733 [00:18<00:09, 6427.03 examples/s]
Map (num_proc=8):  67%|██████▋   | 119622/179733 [00:18<00:09, 6422.86 examples/s]
Map (num_proc=8):  53%|█████▎    | 96010/179733 [00:15<00:12, 6659.27 examples/s]
Map (num_proc=8):  56%|█████▌    | 100576/179733 [00:16<00:12, 6344.98 examples/s]
Map (num_proc=8):  65%|██████▌   | 117402/179733 [00:18<00:10, 6192.84 examples/s]
Map (num_proc=8):  65%|██████▍   | 116584/179733 [00:18<00:09, 6563.84 examples/s]
Map (num_proc=8):  66%|██████▌   | 118604/179733 [00:18<00:09, 6445.38 examples/s]
Map (num_proc=8):  62%|██████▏   | 110669/179733 [00:17<00:10, 6388.50 examples/s]
Map (num_proc=8):  67%|██████▋   | 121214/179733 [00:19<00:09, 6440.04 examples/s]
Map (num_proc=8):  67%|██████▋   | 120280/179733 [00:18<00:09, 6437.08 examples/s]
Map (num_proc=8):  56%|█████▋    | 101219/179733 [00:16<00:12, 6320.55 examples/s]
Map (num_proc=8):  54%|█████▍    | 96700/179733 [00:15<00:12, 6496.29 examples/s]
Map (num_proc=8):  66%|██████▌   | 118051/179733 [00:18<00:09, 6270.98 examples/s]
Map (num_proc=8):  66%|██████▋   | 119253/179733 [00:19<00:09, 6434.67 examples/s]
Map (num_proc=8):  62%|██████▏   | 111315/179733 [00:17<00:10, 6375.24 examples/s]
Map (num_proc=8):  65%|██████▌   | 117281/179733 [00:18<00:10, 6129.12 examples/s]
Map (num_proc=8):  68%|██████▊   | 121902/179733 [00:19<00:08, 6542.69 examples/s]
Map (num_proc=8):  67%|██████▋   | 120931/179733 [00:19<00:09, 6435.23 examples/s]
Map (num_proc=8):  57%|█████▋    | 101880/179733 [00:16<00:12, 6381.35 examples/s]
Map (num_proc=8):  66%|██████▌   | 118707/179733 [00:18<00:09, 6244.01 examples/s]
Map (num_proc=8):  67%|██████▋   | 119899/179733 [00:19<00:09, 6420.07 examples/s]
Map (num_proc=8):  54%|█████▍    | 97384/179733 [00:15<00:13, 6170.81 examples/s]
Map (num_proc=8):  62%|██████▏   | 111988/179733 [00:17<00:10, 6257.03 examples/s]
Map (num_proc=8):  66%|██████▌   | 117939/179733 [00:18<00:09, 6234.79 examples/s]
Map (num_proc=8):  68%|██████▊   | 122571/179733 [00:19<00:08, 6546.10 examples/s]
Map (num_proc=8):  68%|██████▊   | 121594/179733 [00:19<00:08, 6480.58 examples/s]
Map (num_proc=8):  57%|█████▋    | 102524/179733 [00:16<00:12, 6393.23 examples/s]
Map (num_proc=8):  66%|██████▋   | 119358/179733 [00:19<00:09, 6296.70 examples/s]
Map (num_proc=8):  67%|██████▋   | 120546/179733 [00:19<00:09, 6405.38 examples/s]
Map (num_proc=8):  55%|█████▍    | 98048/179733 [00:15<00:13, 6282.77 examples/s]
Map (num_proc=8):  63%|██████▎   | 112635/179733 [00:18<00:10, 6308.44 examples/s]
Map (num_proc=8):  66%|██████▌   | 118580/179733 [00:18<00:09, 6275.72 examples/s]
Map (num_proc=8):  68%|██████▊   | 122268/179733 [00:19<00:08, 6551.22 examples/s]
Map (num_proc=8):  69%|██████▊   | 123258/179733 [00:19<00:08, 6594.53 examples/s]
Map (num_proc=8):  57%|█████▋    | 103168/179733 [00:16<00:12, 6344.55 examples/s]
Map (num_proc=8):  67%|██████▋   | 120020/179733 [00:19<00:09, 6344.46 examples/s]
Map (num_proc=8):  55%|█████▍    | 98700/179733 [00:15<00:12, 6331.27 examples/s]
Map (num_proc=8):  63%|██████▎   | 113282/179733 [00:18<00:10, 6330.93 examples/s]
Map (num_proc=8):  67%|██████▋   | 121220/179733 [00:19<00:09, 6140.87 examples/s]
Map (num_proc=8):  66%|██████▋   | 119242/179733 [00:18<00:09, 6349.00 examples/s]
Map (num_proc=8):  68%|██████▊   | 122928/179733 [00:19<00:08, 6516.06 examples/s]
Map (num_proc=8):  69%|██████▉   | 123941/179733 [00:19<00:08, 6462.05 examples/s]
Map (num_proc=8):  58%|█████▊    | 103845/179733 [00:16<00:11, 6439.85 examples/s]
Map (num_proc=8):  67%|██████▋   | 120688/179733 [00:19<00:09, 6416.76 examples/s]
Map (num_proc=8):  55%|█████▌    | 99346/179733 [00:15<00:12, 6364.33 examples/s]
Map (num_proc=8):  63%|██████▎   | 113924/179733 [00:18<00:10, 6352.32 examples/s]
Map (num_proc=8):  68%|██████▊   | 121901/179733 [00:19<00:09, 6314.60 examples/s]
Map (num_proc=8):  67%|██████▋   | 119906/179733 [00:18<00:09, 6390.29 examples/s]
Map (num_proc=8):  69%|██████▉   | 123605/179733 [00:19<00:08, 6559.80 examples/s]
Map (num_proc=8):  69%|██████▉   | 124617/179733 [00:19<00:08, 6522.72 examples/s]
Map (num_proc=8):  58%|█████▊    | 104494/179733 [00:16<00:11, 6433.54 examples/s]
Map (num_proc=8):  68%|██████▊   | 121372/179733 [00:19<00:09, 6445.64 examples/s]
Map (num_proc=8):  56%|█████▌    | 100007/179733 [00:16<00:12, 6407.43 examples/s]
Map (num_proc=8):  64%|██████▎   | 114571/179733 [00:18<00:10, 6370.33 examples/s]
Map (num_proc=8):  67%|██████▋   | 120579/179733 [00:19<00:09, 6479.43 examples/s]
Map (num_proc=8):  68%|██████▊   | 122547/179733 [00:19<00:09, 6116.71 examples/s]
Map (num_proc=8):  69%|██████▉   | 124299/179733 [00:19<00:08, 6597.34 examples/s]
Map (num_proc=8):  70%|██████▉   | 125310/179733 [00:19<00:08, 6431.81 examples/s]
Map (num_proc=8):  59%|█████▊    | 105172/179733 [00:16<00:11, 6359.36 examples/s]
Map (num_proc=8):  68%|██████▊   | 122052/179733 [00:19<00:08, 6521.48 examples/s]
Map (num_proc=8):  56%|█████▌    | 100678/179733 [00:16<00:12, 6472.68 examples/s]
Map (num_proc=8):  64%|██████▍   | 115240/179733 [00:18<00:10, 6444.49 examples/s]
Map (num_proc=8):  67%|██████▋   | 121237/179733 [00:19<00:09, 6473.82 examples/s]
Map (num_proc=8):  69%|██████▊   | 123178/179733 [00:19<00:09, 6154.08 examples/s]
Map (num_proc=8):  70%|██████▉   | 124991/179733 [00:19<00:08, 6670.62 examples/s]
Map (num_proc=8):  70%|███████   | 125991/179733 [00:19<00:08, 6529.39 examples/s]
Map (num_proc=8):  59%|█████▉    | 105816/179733 [00:17<00:11, 6348.43 examples/s]
Map (num_proc=8):  68%|██████▊   | 122735/179733 [00:19<00:08, 6576.81 examples/s]
Map (num_proc=8):  56%|█████▋    | 101360/179733 [00:16<00:11, 6543.00 examples/s]
Map (num_proc=8):  64%|██████▍   | 115892/179733 [00:18<00:09, 6461.07 examples/s]
Map (num_proc=8):  68%|██████▊   | 121917/179733 [00:19<00:08, 6528.31 examples/s]
Map (num_proc=8):  69%|██████▉   | 123839/179733 [00:19<00:08, 6259.16 examples/s]
Map (num_proc=8):  70%|██████▉   | 125687/179733 [00:19<00:08, 6491.04 examples/s]
Map (num_proc=8):  70%|███████   | 126656/179733 [00:19<00:08, 6555.23 examples/s]
Map (num_proc=8):  59%|█████▉    | 106466/179733 [00:17<00:11, 6373.58 examples/s]
Map (num_proc=8):  69%|██████▊   | 123396/179733 [00:19<00:08, 6567.80 examples/s]
Map (num_proc=8):  65%|██████▍   | 116546/179733 [00:18<00:09, 6463.61 examples/s]
Map (num_proc=8):  57%|█████▋    | 102052/179733 [00:16<00:12, 6442.15 examples/s]
Map (num_proc=8):  68%|██████▊   | 122596/179733 [00:19<00:08, 6587.81 examples/s]
Map (num_proc=8):  69%|██████▉   | 124476/179733 [00:19<00:08, 6272.33 examples/s]
Map (num_proc=8):  70%|███████   | 126343/179733 [00:19<00:08, 6492.68 examples/s]
Map (num_proc=8):  71%|███████   | 127320/179733 [00:19<00:07, 6558.61 examples/s]
Map (num_proc=8):  60%|█████▉    | 107126/179733 [00:17<00:11, 6386.19 examples/s]
Map (num_proc=8):  69%|██████▉   | 124062/179733 [00:19<00:08, 6539.21 examples/s]
Map (num_proc=8):  65%|██████▌   | 117202/179733 [00:18<00:09, 6471.74 examples/s]
Map (num_proc=8):  57%|█████▋    | 102719/179733 [00:16<00:11, 6476.59 examples/s]
Map (num_proc=8):  69%|██████▊   | 123264/179733 [00:19<00:08, 6591.14 examples/s]
Map (num_proc=8):  70%|██████▉   | 125111/179733 [00:19<00:08, 6248.37 examples/s]
Map (num_proc=8):  71%|███████   | 127011/179733 [00:19<00:08, 6517.56 examples/s]
Map (num_proc=8):  71%|███████   | 128016/179733 [00:20<00:07, 6533.96 examples/s]
Map (num_proc=8):  69%|██████▉   | 124728/179733 [00:19<00:08, 6549.72 examples/s]
Map (num_proc=8):  60%|█████▉    | 107794/179733 [00:17<00:11, 6264.67 examples/s]
Map (num_proc=8):  66%|██████▌   | 117870/179733 [00:18<00:09, 6519.09 examples/s]
Map (num_proc=8):  58%|█████▊    | 103384/179733 [00:16<00:11, 6517.29 examples/s]
Map (num_proc=8):  69%|██████▉   | 123926/179733 [00:19<00:08, 6577.10 examples/s]
Map (num_proc=8):  70%|██████▉   | 125750/179733 [00:20<00:08, 6269.43 examples/s]
Map (num_proc=8):  71%|███████   | 127673/179733 [00:20<00:08, 6466.83 examples/s]
Map (num_proc=8):  72%|███████▏  | 128707/179733 [00:20<00:07, 6524.50 examples/s]
Map (num_proc=8):  70%|██████▉   | 125406/179733 [00:19<00:08, 6585.07 examples/s]
Map (num_proc=8):  60%|██████    | 108454/179733 [00:17<00:11, 6320.07 examples/s]
Map (num_proc=8):  58%|█████▊    | 104067/179733 [00:16<00:11, 6520.42 examples/s]
Map (num_proc=8):  66%|██████▌   | 118531/179733 [00:18<00:09, 6376.82 examples/s]
Map (num_proc=8):  69%|██████▉   | 124617/179733 [00:19<00:08, 6651.93 examples/s]
Map (num_proc=8):  70%|███████   | 126392/179733 [00:20<00:08, 6264.16 examples/s]
Map (num_proc=8):  71%|███████▏  | 128366/179733 [00:20<00:07, 6565.66 examples/s]
Map (num_proc=8):  72%|███████▏  | 129379/179733 [00:20<00:07, 6552.09 examples/s]
Map (num_proc=8):  61%|██████    | 109093/179733 [00:17<00:11, 6312.08 examples/s]
Map (num_proc=8):  58%|█████▊    | 104722/179733 [00:16<00:11, 6510.94 examples/s]
Map (num_proc=8):  66%|██████▋   | 119178/179733 [00:19<00:09, 6383.62 examples/s]
Map (num_proc=8):  70%|███████   | 126102/179733 [00:20<00:08, 6288.80 examples/s]
Map (num_proc=8):  70%|██████▉   | 125318/179733 [00:19<00:08, 6688.56 examples/s]
Map (num_proc=8):  71%|███████   | 127036/179733 [00:20<00:08, 6280.47 examples/s]
Map (num_proc=8):  72%|███████▏  | 130058/179733 [00:20<00:07, 6601.18 examples/s]
Map (num_proc=8):  72%|███████▏  | 129057/179733 [00:20<00:07, 6354.52 examples/s]
Map (num_proc=8):  61%|██████    | 109726/179733 [00:17<00:11, 6281.51 examples/s]
Map (num_proc=8):  67%|██████▋   | 119837/179733 [00:19<00:09, 6422.32 examples/s]
Map (num_proc=8):  71%|███████   | 126756/179733 [00:20<00:08, 6336.96 examples/s]
Map (num_proc=8):  59%|█████▊    | 105415/179733 [00:16<00:11, 6591.67 examples/s]
Map (num_proc=8):  71%|███████   | 127674/179733 [00:20<00:08, 6286.32 examples/s]
Map (num_proc=8):  70%|███████   | 125991/179733 [00:19<00:08, 6618.27 examples/s]
Map (num_proc=8):  73%|███████▎  | 130760/179733 [00:20<00:07, 6645.28 examples/s]
Map (num_proc=8):  72%|███████▏  | 129713/179733 [00:20<00:07, 6392.29 examples/s]
Map (num_proc=8):  61%|██████▏   | 110359/179733 [00:17<00:11, 6293.66 examples/s]
Map (num_proc=8):  71%|███████   | 127406/179733 [00:20<00:08, 6371.91 examples/s]
Map (num_proc=8):  67%|██████▋   | 120507/179733 [00:19<00:09, 6487.61 examples/s]
Map (num_proc=8):  59%|█████▉    | 106083/179733 [00:16<00:11, 6596.40 examples/s]
Map (num_proc=8):  70%|███████   | 126669/179733 [00:20<00:07, 6658.27 examples/s]
Map (num_proc=8):  71%|███████▏  | 128324/179733 [00:20<00:08, 6325.81 examples/s]
Map (num_proc=8):  73%|███████▎  | 130376/179733 [00:20<00:07, 6441.21 examples/s]
Map (num_proc=8):  73%|███████▎  | 131434/179733 [00:20<00:07, 6454.70 examples/s]
Map (num_proc=8):  62%|██████▏   | 111022/179733 [00:17<00:10, 6354.01 examples/s]
Map (num_proc=8):  67%|██████▋   | 121162/179733 [00:19<00:09, 6486.84 examples/s]
Map (num_proc=8):  71%|███████   | 128055/179733 [00:20<00:08, 6361.47 examples/s]
Map (num_proc=8):  59%|█████▉    | 106759/179733 [00:17<00:11, 6580.87 examples/s]
Map (num_proc=8):  72%|███████▏  | 128969/179733 [00:20<00:08, 6345.27 examples/s]
Map (num_proc=8):  71%|███████   | 127381/179733 [00:20<00:07, 6686.90 examples/s]
Map (num_proc=8):  73%|███████▎  | 131045/179733 [00:20<00:07, 6492.31 examples/s]
Map (num_proc=8):  73%|███████▎  | 132098/179733 [00:20<00:07, 6500.17 examples/s]
Map (num_proc=8):  62%|██████▏   | 111681/179733 [00:17<00:10, 6392.77 examples/s]
Map (num_proc=8):  68%|██████▊   | 121811/179733 [00:19<00:08, 6471.57 examples/s]
Map (num_proc=8):  60%|█████▉    | 107424/179733 [00:17<00:11, 6470.88 examples/s]
Map (num_proc=8):  72%|███████▏  | 128732/179733 [00:20<00:08, 6160.34 examples/s]
Map (num_proc=8):  71%|███████   | 128053/179733 [00:20<00:07, 6688.07 examples/s]
Map (num_proc=8):  72%|███████▏  | 129641/179733 [00:20<00:07, 6328.42 examples/s]
Map (num_proc=8):  73%|███████▎  | 131701/179733 [00:20<00:07, 6498.34 examples/s]
Map (num_proc=8):  74%|███████▍  | 132779/179733 [00:20<00:07, 6559.63 examples/s]
Map (num_proc=8):  63%|██████▎   | 112351/179733 [00:18<00:10, 6458.24 examples/s]
Map (num_proc=8):  68%|██████▊   | 122470/179733 [00:19<00:08, 6490.91 examples/s]
Map (num_proc=8):  60%|██████    | 108079/179733 [00:17<00:11, 6482.34 examples/s]
Map (num_proc=8):  72%|███████▏  | 129370/179733 [00:20<00:08, 6209.51 examples/s]
Map (num_proc=8):  72%|███████▏  | 128736/179733 [00:20<00:07, 6691.03 examples/s]
Map (num_proc=8):  72%|███████▏  | 130306/179733 [00:20<00:07, 6391.64 examples/s]
Map (num_proc=8):  74%|███████▎  | 132387/179733 [00:20<00:07, 6477.88 examples/s]
Map (num_proc=8):  74%|███████▍  | 133437/179733 [00:20<00:07, 6531.45 examples/s]
Map (num_proc=8):  63%|██████▎   | 113040/179733 [00:18<00:10, 6438.69 examples/s]
Map (num_proc=8):  69%|██████▊   | 123160/179733 [00:19<00:08, 6506.70 examples/s]
Map (num_proc=8):  72%|███████▏  | 130042/179733 [00:20<00:07, 6337.74 examples/s]
Map (num_proc=8):  73%|███████▎  | 130985/179733 [00:20<00:07, 6469.21 examples/s]
Map (num_proc=8):  60%|██████    | 108736/179733 [00:17<00:11, 6061.13 examples/s]
Map (num_proc=8):  72%|███████▏  | 129427/179733 [00:20<00:07, 6501.24 examples/s]
Map (num_proc=8):  74%|███████▍  | 133051/179733 [00:20<00:07, 6478.28 examples/s]
Map (num_proc=8):  75%|███████▍  | 134123/179733 [00:20<00:07, 6394.65 examples/s]
Map (num_proc=8):  63%|██████▎   | 113696/179733 [00:18<00:10, 6262.68 examples/s]
Map (num_proc=8):  69%|██████▉   | 123829/179733 [00:19<00:08, 6529.02 examples/s]
Map (num_proc=8):  73%|███████▎  | 130704/179733 [00:20<00:07, 6396.13 examples/s]
Map (num_proc=8):  61%|██████    | 109382/179733 [00:17<00:11, 6153.68 examples/s]
Map (num_proc=8):  73%|███████▎  | 131657/179733 [00:20<00:07, 6364.63 examples/s]
Map (num_proc=8):  72%|███████▏  | 130097/179733 [00:20<00:07, 6530.55 examples/s]
Map (num_proc=8):  74%|███████▍  | 133704/179733 [00:21<00:07, 6474.65 examples/s]
Map (num_proc=8):  75%|███████▌  | 134801/179733 [00:21<00:06, 6481.23 examples/s]
Map (num_proc=8):  64%|██████▎   | 114352/179733 [00:18<00:10, 6303.61 examples/s]
Map (num_proc=8):  73%|███████▎  | 131349/179733 [00:20<00:07, 6388.24 examples/s]
Map (num_proc=8):  69%|██████▉   | 124494/179733 [00:19<00:08, 6271.74 examples/s]
Map (num_proc=8):  61%|██████    | 110034/179733 [00:17<00:11, 6229.17 examples/s]
Map (num_proc=8):  73%|███████▎  | 130765/179733 [00:20<00:07, 6556.69 examples/s]
Map (num_proc=8):  74%|███████▎  | 132330/179733 [00:21<00:07, 6414.94 examples/s]
Map (num_proc=8):  75%|███████▍  | 134359/179733 [00:21<00:07, 6481.77 examples/s]
Map (num_proc=8):  75%|███████▌  | 135458/179733 [00:21<00:06, 6449.11 examples/s]
Map (num_proc=8):  64%|██████▍   | 115013/179733 [00:18<00:10, 6363.57 examples/s]
Map (num_proc=8):  73%|███████▎  | 132003/179733 [00:20<00:07, 6407.71 examples/s]
Map (num_proc=8):  70%|██████▉   | 125166/179733 [00:19<00:08, 6379.87 examples/s]
Map (num_proc=8):  62%|██████▏   | 110684/179733 [00:17<00:10, 6282.73 examples/s]
Map (num_proc=8):  73%|███████▎  | 131434/179733 [00:20<00:07, 6576.11 examples/s]
Map (num_proc=8):  74%|███████▍  | 132985/179733 [00:21<00:07, 6443.43 examples/s]
Map (num_proc=8):  75%|███████▌  | 135021/179733 [00:21<00:06, 6491.02 examples/s]
Map (num_proc=8):  76%|███████▌  | 136116/179733 [00:21<00:06, 6476.69 examples/s]
Map (num_proc=8):  64%|██████▍   | 115669/179733 [00:18<00:10, 6399.06 examples/s]
Map (num_proc=8):  74%|███████▍  | 132667/179733 [00:21<00:07, 6457.98 examples/s]
Map (num_proc=8):  70%|███████   | 125824/179733 [00:20<00:08, 6402.46 examples/s]
Map (num_proc=8):  74%|███████▎  | 132112/179733 [00:20<00:07, 6616.92 examples/s]
Map (num_proc=8):  62%|██████▏   | 111357/179733 [00:17<00:10, 6375.60 examples/s]
Map (num_proc=8):  74%|███████▍  | 133668/179733 [00:21<00:07, 6378.69 examples/s]
Map (num_proc=8):  75%|███████▌  | 135690/179733 [00:21<00:06, 6517.10 examples/s]
Map (num_proc=8):  76%|███████▌  | 136787/179733 [00:21<00:06, 6529.60 examples/s]
Map (num_proc=8):  65%|██████▍   | 116314/179733 [00:18<00:09, 6393.85 examples/s]
Map (num_proc=8):  74%|███████▍  | 133322/179733 [00:21<00:07, 6476.99 examples/s]
Map (num_proc=8):  70%|███████   | 126505/179733 [00:20<00:08, 6427.12 examples/s]
Map (num_proc=8):  62%|██████▏   | 112003/179733 [00:17<00:10, 6386.70 examples/s]
Map (num_proc=8):  74%|███████▍  | 132804/179733 [00:20<00:07, 6525.51 examples/s]
Map (num_proc=8):  75%|███████▍  | 134328/179733 [00:21<00:07, 6425.11 examples/s]
Map (num_proc=8):  76%|███████▌  | 136375/179733 [00:21<00:06, 6393.69 examples/s]
Map (num_proc=8):  76%|███████▋  | 137461/179733 [00:21<00:06, 6549.60 examples/s]
Map (num_proc=8):  65%|██████▌   | 116982/179733 [00:18<00:09, 6311.40 examples/s]
Map (num_proc=8):  75%|███████▍  | 134015/179733 [00:21<00:07, 6453.69 examples/s]
Map (num_proc=8):  71%|███████   | 127174/179733 [00:20<00:08, 6336.68 examples/s]
Map (num_proc=8):  63%|██████▎   | 112650/179733 [00:18<00:10, 6403.18 examples/s]
Map (num_proc=8):  74%|███████▍  | 133479/179733 [00:21<00:07, 6560.48 examples/s]
Map (num_proc=8):  75%|███████▌  | 134996/179733 [00:21<00:07, 6244.57 examples/s]
Map (num_proc=8):  76%|███████▌  | 137032/179733 [00:21<00:06, 6432.80 examples/s]
Map (num_proc=8):  77%|███████▋  | 138138/179733 [00:21<00:06, 6507.92 examples/s]
Map (num_proc=8):  65%|██████▌   | 117645/179733 [00:18<00:09, 6209.20 examples/s]
Map (num_proc=8):  75%|███████▍  | 134676/179733 [00:21<00:06, 6467.39 examples/s]
Map (num_proc=8):  71%|███████   | 127842/179733 [00:20<00:08, 6433.59 examples/s]
Map (num_proc=8):  63%|██████▎   | 113331/179733 [00:18<00:10, 6431.82 examples/s]
Map (num_proc=8):  75%|███████▍  | 134136/179733 [00:21<00:06, 6521.16 examples/s]
Map (num_proc=8):  75%|███████▌  | 135634/179733 [00:21<00:07, 6252.01 examples/s]
Map (num_proc=8):  77%|███████▋  | 137700/179733 [00:21<00:06, 6478.93 examples/s]
Map (num_proc=8):  77%|███████▋  | 138826/179733 [00:21<00:06, 6440.64 examples/s]
Map (num_proc=8):  66%|██████▌   | 118294/179733 [00:18<00:09, 6244.52 examples/s]
Map (num_proc=8):  75%|███████▌  | 135325/179733 [00:21<00:06, 6443.42 examples/s]
Map (num_proc=8):  72%|███████▏  | 128512/179733 [00:20<00:07, 6488.28 examples/s]
Map (num_proc=8):  63%|██████▎   | 113987/179733 [00:18<00:10, 6459.11 examples/s]
Map (num_proc=8):  76%|███████▌  | 136292/179733 [00:21<00:06, 6310.63 examples/s]
Map (num_proc=8):  75%|███████▌  | 134823/179733 [00:21<00:07, 6241.93 examples/s]
Map (num_proc=8):  77%|███████▋  | 138379/179733 [00:21<00:06, 6541.45 examples/s]
Map (num_proc=8):  78%|███████▊  | 139485/179733 [00:21<00:06, 6462.97 examples/s]
Map (num_proc=8):  66%|██████▌   | 118938/179733 [00:19<00:09, 6252.44 examples/s]
Map (num_proc=8):  76%|███████▌  | 135979/179733 [00:21<00:06, 6453.39 examples/s]
Map (num_proc=8):  72%|███████▏  | 129168/179733 [00:20<00:07, 6489.31 examples/s]
Map (num_proc=8):  64%|██████▍   | 114636/179733 [00:18<00:10, 6438.41 examples/s]
Map (num_proc=8):  76%|███████▌  | 136959/179733 [00:21<00:06, 6412.43 examples/s]
Map (num_proc=8):  75%|███████▌  | 135496/179733 [00:21<00:06, 6363.41 examples/s]
Map (num_proc=8):  77%|███████▋  | 139068/179733 [00:21<00:06, 6602.01 examples/s]
Map (num_proc=8):  78%|███████▊  | 140167/179733 [00:21<00:06, 6539.70 examples/s]
Map (num_proc=8):  67%|██████▋   | 119574/179733 [00:19<00:09, 6269.70 examples/s]
Map (num_proc=8):  76%|███████▌  | 136638/179733 [00:21<00:06, 6463.98 examples/s]
Map (num_proc=8):  72%|███████▏  | 129829/179733 [00:20<00:07, 6470.73 examples/s]
Map (num_proc=8):  64%|██████▍   | 115308/179733 [00:18<00:10, 6341.07 examples/s]
Map (num_proc=8):  77%|███████▋  | 137608/179733 [00:21<00:06, 6397.72 examples/s]
Map (num_proc=8):  76%|███████▌  | 136160/179733 [00:21<00:06, 6403.74 examples/s]
Map (num_proc=8):  78%|███████▊  | 139740/179733 [00:21<00:06, 6630.40 examples/s]
Map (num_proc=8):  78%|███████▊  | 140833/179733 [00:22<00:05, 6571.34 examples/s]
Map (num_proc=8):  67%|██████▋   | 120229/179733 [00:19<00:09, 6300.49 examples/s]
Map (num_proc=8):  76%|███████▋  | 137289/179733 [00:21<00:06, 6461.38 examples/s]
Map (num_proc=8):  73%|███████▎  | 130486/179733 [00:20<00:07, 6460.82 examples/s]
Map (num_proc=8):  65%|██████▍   | 115970/179733 [00:18<00:09, 6397.91 examples/s]
Map (num_proc=8):  77%|███████▋  | 138271/179733 [00:22<00:06, 6444.83 examples/s]
Map (num_proc=8):  76%|███████▌  | 136805/179733 [00:21<00:06, 6394.74 examples/s]
Map (num_proc=8):  78%|███████▊  | 140410/179733 [00:22<00:05, 6618.95 examples/s]
Map (num_proc=8):  79%|███████▊  | 141532/179733 [00:22<00:05, 6666.91 examples/s]
Map (num_proc=8):  77%|███████▋  | 137949/179733 [00:21<00:06, 6496.52 examples/s]
Map (num_proc=8):  67%|██████▋   | 120871/179733 [00:19<00:09, 6307.78 examples/s]
Map (num_proc=8):  73%|███████▎  | 131158/179733 [00:20<00:07, 6464.98 examples/s]
Map (num_proc=8):  65%|██████▍   | 116647/179733 [00:18<00:09, 6479.84 examples/s]
Map (num_proc=8):  77%|███████▋  | 138924/179733 [00:22<00:06, 6449.79 examples/s]
Map (num_proc=8):  76%|███████▋  | 137482/179733 [00:21<00:06, 6451.22 examples/s]
Map (num_proc=8):  79%|███████▊  | 141095/179733 [00:22<00:05, 6657.70 examples/s]
Map (num_proc=8):  79%|███████▉  | 142234/179733 [00:22<00:05, 6703.22 examples/s]
Map (num_proc=8):  68%|██████▊   | 121533/179733 [00:19<00:09, 6383.39 examples/s]
Map (num_proc=8):  77%|███████▋  | 138617/179733 [00:21<00:06, 6525.19 examples/s]
Map (num_proc=8):  73%|███████▎  | 131834/179733 [00:20<00:07, 6423.03 examples/s]
Map (num_proc=8):  65%|██████▌   | 117317/179733 [00:18<00:09, 6530.80 examples/s]
Map (num_proc=8):  78%|███████▊  | 139583/179733 [00:22<00:06, 6469.19 examples/s]
Map (num_proc=8):  77%|███████▋  | 138143/179733 [00:21<00:06, 6464.93 examples/s]
Map (num_proc=8):  79%|███████▉  | 141764/179733 [00:22<00:05, 6628.87 examples/s]
Map (num_proc=8):  80%|███████▉  | 142942/179733 [00:22<00:05, 6669.96 examples/s]
Map (num_proc=8):  77%|███████▋  | 139278/179733 [00:22<00:06, 6540.27 examples/s]
Map (num_proc=8):  68%|██████▊   | 122196/179733 [00:19<00:08, 6425.06 examples/s]
Map (num_proc=8):  74%|███████▎  | 132479/179733 [00:21<00:07, 6418.15 examples/s]
Map (num_proc=8):  66%|██████▌   | 117980/179733 [00:18<00:09, 6544.64 examples/s]
Map (num_proc=8):  78%|███████▊  | 140251/179733 [00:22<00:06, 6508.26 examples/s]
Map (num_proc=8):  77%|███████▋  | 138802/179733 [00:21<00:06, 6493.78 examples/s]
Map (num_proc=8):  79%|███████▉  | 142439/179733 [00:22<00:05, 6651.86 examples/s]
Map (num_proc=8):  80%|███████▉  | 143642/179733 [00:22<00:05, 6642.48 examples/s]
Map (num_proc=8):  68%|██████▊   | 122871/179733 [00:19<00:08, 6514.85 examples/s]
Map (num_proc=8):  78%|███████▊  | 139938/179733 [00:22<00:06, 6525.76 examples/s]
Map (num_proc=8):  74%|███████▍  | 133143/179733 [00:21<00:07, 6448.67 examples/s]
Map (num_proc=8):  66%|██████▌   | 118643/179733 [00:18<00:09, 6548.30 examples/s]
Map (num_proc=8):  78%|███████▊  | 140905/179733 [00:22<00:05, 6493.84 examples/s]
Map (num_proc=8):  78%|███████▊  | 139461/179733 [00:21<00:06, 6502.70 examples/s]
Map (num_proc=8):  80%|███████▉  | 143135/179733 [00:22<00:05, 6693.53 examples/s]
Map (num_proc=8):  80%|████████  | 144333/179733 [00:22<00:05, 6618.14 examples/s]
Map (num_proc=8):  69%|██████▊   | 123529/179733 [00:19<00:08, 6514.81 examples/s]
Map (num_proc=8):  78%|███████▊  | 140616/179733 [00:22<00:05, 6575.88 examples/s]
Map (num_proc=8):  74%|███████▍  | 133823/179733 [00:21<00:07, 6526.38 examples/s]
Map (num_proc=8):  66%|██████▋   | 119333/179733 [00:19<00:09, 6519.01 examples/s]
Map (num_proc=8):  78%|███████▊  | 140132/179733 [00:22<00:06, 6546.58 examples/s]
Map (num_proc=8):  79%|███████▉  | 141583/179733 [00:22<00:06, 6201.07 examples/s]
Map (num_proc=8):  80%|████████  | 143827/179733 [00:22<00:05, 6349.65 examples/s]
Map (num_proc=8):  81%|████████  | 145030/179733 [00:22<00:05, 6490.89 examples/s]
Map (num_proc=8):  79%|███████▊  | 141279/179733 [00:22<00:05, 6566.83 examples/s]
Map (num_proc=8):  69%|██████▉   | 124215/179733 [00:19<00:08, 6575.70 examples/s]
Map (num_proc=8):  75%|███████▍  | 134478/179733 [00:21<00:06, 6511.83 examples/s]
Map (num_proc=8):  67%|██████▋   | 120017/179733 [00:19<00:09, 6478.99 examples/s]
Map (num_proc=8):  78%|███████▊  | 140803/179733 [00:22<00:05, 6557.22 examples/s]
Map (num_proc=8):  79%|███████▉  | 142222/179733 [00:22<00:06, 6241.23 examples/s]
Map (num_proc=8):  80%|████████  | 144484/179733 [00:22<00:05, 6367.86 examples/s]
Map (num_proc=8):  81%|████████  | 145681/179733 [00:22<00:05, 6461.07 examples/s]
Map (num_proc=8):  79%|███████▉  | 141956/179733 [00:22<00:05, 6597.43 examples/s]
Map (num_proc=8):  69%|██████▉   | 124888/179733 [00:20<00:08, 6548.55 examples/s]
Map (num_proc=8):  75%|███████▌  | 135167/179733 [00:21<00:06, 6450.66 examples/s]
Map (num_proc=8):  67%|██████▋   | 120681/179733 [00:19<00:09, 6475.61 examples/s]
Map (num_proc=8):  79%|███████▊  | 141470/179733 [00:22<00:05, 6566.31 examples/s]
Map (num_proc=8):  79%|███████▉  | 142866/179733 [00:22<00:05, 6273.12 examples/s]
Map (num_proc=8):  81%|████████  | 145124/179733 [00:22<00:05, 6366.82 examples/s]
Map (num_proc=8):  81%|████████▏ | 146340/179733 [00:22<00:05, 6465.07 examples/s]
Map (num_proc=8):  79%|███████▉  | 142616/179733 [00:22<00:05, 6560.45 examples/s]
Map (num_proc=8):  70%|██████▉   | 125547/179733 [00:20<00:08, 6542.60 examples/s]
Map (num_proc=8):  76%|███████▌  | 135826/179733 [00:21<00:06, 6483.01 examples/s]
Map (num_proc=8):  68%|██████▊   | 121358/179733 [00:19<00:08, 6536.87 examples/s]
Map (num_proc=8):  79%|███████▉  | 142158/179733 [00:22<00:05, 6623.35 examples/s]
Map (num_proc=8):  80%|███████▉  | 143515/179733 [00:22<00:05, 6301.62 examples/s]
Map (num_proc=8):  81%|████████  | 145794/179733 [00:22<00:05, 6424.03 examples/s]
Map (num_proc=8):  80%|███████▉  | 143305/179733 [00:22<00:05, 6516.11 examples/s]
Map (num_proc=8):  82%|████████▏ | 147014/179733 [00:22<00:05, 6299.49 examples/s]
Map (num_proc=8):  70%|███████   | 126228/179733 [00:20<00:08, 6366.82 examples/s]
Map (num_proc=8):  76%|███████▌  | 136487/179733 [00:21<00:06, 6444.53 examples/s]
Map (num_proc=8):  68%|██████▊   | 122020/179733 [00:19<00:08, 6521.96 examples/s]
Map (num_proc=8):  79%|███████▉  | 142854/179733 [00:22<00:05, 6640.54 examples/s]
Map (num_proc=8):  80%|████████  | 144181/179733 [00:22<00:05, 6374.63 examples/s]
Map (num_proc=8):  81%|████████▏ | 146452/179733 [00:22<00:05, 6401.99 examples/s]
Map (num_proc=8):  82%|████████▏ | 147668/179733 [00:23<00:05, 6356.10 examples/s]
Map (num_proc=8):  80%|████████  | 143958/179733 [00:22<00:05, 6435.70 examples/s]
Map (num_proc=8):  71%|███████   | 126880/179733 [00:20<00:08, 6388.95 examples/s]
Map (num_proc=8):  76%|███████▋  | 137134/179733 [00:21<00:06, 6445.54 examples/s]
Map (num_proc=8):  68%|██████▊   | 122695/179733 [00:19<00:08, 6576.25 examples/s]
Map (num_proc=8):  80%|███████▉  | 143540/179733 [00:22<00:05, 6673.68 examples/s]
Map (num_proc=8):  81%|████████  | 144844/179733 [00:23<00:05, 6425.31 examples/s]
Map (num_proc=8):  82%|████████▏ | 147135/179733 [00:23<00:05, 6277.81 examples/s]
Map (num_proc=8):  83%|████████▎ | 148336/179733 [00:23<00:04, 6443.00 examples/s]
Map (num_proc=8):  80%|████████  | 144606/179733 [00:22<00:05, 6421.14 examples/s]
Map (num_proc=8):  71%|███████   | 127538/179733 [00:20<00:08, 6424.45 examples/s]
Map (num_proc=8):  69%|██████▊   | 123374/179733 [00:19<00:08, 6621.76 examples/s]
Map (num_proc=8):  77%|███████▋  | 137817/179733 [00:21<00:06, 6380.74 examples/s]
Map (num_proc=8):  80%|████████  | 144231/179733 [00:22<00:05, 6711.58 examples/s]
Map (num_proc=8):  81%|████████  | 145510/179733 [00:23<00:05, 6473.86 examples/s]
Map (num_proc=8):  82%|████████▏ | 147772/179733 [00:23<00:05, 6287.99 examples/s]
Map (num_proc=8):  83%|████████▎ | 148998/179733 [00:23<00:04, 6442.77 examples/s]
Map (num_proc=8):  81%|████████  | 145288/179733 [00:23<00:05, 6460.51 examples/s]
Map (num_proc=8):  71%|███████▏  | 128214/179733 [00:20<00:07, 6492.16 examples/s]
Map (num_proc=8):  69%|██████▉   | 124042/179733 [00:19<00:08, 6612.04 examples/s]
Map (num_proc=8):  77%|███████▋  | 138462/179733 [00:22<00:06, 6384.83 examples/s]
Map (num_proc=8):  81%|████████  | 144925/179733 [00:22<00:05, 6661.64 examples/s]
Map (num_proc=8):  81%|████████▏ | 146172/179733 [00:23<00:05, 6503.15 examples/s]
Map (num_proc=8):  83%|████████▎ | 148432/179733 [00:23<00:04, 6361.46 examples/s]
Map (num_proc=8):  83%|████████▎ | 149675/179733 [00:23<00:04, 6509.19 examples/s]
Map (num_proc=8):  81%|████████  | 145938/179733 [00:23<00:05, 6452.57 examples/s]
Map (num_proc=8):  72%|███████▏  | 128865/179733 [00:20<00:07, 6479.70 examples/s]
Map (num_proc=8):  77%|██████��▋  | 139134/179733 [00:22<00:06, 6462.98 examples/s]
Map (num_proc=8):  69%|██████▉   | 124737/179733 [00:19<00:08, 6640.61 examples/s]
Map (num_proc=8):  81%|████████  | 145602/179733 [00:22<00:05, 6664.88 examples/s]
Map (num_proc=8):  82%|████████▏ | 146828/179733 [00:23<00:05, 6502.41 examples/s]
Map (num_proc=8):  83%|████████▎ | 149089/179733 [00:23<00:04, 6392.16 examples/s]
Map (num_proc=8):  84%|████████▎ | 150344/179733 [00:23<00:04, 6548.04 examples/s]
Map (num_proc=8):  82%|████████▏ | 146606/179733 [00:23<00:05, 6483.48 examples/s]
Map (num_proc=8):  72%|███████▏  | 129543/179733 [00:20<00:07, 6533.50 examples/s]
Map (num_proc=8):  78%|███████▊  | 139807/179733 [00:22<00:06, 6518.80 examples/s]
Map (num_proc=8):  70%|██████▉   | 125403/179733 [00:19<00:08, 6623.51 examples/s]
Map (num_proc=8):  82%|████████▏ | 147517/179733 [00:23<00:04, 6548.83 examples/s]
Map (num_proc=8):  81%|████████▏ | 146305/179733 [00:23<00:05, 6605.55 examples/s]
Map (num_proc=8):  83%|████████▎ | 149741/179733 [00:23<00:04, 6411.62 examples/s]
Map (num_proc=8):  84%|████████▍ | 151000/179733 [00:23<00:04, 6497.76 examples/s]
Map (num_proc=8):  82%|████████▏ | 147265/179733 [00:23<00:05, 6459.95 examples/s]
Map (num_proc=8):  72%|███████▏  | 130199/179733 [00:20<00:07, 6485.33 examples/s]
Map (num_proc=8):  78%|███████▊  | 140474/179733 [00:22<00:06, 6527.78 examples/s]
Map (num_proc=8):  70%|███████   | 126070/179733 [00:20<00:08, 6621.56 examples/s]
Map (num_proc=8):  82%|████████▏ | 148209/179733 [00:23<00:04, 6591.66 examples/s]
Map (num_proc=8):  82%|████████▏ | 146987/179733 [00:23<00:05, 6417.89 examples/s]
Map (num_proc=8):  84%|████████▎ | 150401/179733 [00:23<00:04, 6458.09 examples/s]
Map (num_proc=8):  84%|████████▍ | 151655/179733 [00:23<00:04, 6466.37 examples/s]
Map (num_proc=8):  73%|███████▎  | 130871/179733 [00:20<00:07, 6532.63 examples/s]
Map (num_proc=8):  82%|████████▏ | 147951/179733 [00:23<00:05, 6240.04 examples/s]
Map (num_proc=8):  79%|███████▊  | 141141/179733 [00:22<00:05, 6542.98 examples/s]
Map (num_proc=8):  71%|███████   | 126748/179733 [00:20<00:07, 6631.71 examples/s]
Map (num_proc=8):  83%|████████▎ | 148881/179733 [00:23<00:04, 6618.05 examples/s]
Map (num_proc=8):  82%|████████▏ | 147632/179733 [00:23<00:05, 6397.08 examples/s]
Map (num_proc=8):  84%|████████▍ | 151058/179733 [00:23<00:04, 6471.47 examples/s]
Map (num_proc=8):  85%|████████▍ | 152360/179733 [00:23<00:04, 6614.79 examples/s]
Map (num_proc=8):  73%|███████▎  | 131558/179733 [00:21<00:07, 6367.87 examples/s]
Map (num_proc=8):  83%|████████▎ | 148591/179733 [00:23<00:04, 6262.32 examples/s]
Map (num_proc=8):  79%|███████▉  | 141809/179733 [00:22<00:05, 6561.18 examples/s]
Map (num_proc=8):  71%|███████   | 127448/179733 [00:20<00:07, 6631.12 examples/s]
Map (num_proc=8):  83%|████████▎ | 149558/179733 [00:23<00:04, 6487.68 examples/s]
Map (num_proc=8):  84%|████████▍ | 151707/179733 [00:23<00:04, 6469.33 examples/s]
Map (num_proc=8):  83%|████████▎ | 148316/179733 [00:23<00:05, 6069.47 examples/s]
Map (num_proc=8):  85%|████████▌ | 153035/179733 [00:23<00:04, 6606.56 examples/s]
Map (num_proc=8):  83%|████████▎ | 149251/179733 [00:23<00:04, 6336.25 examples/s]
Map (num_proc=8):  79%|███████▉  | 142502/179733 [00:22<00:05, 6635.05 examples/s]
Map (num_proc=8):  71%|███████▏  | 128140/179733 [00:20<00:07, 6671.02 examples/s]
Map (num_proc=8):  74%|███████▎  | 132236/179733 [00:21<00:07, 6153.17 examples/s]
Map (num_proc=8):  84%|████████▎ | 150237/179733 [00:23<00:04, 6532.77 examples/s]
Map (num_proc=8):  85%|████████▍ | 152359/179733 [00:23<00:04, 6464.80 examples/s]
Map (num_proc=8):  83%|████████▎ | 148976/179733 [00:23<00:04, 6195.98 examples/s]
Map (num_proc=8):  86%|████████▌ | 153720/179733 [00:23<00:03, 6517.84 examples/s]
Map (num_proc=8):  83%|████████▎ | 149926/179733 [00:23<00:04, 6427.00 examples/s]
Map (num_proc=8):  80%|███████▉  | 143174/179733 [00:22<00:05, 6426.20 examples/s]
Map (num_proc=8):  74%|███████▍  | 132885/179733 [00:21<00:07, 6221.16 examples/s]
Map (num_proc=8):  72%|███████▏  | 128841/179733 [00:20<00:07, 6377.14 examples/s]
Map (num_proc=8):  84%|████████▍ | 150894/179733 [00:23<00:04, 6538.21 examples/s]
Map (num_proc=8):  85%|████████▌ | 153027/179733 [00:23<00:04, 6508.39 examples/s]
Map (num_proc=8):  83%|████████▎ | 149622/179733 [00:23<00:04, 6250.16 examples/s]
Map (num_proc=8):  86%|████████▌ | 154380/179733 [00:24<00:03, 6516.07 examples/s]
Map (num_proc=8):  84%|████████▍ | 150572/179733 [00:23<00:04, 6411.93 examples/s]
Map (num_proc=8):  74%|███████▍  | 133541/179733 [00:21<00:07, 6292.08 examples/s]
Map (num_proc=8):  80%|████████  | 143848/179733 [00:22<00:05, 6359.95 examples/s]
Map (num_proc=8):  72%|███████▏  | 129526/179733 [00:20<00:07, 6492.70 examples/s]
Map (num_proc=8):  84%|████████▍ | 151590/179733 [00:24<00:04, 6412.67 examples/s]
Map (num_proc=8):  86%|████████▌ | 153695/179733 [00:24<00:03, 6537.24 examples/s]
Map (num_proc=8):  84%|████████▎ | 150292/179733 [00:23<00:04, 6348.91 examples/s]
Map (num_proc=8):  86%|████████▋ | 155069/179733 [00:24<00:03, 6577.74 examples/s]
Map (num_proc=8):  84%|████████▍ | 151251/179733 [00:23<00:04, 6508.72 examples/s]
Map (num_proc=8):  75%|███████▍  | 134186/179733 [00:21<00:07, 6309.26 examples/s]
Map (num_proc=8):  80%|████████  | 144499/179733 [00:22<00:05, 6382.72 examples/s]
Map (num_proc=8):  72%|███████▏  | 130181/179733 [00:20<00:07, 6482.59 examples/s]
Map (num_proc=8):  85%|████████▍ | 152246/179733 [00:24<00:04, 6315.07 examples/s]
Map (num_proc=8):  86%|████████▌ | 154379/179733 [00:24<00:03, 6523.22 examples/s]
Map (num_proc=8):  84%|████████▍ | 150972/179733 [00:23<00:04, 6465.13 examples/s]
Map (num_proc=8):  87%|████████▋ | 155750/179733 [00:24<00:03, 6612.75 examples/s]
Map (num_proc=8):  85%|████████▍ | 151920/179733 [00:24<00:04, 6539.84 examples/s]
Map (num_proc=8):  75%|███████▌  | 134833/179733 [00:21<00:07, 6327.14 examples/s]
Map (num_proc=8):  81%|████████  | 145154/179733 [00:23<00:05, 6296.02 examples/s]
Map (num_proc=8):  73%|███████▎  | 130861/179733 [00:20<00:07, 6326.46 examples/s]
Map (num_proc=8):  85%|████████▌ | 152902/179733 [00:24<00:04, 6365.67 examples/s]
Map (num_proc=8):  84%|████████▍ | 151627/179733 [00:23<00:04, 6470.50 examples/s]
Map (num_proc=8):  86%|████████▋ | 155071/179733 [00:24<00:03, 6498.52 examples/s]
Map (num_proc=8):  87%|████████▋ | 156418/179733 [00:24<00:03, 6605.56 examples/s]
Map (num_proc=8):  85%|████████▍ | 152584/179733 [00:24<00:04, 6553.92 examples/s]
Map (num_proc=8):  75%|███████▌  | 135501/179733 [00:21<00:06, 6418.29 examples/s]
Map (num_proc=8):  81%|████████  | 145804/179733 [00:23<00:05, 6335.69 examples/s]
Map (num_proc=8):  73%|███████▎  | 131524/179733 [00:20<00:07, 6386.52 examples/s]
Map (num_proc=8):  85%|████████▍ | 152321/179733 [00:23<00:04, 6506.68 examples/s]
Map (num_proc=8):  87%|████████▋ | 155739/179733 [00:24<00:03, 6514.02 examples/s]
Map (num_proc=8):  85%|████████▌ | 153562/179733 [00:24<00:04, 6042.53 examples/s]
Map (num_proc=8):  87%|████████▋ | 157088/179733 [00:24<00:03, 6623.94 examples/s]
Map (num_proc=8):  85%|████████▌ | 153250/179733 [00:24<00:04, 6560.06 examples/s]
Map (num_proc=8):  76%|███████▌  | 136167/179733 [00:21<00:06, 6474.91 examples/s]
Map (num_proc=8):  81%|████████▏ | 146481/179733 [00:23<00:05, 6367.87 examples/s]
Map (num_proc=8):  74%|███████▎  | 132172/179733 [00:21<00:07, 6408.70 examples/s]
Map (num_proc=8):  85%|████████▌ | 153003/179733 [00:24<00:04, 6576.40 examples/s]
Map (num_proc=8):  86%|████████▌ | 154213/179733 [00:24<00:04, 6153.17 examples/s]
Map (num_proc=8):  87%|████████▋ | 156434/179733 [00:24<00:03, 6566.41 examples/s]
Map (num_proc=8):  88%|████████▊ | 157789/179733 [00:24<00:03, 6678.17 examples/s]
Map (num_proc=8):  86%|████████▌ | 153911/179733 [00:24<00:03, 6557.51 examples/s]
Map (num_proc=8):  76%|███████▌  | 136839/179733 [00:21<00:06, 6440.50 examples/s]
Map (num_proc=8):  82%|████████▏ | 147127/179733 [00:23<00:05, 6344.75 examples/s]
Map (num_proc=8):  74%|███████▍  | 132838/179733 [00:21<00:07, 6448.01 examples/s]
Map (num_proc=8):  86%|████████▌ | 153700/179733 [00:24<00:03, 6629.38 examples/s]
Map (num_proc=8):  86%|████████▌ | 154861/179733 [00:24<00:03, 6223.78 examples/s]
Map (num_proc=8):  87%|████████▋ | 157123/179733 [00:24<00:03, 6605.80 examples/s]
Map (num_proc=8):  86%|████████▌ | 154575/179733 [00:24<00:03, 6544.53 examples/s]
Map (num_proc=8):  88%|████████▊ | 158493/179733 [00:24<00:03, 6561.90 examples/s]
Map (num_proc=8):  77%|███████▋  | 137498/179733 [00:21<00:06, 6445.09 examples/s]
Map (num_proc=8):  82%|████████▏ | 147767/179733 [00:23<00:05, 6342.40 examples/s]
Map (num_proc=8):  74%|███████▍  | 133488/179733 [00:21<00:07, 6444.06 examples/s]
Map (num_proc=8):  86%|████████▌ | 154384/179733 [00:24<00:03, 6655.43 examples/s]
Map (num_proc=8):  87%|████████▋ | 155526/179733 [00:24<00:03, 6310.39 examples/s]
Map (num_proc=8):  88%|████████▊ | 157794/179733 [00:24<00:03, 6624.58 examples/s]
Map (num_proc=8):  86%|████████▋ | 155241/179733 [00:24<00:03, 6532.48 examples/s]
Map (num_proc=8):  89%|████████▊ | 159154/179733 [00:24<00:03, 6505.97 examples/s]
Map (num_proc=8):  77%|███████▋  | 138157/179733 [00:22<00:06, 6447.67 examples/s]
Map (num_proc=8):  83%|████████▎ | 148415/179733 [00:23<00:04, 6350.87 examples/s]
Map (num_proc=8):  75%|███████▍  | 134137/179733 [00:21<00:07, 6443.06 examples/s]
Map (num_proc=8):  86%|████████▋ | 155056/179733 [00:24<00:03, 6635.89 examples/s]
Map (num_proc=8):  87%|████████▋ | 156190/179733 [00:24<00:03, 6388.08 examples/s]
Map (num_proc=8):  88%|████████▊ | 158490/179733 [00:24<00:03, 6623.53 examples/s]
Map (num_proc=8):  87%|████████▋ | 155896/179733 [00:24<00:03, 6528.86 examples/s]
Map (num_proc=8):  89%|████████▉ | 159837/179733 [00:24<00:03, 6330.26 examples/s]
Map (num_proc=8):  77%|███████▋  | 138830/179733 [00:22<00:06, 6457.77 examples/s]
Map (num_proc=8):  83%|████████▎ | 149094/179733 [00:23<00:04, 6441.75 examples/s]
Map (num_proc=8):  75%|███████▌  | 134836/179733 [00:21<00:06, 6577.96 examples/s]
Map (num_proc=8):  87%|████████▋ | 155765/179733 [00:24<00:03, 6732.06 examples/s]
Map (num_proc=8):  87%|████████▋ | 156833/179733 [00:24<00:03, 6383.81 examples/s]
Map (num_proc=8):  89%|████████▊ | 159187/179733 [00:24<00:03, 6231.76 examples/s]
Map (num_proc=8):  87%|████████▋ | 156579/179733 [00:24<00:03, 6415.46 examples/s]
Map (num_proc=8):  89%|████████▉ | 160507/179733 [00:25<00:02, 6423.72 examples/s]
Map (num_proc=8):  78%|███████▊  | 139481/179733 [00:22<00:06, 6451.76 examples/s]
Map (num_proc=8):  83%|████████▎ | 149759/179733 [00:23<00:04, 6491.98 examples/s]
Map (num_proc=8):  75%|███████▌  | 135511/179733 [00:21<00:06, 6589.12 examples/s]
Map (num_proc=8):  87%|████████▋ | 156446/179733 [00:24<00:03, 6737.33 examples/s]
Map (num_proc=8):  88%|████████▊ | 157473/179733 [00:25<00:03, 6373.34 examples/s]
Map (num_proc=8):  89%|████████▉ | 159844/179733 [00:25<00:03, 6199.07 examples/s]
Map (num_proc=8):  87%|████████▋ | 157244/179733 [00:24<00:03, 6476.82 examples/s]
Map (num_proc=8):  90%|████████▉ | 161186/179733 [00:25<00:02, 6509.52 examples/s]
Map (num_proc=8):  78%|███████▊  | 140140/179733 [00:22<00:06, 6483.19 examples/s]
Map (num_proc=8):  76%|███████▌  | 136175/179733 [00:21<00:06, 6578.15 examples/s]
Map (num_proc=8):  84%|████████▎ | 150451/179733 [00:23<00:04, 6429.67 examples/s]
Map (num_proc=8):  87%|████████▋ | 157145/179733 [00:24<00:03, 6743.56 examples/s]
Map (num_proc=8):  88%|████████▊ | 158134/179733 [00:25<00:03, 6314.64 examples/s]
Map (num_proc=8):  89%|████████▉ | 160468/179733 [00:25<00:03, 6170.96 examples/s]
Map (num_proc=8):  88%|████████▊ | 157894/179733 [00:24<00:03, 6454.49 examples/s]
Map (num_proc=8):  78%|███████▊  | 140806/179733 [00:22<00:05, 6520.50 examples/s]
Map (num_proc=8):  90%|█████████ | 161858/179733 [00:25<00:02, 6434.40 examples/s]
Map (num_proc=8):  84%|████████▍ | 151110/179733 [00:23<00:04, 6438.18 examples/s]
Map (num_proc=8):  76%|███████▌  | 136849/179733 [00:21<00:06, 6557.38 examples/s]
Map (num_proc=8):  88%|████████▊ | 157820/179733 [00:24<00:03, 6735.23 examples/s]
Map (num_proc=8):  88%|████████▊ | 158794/179733 [00:25<00:03, 6376.89 examples/s]
Map (num_proc=8):  90%|████████▉ | 161137/179733 [00:25<00:02, 6293.37 examples/s]
Map (num_proc=8):  88%|████████▊ | 158544/179733 [00:25<00:03, 6444.51 examples/s]
Map (num_proc=8):  79%|███████▊  | 141484/179733 [00:22<00:05, 6564.82 examples/s]
Map (num_proc=8):  90%|█████████ | 162545/179733 [00:25<00:02, 6530.04 examples/s]
Map (num_proc=8):  84%|████████▍ | 151771/179733 [00:24<00:04, 6460.81 examples/s]
Map (num_proc=8):  77%|███████▋  | 137533/179733 [00:21<00:06, 6502.93 examples/s]
Map (num_proc=8):  89%|████████▊ | 159437/179733 [00:25<00:03, 6366.39 examples/s]
Map (num_proc=8):  88%|████████▊ | 158510/179733 [00:24<00:03, 6606.93 examples/s]
Map (num_proc=8):  89%|████████▊ | 159204/179733 [00:25<00:03, 6459.68 examples/s]
Map (num_proc=8):  90%|█████████ | 161810/179733 [00:25<00:02, 6355.08 examples/s]
Map (num_proc=8):  79%|███████▉  | 142148/179733 [00:22<00:05, 6550.28 examples/s]
Map (num_proc=8):  91%|█████████ | 163200/179733 [00:25<00:02, 6503.89 examples/s]
Map (num_proc=8):  85%|████████▍ | 152419/179733 [00:24<00:04, 6447.12 examples/s]
Map (num_proc=8):  77%|███████▋  | 138184/179733 [00:21<00:06, 6489.94 examples/s]
Map (num_proc=8):  89%|████████▉ | 160080/179733 [00:25<00:03, 6348.45 examples/s]
Map (num_proc=8):  89%|████████▊ | 159189/179733 [00:24<00:03, 6584.11 examples/s]
Map (num_proc=8):  89%|████████▉ | 159854/179733 [00:25<00:03, 6458.66 examples/s]
Map (num_proc=8):  90%|█████████ | 162477/179733 [00:25<00:02, 6412.99 examples/s]
Map (num_proc=8):  79%|███████▉  | 142805/179733 [00:22<00:05, 6528.98 examples/s]
Map (num_proc=8):  91%|█████████ | 163870/179733 [00:25<00:02, 6237.96 examples/s]
Map (num_proc=8):  85%|████████▌ | 153077/179733 [00:24<00:04, 6465.73 examples/s]
Map (num_proc=8):  77%|███████▋  | 138860/179733 [00:22<00:06, 6530.25 examples/s]
Map (num_proc=8):  89%|████████▉ | 160719/179733 [00:25<00:02, 6339.09 examples/s]
Map (num_proc=8):  89%|████████▉ | 159862/179733 [00:25<00:03, 6563.29 examples/s]
Map (num_proc=8):  91%|█████████ | 163135/179733 [00:25<00:02, 6445.32 examples/s]
Map (num_proc=8):  89%|████████▉ | 160532/179733 [00:25<00:02, 6403.15 examples/s]
Map (num_proc=8):  80%|███████▉  | 143468/179733 [00:22<00:05, 6544.48 examples/s]
Map (num_proc=8):  92%|█████████▏| 164513/179733 [00:25<00:02, 6254.83 examples/s]
Map (num_proc=8):  86%|████████▌ | 153750/179733 [00:24<00:03, 6506.04 examples/s]
Map (num_proc=8):  78%|███████▊  | 139519/179733 [00:22<00:06, 6542.59 examples/s]
Map (num_proc=8):  90%|████████▉ | 161379/179733 [00:25<00:02, 6369.15 examples/s]
Map (num_proc=8):  89%|████████▉ | 160534/179733 [00:25<00:02, 6590.01 examples/s]
Map (num_proc=8):  91%|█████████ | 163786/179733 [00:25<00:02, 6437.39 examples/s]
Map (num_proc=8):  90%|████████▉ | 161192/179733 [00:25<00:02, 6425.65 examples/s]
Map (num_proc=8):  80%|████████  | 144133/179733 [00:23<00:05, 6462.58 examples/s]
Map (num_proc=8):  92%|█████████▏| 165203/179733 [00:25<00:02, 6428.02 examples/s]
Map (num_proc=8):  86%|████████▌ | 154421/179733 [00:24<00:03, 6531.26 examples/s]
Map (num_proc=8):  78%|███████▊  | 140187/179733 [00:22<00:06, 6565.30 examples/s]
Map (num_proc=8):  90%|█████████ | 162042/179733 [00:25<00:02, 6424.82 examples/s]
Map (num_proc=8):  91%|█████████▏| 164445/179733 [00:25<00:02, 6444.90 examples/s]
Map (num_proc=8):  90%|████████▉ | 161231/179733 [00:25<00:03, 6089.90 examples/s]
Map (num_proc=8):  90%|█████████ | 161840/179733 [00:25<00:02, 6420.94 examples/s]
Map (num_proc=8):  81%|████████  | 144789/179733 [00:23<00:05, 6379.27 examples/s]
Map (num_proc=8):  92%|█████████▏| 165873/179733 [00:25<00:02, 6466.07 examples/s]
Map (num_proc=8):  86%|████████▋ | 155082/179733 [00:24<00:03, 6526.08 examples/s]
Map (num_proc=8):  78%|███████▊  | 140881/179733 [00:22<00:05, 6648.05 examples/s]
Map (num_proc=8):  91%|█████████ | 162716/179733 [00:25<00:02, 6487.91 examples/s]
Map (num_proc=8):  92%|█████████▏| 165105/179733 [00:25<00:02, 6456.25 examples/s]
Map (num_proc=8):  90%|█████████ | 161877/179733 [00:25<00:02, 6167.54 examples/s]
Map (num_proc=8):  90%|█████████ | 162486/179733 [00:25<00:02, 6413.34 examples/s]
Map (num_proc=8):  81%|████████  | 145447/179733 [00:23<00:05, 6417.55 examples/s]
Map (num_proc=8):  93%|█████████▎| 166548/179733 [00:25<00:02, 6538.68 examples/s]
Map (num_proc=8):  79%|███████▉  | 141552/179733 [00:22<00:05, 6639.97 examples/s]
Map (num_proc=8):  87%|████████▋ | 155755/179733 [00:24<00:03, 6457.82 examples/s]
Map (num_proc=8):  91%|█████████ | 163375/179733 [00:25<00:02, 6488.56 examples/s]
Map (num_proc=8):  92%|█████████▏| 165774/179733 [00:25<00:02, 6499.20 examples/s]
Map (num_proc=8):  91%|█████████ | 163131/179733 [00:25<00:02, 6421.61 examples/s]
Map (num_proc=8):  90%|█████████ | 162534/179733 [00:25<00:02, 6250.13 examples/s]
Map (num_proc=8):  81%|████████▏ | 146127/179733 [00:23<00:05, 6412.46 examples/s]
Map (num_proc=8):  93%|█████████▎| 167233/179733 [00:26<00:01, 6564.27 examples/s]
Map (num_proc=8):  87%|████████▋ | 156409/179733 [00:24<00:03, 6477.31 examples/s]
Map (num_proc=8):  79%|███████▉  | 142221/179733 [00:22<00:05, 6625.05 examples/s]
Map (num_proc=8):  91%|█████████▏| 164039/179733 [00:26<00:02, 6460.07 examples/s]
Map (num_proc=8):  93%|█████████▎| 166438/179733 [00:26<00:02, 6522.84 examples/s]
Map (num_proc=8):  91%|█████████ | 163799/179733 [00:25<00:02, 6460.54 examples/s]
Map (num_proc=8):  91%|█████████ | 163203/179733 [00:25<00:02, 6355.84 examples/s]
Map (num_proc=8):  82%|████████▏ | 146771/179733 [00:23<00:05, 6387.23 examples/s]
Map (num_proc=8):  93%|█████████▎| 167927/179733 [00:26<00:01, 6599.73 examples/s]
Map (num_proc=8):  80%|███████▉  | 142921/179733 [00:22<00:05, 6698.56 examples/s]
Map (num_proc=8):  87%|████████▋ | 157095/179733 [00:24<00:03, 6506.90 examples/s]
Map (num_proc=8):  92%|█████████▏| 164719/179733 [00:26<00:02, 6489.44 examples/s]
Map (num_proc=8):  91%|█████████ | 163856/179733 [00:25<00:02, 6395.95 examples/s]
Map (num_proc=8):  92%|█████████▏| 164477/179733 [00:25<00:02, 6522.37 examples/s]
Map (num_proc=8):  93%|█████████▎| 167110/179733 [00:26<00:01, 6378.44 examples/s]
Map (num_proc=8):  82%|████████▏ | 147422/179733 [00:23<00:05, 6403.17 examples/s]
Map (num_proc=8):  94%|█████████▍| 168611/179733 [00:26<00:01, 6624.24 examples/s]
Map (num_proc=8):  80%|███████▉  | 143615/179733 [00:22<00:05, 6654.23 examples/s]
Map (num_proc=8):  88%|████████▊ | 157762/179733 [00:25<00:03, 6367.08 examples/s]
Map (num_proc=8):  92%|█████████▏| 165374/179733 [00:26<00:02, 6420.29 examples/s]
Map (num_proc=8):  92%|█████████▏| 164499/179733 [00:25<00:02, 6386.49 examples/s]
Map (num_proc=8):  92%|█████████▏| 165143/179733 [00:26<00:02, 6544.84 examples/s]
Map (num_proc=8):  93%|█████████▎| 167787/179733 [00:26<00:01, 6458.12 examples/s]
Map (num_proc=8):  82%|████████▏ | 148067/179733 [00:23<00:04, 6369.83 examples/s]
Map (num_proc=8):  94%|█████████▍| 169302/179733 [00:26<00:01, 6486.83 examples/s]
Map (num_proc=8):  80%|████████  | 144307/179733 [00:22<00:05, 6649.78 examples/s]
Map (num_proc=8):  88%|████████▊ | 158424/179733 [00:25<00:03, 6329.64 examples/s]
Map (num_proc=8):  92%|█████████▏| 166030/179733 [00:26<00:02, 6424.94 examples/s]
Map (num_proc=8):  92%|█████████▏| 165147/179733 [00:25<00:02, 6407.39 examples/s]
Map (num_proc=8):  92%|█████████▏| 165803/179733 [00:26<00:02, 6530.90 examples/s]
Map (num_proc=8):  94%|█████████▎| 168448/179733 [00:26<00:01, 6490.21 examples/s]
Map (num_proc=8):  83%|████████▎ | 148709/179733 [00:23<00:04, 6360.61 examples/s]
Map (num_proc=8):  95%|█████████▍| 169987/179733 [00:26<00:01, 6508.47 examples/s]
Map (num_proc=8):  81%|████████  | 144976/179733 [00:22<00:05, 6529.83 examples/s]
Map (num_proc=8):  89%|████████▊ | 159064/179733 [00:25<00:03, 6330.54 examples/s]
Map (num_proc=8):  93%|█████████▎| 166675/179733 [00:26<00:02, 6241.70 examples/s]
Map (num_proc=8):  92%|█████████▏| 165793/179733 [00:26<00:02, 6400.35 examples/s]
Map (num_proc=8):  94%|█████████▍| 169138/179733 [00:26<00:01, 6524.85 examples/s]
Map (num_proc=8):  93%|█████████▎| 166494/179733 [00:26<00:02, 6463.14 examples/s]
Map (num_proc=8):  83%|████████▎ | 149349/179733 [00:23<00:04, 6340.88 examples/s]
Map (num_proc=8):  89%|████████▉ | 159725/179733 [00:25<00:03, 6397.34 examples/s]
Map (num_proc=8):  81%|████████  | 145649/179733 [00:23<00:05, 6531.84 examples/s]
Map (num_proc=8):  95%|█████████▍| 170666/179733 [00:26<00:01, 6375.89 examples/s]
Map (num_proc=8):  93%|█████████▎| 167312/179733 [00:26<00:01, 6255.37 examples/s]
Map (num_proc=8):  93%|█████████▎| 166465/179733 [00:26<00:02, 6468.86 examples/s]
Map (num_proc=8):  93%|█████████▎| 167144/179733 [00:26<00:01, 6434.62 examples/s]
Map (num_proc=8):  94%|█████████▍| 169798/179733 [00:26<00:01, 6453.41 examples/s]
Map (num_proc=8):  83%|████████▎ | 149994/179733 [00:23<00:04, 6337.36 examples/s]
Map (num_proc=8):  89%|████████▉ | 160395/179733 [00:25<00:02, 6464.03 examples/s]
Map (num_proc=8):  95%|█████████▌| 171342/179733 [00:26<00:01, 6452.88 examples/s]
Map (num_proc=8):  81%|████████▏ | 146327/179733 [00:23<00:05, 6266.32 examples/s]
Map (num_proc=8):  93%|█████████▎| 167970/179733 [00:26<00:01, 6205.43 examples/s]
Map (num_proc=8):  93%|█████████▎| 167125/179733 [00:26<00:01, 6483.13 examples/s]
Map (num_proc=8):  93%|█████████▎| 167788/179733 [00:26<00:01, 6423.93 examples/s]
Map (num_proc=8):  95%|█████████▍| 170463/179733 [00:26<00:01, 6486.16 examples/s]
Map (num_proc=8):  84%|████████▍ | 150646/179733 [00:24<00:04, 6358.13 examples/s]
Map (num_proc=8):  96%|█████████▌| 172011/179733 [00:26<00:01, 6502.80 examples/s]
Map (num_proc=8):  90%|████████▉ | 161066/179733 [00:25<00:02, 6298.37 examples/s]
Map (num_proc=8):  82%|████████▏ | 146989/179733 [00:23<00:05, 6358.92 examples/s]
Map (num_proc=8):  94%|█████████▍| 168626/179733 [00:26<00:01, 6289.43 examples/s]
Map (num_proc=8):  93%|█████████▎| 167788/179733 [00:26<00:01, 6520.96 examples/s]
Map (num_proc=8):  94%|█████████▎| 168443/179733 [00:26<00:01, 6442.36 examples/s]
Map (num_proc=8):  84%|████████▍ | 151313/179733 [00:24<00:04, 6420.25 examples/s]
Map (num_proc=8):  95%|█████████▌| 171146/179733 [00:26<00:01, 6053.07 examples/s]
Map (num_proc=8):  96%|█████████▌| 172663/179733 [00:26<00:01, 6503.61 examples/s]
Map (num_proc=8):  90%|████████▉ | 161724/179733 [00:25<00:02, 6333.62 examples/s]
Map (num_proc=8):  82%|████████▏ | 147648/179733 [00:23<00:05, 6407.25 examples/s]
Map (num_proc=8):  94%|█████████▍| 169296/179733 [00:26<00:01, 6385.56 examples/s]
Map (num_proc=8):  94%|█████████▎| 168450/179733 [00:26<00:01, 6526.71 examples/s]
Map (num_proc=8):  94%|█████████▍| 169094/179733 [00:26<00:01, 6434.62 examples/s]
Map (num_proc=8):  85%|████████▍ | 151978/179733 [00:24<00:04, 6464.13 examples/s]
Map (num_proc=8):  96%|█████████▌| 171801/179733 [00:26<00:01, 6164.78 examples/s]
Map (num_proc=8):  96%|█████████▋| 173355/179733 [00:27<00:00, 6606.69 examples/s]
Map (num_proc=8):  90%|█████████ | 162381/179733 [00:25<00:02, 6387.21 examples/s]
Map (num_proc=8):  83%|████████▎ | 148292/179733 [00:23<00:04, 6374.40 examples/s]
Map (num_proc=8):  94%|█████████▍| 169138/179733 [00:26<00:01, 6543.60 examples/s]
Map (num_proc=8):  95%|█████████▍| 169977/179733 [00:26<00:01, 6154.73 examples/s]
Map (num_proc=8):  94%|█████████▍| 169738/179733 [00:26<00:01, 6431.31 examples/s]
Map (num_proc=8):  85%|████████▍ | 152661/179733 [00:24<00:04, 6509.73 examples/s]
Map (num_proc=8):  96%|█████████▌| 172450/179733 [00:27<00:01, 6250.74 examples/s]
Map (num_proc=8):  91%|█████████ | 163035/179733 [00:25<00:02, 6399.79 examples/s]
Map (num_proc=8):  97%|█████████▋| 174048/179733 [00:27<00:00, 6516.73 examples/s]
Map (num_proc=8):  83%|████████▎ | 148957/179733 [00:23<00:04, 6238.29 examples/s]
Map (num_proc=8):  94%|█████████▍| 169807/179733 [00:26<00:01, 6573.55 examples/s]
Map (num_proc=8):  95%|█████████▍| 170632/179733 [00:27<00:01, 6250.52 examples/s]
Map (num_proc=8):  95%|█████████▍| 170397/179733 [00:26<00:01, 6444.72 examples/s]
Map (num_proc=8):  85%|████████▌ | 153325/179733 [00:24<00:04, 6516.33 examples/s]
Map (num_proc=8):  96%|█████████▋| 173132/179733 [00:27<00:01, 6404.72 examples/s]
Map (num_proc=8):  91%|█████████ | 163702/179733 [00:25<00:02, 6460.75 examples/s]
Map (num_proc=8):  97%|█████████▋| 174723/179733 [00:27<00:00, 6550.98 examples/s]
Map (num_proc=8):  83%|████████▎ | 149623/179733 [00:23<00:04, 6333.76 examples/s]
Map (num_proc=8):  95%|█████████▍| 170494/179733 [00:26<00:01, 6622.10 examples/s]
Map (num_proc=8):  95%|█████████▌| 171265/179733 [00:27<00:01, 6258.26 examples/s]
Map (num_proc=8):  95%|█████████▌| 171060/179733 [00:27<00:01, 6482.83 examples/s]
Map (num_proc=8):  86%|████████▌ | 153986/179733 [00:24<00:03, 6525.75 examples/s]
Map (num_proc=8):  97%|█████████▋| 173799/179733 [00:27<00:00, 6458.10 examples/s]
Map (num_proc=8):  91%|█████████▏| 164378/179733 [00:26<00:02, 6534.92 examples/s]
Map (num_proc=8):  98%|█████████▊| 175392/179733 [00:27<00:00, 6558.69 examples/s]
Map (num_proc=8):  84%|████████▎ | 150296/179733 [00:23<00:04, 6421.54 examples/s]
Map (num_proc=8):  96%|█████████▌| 171901/179733 [00:27<00:01, 6259.44 examples/s]
Map (num_proc=8):  95%|█████████▌| 171185/179733 [00:26<00:01, 6619.01 examples/s]
Map (num_proc=8):  96%|█████████▌| 171721/179733 [00:27<00:01, 6492.25 examples/s]
Map (num_proc=8):  86%|████████▌ | 154646/179733 [00:24<00:03, 6536.09 examples/s]
Map (num_proc=8):  97%|█████████▋| 174458/179733 [00:27<00:00, 6470.09 examples/s]
Map (num_proc=8):  92%|█████████▏| 165075/179733 [00:26<00:02, 6533.99 examples/s]
Map (num_proc=8):  84%|████████▍ | 150959/179733 [00:23<00:04, 6469.23 examples/s]
Map (num_proc=8):  98%|█████████▊| 176066/179733 [00:27<00:00, 6314.31 examples/s]
Map (num_proc=8):  96%|█████████▌| 171850/179733 [00:26<00:01, 6604.89 examples/s]
Map (num_proc=8):  96%|█████████▌| 172554/179733 [00:27<00:01, 6307.09 examples/s]
Map (num_proc=8):  96%|█████████▌| 172396/179733 [00:27<00:01, 6532.91 examples/s]
Map (num_proc=8):  86%|████████▋ | 155306/179733 [00:24<00:03, 6528.37 examples/s]
Map (num_proc=8):  97%|█████████▋| 175122/179733 [00:27<00:00, 6436.47 examples/s]
Map (num_proc=8):  92%|█████████▏| 165746/179733 [00:26<00:02, 6406.97 examples/s]
Map (num_proc=8):  84%|████████▍ | 151610/179733 [00:24<00:04, 6446.93 examples/s]
Map (num_proc=8):  98%|█████████▊| 176721/179733 [00:27<00:00, 6271.56 examples/s]
Map (num_proc=8):  96%|█████████▋| 173244/179733 [00:27<00:01, 6475.09 examples/s]
Map (num_proc=8):  96%|█████████▌| 172540/179733 [00:27<00:01, 6670.86 examples/s]
Map (num_proc=8):  96%|█████████▋| 173060/179733 [00:27<00:01, 6541.64 examples/s]
Map (num_proc=8):  87%|████████▋ | 155974/179733 [00:24<00:03, 6555.77 examples/s]
Map (num_proc=8):  98%|█████████▊| 175774/179733 [00:27<00:00, 6460.46 examples/s]
Map (num_proc=8):  93%|█████████▎| 166414/179733 [00:26<00:02, 6471.15 examples/s]
Map (num_proc=8):  85%|████████▍ | 152303/179733 [00:24<00:04, 6553.52 examples/s]
Map (num_proc=8):  99%|█████████▊| 177365/179733 [00:27<00:00, 6038.58 examples/s]
Map (num_proc=8):  96%|█████████▋| 173235/179733 [00:27<00:00, 6646.71 examples/s]
Map (num_proc=8):  97%|█████████▋| 173923/179733 [00:27<00:00, 6429.78 examples/s]
Map (num_proc=8):  97%|█████████▋| 173736/179733 [00:27<00:00, 6414.82 examples/s]
Map (num_proc=8):  87%|████████▋ | 156637/179733 [00:24<00:03, 6543.61 examples/s]
Map (num_proc=8):  98%|█████████▊| 176442/179733 [00:27<00:00, 6430.91 examples/s]
Map (num_proc=8):  93%|█████████▎| 167070/179733 [00:26<00:01, 6478.22 examples/s]
Map (num_proc=8):  85%|████████▌ | 152974/179733 [00:24<00:04, 6561.83 examples/s]
Map (num_proc=8):  97%|█████████▋| 174606/179733 [00:27<00:00, 6461.48 examples/s]
Map (num_proc=8):  97%|█████████▋| 173936/179733 [00:27<00:00, 6545.42 examples/s]
Map (num_proc=8):  97%|█████████▋| 174384/179733 [00:27<00:00, 6382.29 examples/s]
Map (num_proc=8):  99%|█████████▉| 177980/179733 [00:27<00:00, 5522.00 examples/s]
Map (num_proc=8):  88%|████████▊ | 157333/179733 [00:25<00:03, 6357.68 examples/s]
Map (num_proc=8):  99%|█████████▊| 177106/179733 [00:27<00:00, 6266.89 examples/s]
Map (num_proc=8):  85%|████████▌ | 153649/179733 [00:24<00:03, 6586.42 examples/s]
Map (num_proc=8):  93%|█████████▎| 167749/179733 [00:26<00:01, 6345.68 examples/s]
Map (num_proc=8):  98%|█████████▊| 175260/179733 [00:27<00:00, 6458.99 examples/s]
Map (num_proc=8):  97%|█████████▋| 174610/179733 [00:27<00:00, 6571.73 examples/s]
Map (num_proc=8):  97%|█████████▋| 175028/179733 [00:27<00:00, 6352.22 examples/s]
Map (num_proc=8):  88%|████████▊ | 157975/179733 [00:25<00:03, 6357.47 examples/s]
Map (num_proc=8):  99%|█████████▉| 178574/179733 [00:27<00:00, 5383.32 examples/s]
Map (num_proc=8):  86%|████████▌ | 154326/179733 [00:24<00:03, 6611.49 examples/s]
Map (num_proc=8):  94%|█████████▎| 168409/179733 [00:26<00:01, 6399.87 examples/s]
Map (num_proc=8):  99%|█████████▉| 177779/179733 [00:27<00:00, 5780.61 examples/s]
Map (num_proc=8):  98%|█████████▊| 175303/179733 [00:27<00:00, 6659.24 examples/s]
Map (num_proc=8):  98%|█████████▊| 175929/179733 [00:27<00:00, 6260.39 examples/s]
Map (num_proc=8):  98%|█████████▊| 175706/179733 [00:27<00:00, 6118.68 examples/s]
Map (num_proc=8):  88%|████████▊ | 158627/179733 [00:25<00:03, 6368.66 examples/s]
Map (num_proc=8):  86%|████████▌ | 155014/179733 [00:24<00:03, 6666.23 examples/s]
Map (num_proc=8):  94%|█████████▍| 169087/179733 [00:26<00:01, 6388.68 examples/s]
Map (num_proc=8):  99%|█████████▉| 178378/179733 [00:27<00:00, 5776.87 examples/s]
Map (num_proc=8):  98%|█████████▊| 175985/179733 [00:27<00:00, 6637.08 examples/s]
Map (num_proc=8): 100%|█████████▉| 179157/179733 [00:28<00:00, 4471.40 examples/s]
Map (num_proc=8):  89%|████████▊ | 159267/179733 [00:25<00:03, 6352.30 examples/s]
Map (num_proc=8):  98%|█████████▊| 176323/179733 [00:27<00:00, 5924.28 examples/s]
Map (num_proc=8):  87%|████████▋ | 155710/179733 [00:24<00:03, 6722.40 examples/s]
Map (num_proc=8):  98%|█████████▊| 176558/179733 [00:28<00:00, 5395.53 examples/s]
Map (num_proc=8):  94%|█████████▍| 169759/179733 [00:26<00:01, 6473.16 examples/s]
Map (num_proc=8):  98%|█████████▊| 176665/179733 [00:27<00:00, 6654.21 examples/s]
Map (num_proc=8):  89%|████████▉ | 159928/179733 [00:25<00:03, 6388.49 examples/s]
Map (num_proc=8):  98%|█████████▊| 176934/179733 [00:27<00:00, 5898.40 examples/s]
Map (num_proc=8): 100%|█████████▉| 178991/179733 [00:28<00:00, 4882.83 examples/s]
Map (num_proc=8):  87%|████████▋ | 156386/179733 [00:24<00:03, 6602.24 examples/s]
Map (num_proc=8):  95%|█████████▍| 170449/179733 [00:26<00:01, 6573.33 examples/s]
Map (num_proc=8):  99%|█████████▊| 177343/179733 [00:27<00:00, 6615.97 examples/s]
Map (num_proc=8):  89%|████████▉ | 160593/179733 [00:25<00:03, 6326.04 examples/s]
Map (num_proc=8):  99%|█████████▊| 177154/179733 [00:28<00:00, 4530.44 examples/s]
Map (num_proc=8):  87%|████████▋ | 157047/179733 [00:24<00:03, 6592.57 examples/s]
Map (num_proc=8):  95%|█████████▌| 171118/179733 [00:27<00:01, 6573.64 examples/s]
Map (num_proc=8):  99%|█████████▉| 177545/179733 [00:28<00:00, 5031.51 examples/s]
Map (num_proc=8):  90%|████████▉ | 161252/179733 [00:25<00:02, 6364.63 examples/s]
Map (num_proc=8):  99%|█████████▉| 178027/179733 [00:27<00:00, 6123.96 examples/s]
Map (num_proc=8): 100%|█████████▉| 179533/179733 [00:28<00:00, 3991.93 examples/s]
Map (num_proc=8):  88%|████████▊ | 157756/179733 [00:24<00:03, 6725.72 examples/s]
Map (num_proc=8): 100%|█████████▉| 179645/179733 [00:28<00:00, 2836.70 examples/s]
Map (num_proc=8):  96%|█████████▌| 171791/179733 [00:27<00:01, 6603.17 examples/s]
Map (num_proc=8):  90%|█████████ | 161916/179733 [00:25<00:02, 6389.16 examples/s]
Map (num_proc=8):  99%|█████████▉| 178076/179733 [00:28<00:00, 4469.15 examples/s]
Map (num_proc=8):  96%|█████████▌| 172472/179733 [00:27<00:01, 6633.21 examples/s]
Map (num_proc=8):  88%|████████▊ | 158464/179733 [00:25<00:03, 6633.45 examples/s]
Map (num_proc=8):  99%|█████████▉| 178675/179733 [00:28<00:00, 5448.18 examples/s]
Map (num_proc=8):  99%|█████████▉| 177683/179733 [00:28<00:00, 3390.10 examples/s]
Map (num_proc=8):  90%|█████████ | 162569/179733 [00:25<00:02, 6401.74 examples/s]
Map (num_proc=8):  96%|█████████▋| 173149/179733 [00:27<00:00, 6651.55 examples/s]
Map (num_proc=8):  89%|████████▊ | 159139/179733 [00:25<00:03, 6627.84 examples/s]
Map (num_proc=8):  91%|█████████ | 163238/179733 [00:25<00:02, 6448.28 examples/s]
Map (num_proc=8):  99%|█████████▉| 178553/179733 [00:28<00:00, 3661.79 examples/s]
Map (num_proc=8):  89%|████████▉ | 159815/179733 [00:25<00:03, 6639.26 examples/s]
Map (num_proc=8):  97%|█████████▋| 173863/179733 [00:27<00:00, 6600.52 examples/s]
Map (num_proc=8):  91%|█████████ | 163900/179733 [00:26<00:02, 6483.44 examples/s]
Map (num_proc=8):  99%|█████████▉| 178101/179733 [00:28<00:00, 2736.33 examples/s]
Map (num_proc=8):  89%|████████▉ | 160499/179733 [00:25<00:02, 6634.69 examples/s]
Map (num_proc=8): 100%|██████████| 179733/179733 [00:28<00:00, 6224.58 examples/s]
+
Map (num_proc=8):  97%|█████████▋| 174531/179733 [00:27<00:00, 6606.72 examples/s]
Map (num_proc=8): 100%|██████████| 179733/179733 [00:28<00:00, 6239.20 examples/s]
+
Filter:   0%|          | 0/179733 [00:00<?, ? examples/s]
Filter:   0%|          | 0/179733 [00:00<?, ? examples/s]
Map (num_proc=8):  92%|█████████▏| 164560/179733 [00:26<00:02, 6498.40 examples/s]
Map (num_proc=8): 100%|█████████▉| 179266/179733 [00:28<00:00, 3241.23 examples/s]
Map (num_proc=8):  90%|████████▉ | 161166/179733 [00:25<00:02, 6628.25 examples/s]
Map (num_proc=8):  97%|█████████▋| 175199/179733 [00:27<00:00, 6602.45 examples/s]
Filter:  13%|█▎        | 23000/179733 [00:00<00:00, 212585.10 examples/s]
Filter:  13%|█▎        | 23000/179733 [00:00<00:00, 220981.41 examples/s]
Map (num_proc=8):  92%|█████████▏| 165243/179733 [00:26<00:02, 6417.42 examples/s]
Map (num_proc=8):  99%|█████████▉| 178463/179733 [00:28<00:00, 2434.63 examples/s]
Map (num_proc=8):  90%|█████████ | 161851/179733 [00:25<00:02, 6656.36 examples/s]
Map (num_proc=8):  98%|█████████▊| 175902/179733 [00:27<00:00, 6545.50 examples/s]
Filter:  26%|██▌       | 46000/179733 [00:00<00:00, 223427.84 examples/s]
Filter:  25%|██▌       | 45000/179733 [00:00<00:00, 211391.91 examples/s]
Map (num_proc=8):  92%|█████████▏| 165912/179733 [00:26<00:02, 6476.95 examples/s]
Map (num_proc=8):  90%|█████████ | 162532/179733 [00:25<00:02, 6685.76 examples/s]
Map (num_proc=8):  98%|█████████▊| 176595/179733 [00:27<00:00, 6585.61 examples/s]
Map (num_proc=8): 100%|█████████▉| 179719/179733 [00:28<00:00, 2729.60 examples/s]
Filter:  39%|███▉      | 70000/179733 [00:00<00:00, 224915.75 examples/s]
Filter:  38%|███▊      | 69000/179733 [00:00<00:00, 218798.96 examples/s]
Map (num_proc=8):  93%|█████████▎| 166587/179733 [00:26<00:02, 6527.36 examples/s]
Map (num_proc=8):  99%|█████████▉| 178747/179733 [00:29<00:00, 2143.64 examples/s]
Map (num_proc=8): 100%|█████████▉| 178977/179733 [00:28<00:00, 1987.30 examples/s]
Map (num_proc=8):  91%|█████████ | 163224/179733 [00:25<00:02, 6606.83 examples/s]
Map (num_proc=8): 100%|██████████| 179733/179733 [00:28<00:00, 6241.45 examples/s]
+
Map (num_proc=8):  99%|█████████▊| 177277/179733 [00:28<00:00, 6024.92 examples/s]
Filter:  52%|█████▏    | 93000/179733 [00:00<00:00, 224149.96 examples/s]
Map (num_proc=8):  93%|█████████▎| 167268/179733 [00:26<00:01, 6496.02 examples/s]
Filter:  51%|█████     | 92000/179733 [00:00<00:00, 217740.68 examples/s]
Filter:   0%|          | 0/179733 [00:00<?, ? examples/s]
Map (num_proc=8):  91%|█████████ | 163921/179733 [00:25<00:02, 6655.66 examples/s]
Map (num_proc=8): 100%|█████████▉| 179007/179733 [00:29<00:00, 2008.67 examples/s]
Filter:  65%|██████▌   | 117000/179733 [00:00<00:00, 227243.18 examples/s]
Map (num_proc=8):  93%|█████████▎| 167947/179733 [00:26<00:01, 6567.49 examples/s]
Filter:  65%|██████▍   | 116000/179733 [00:00<00:00, 220916.59 examples/s]
Filter:  13%|█▎        | 23000/179733 [00:00<00:00, 219119.23 examples/s]
Map (num_proc=8):  99%|█████████▉| 177897/179733 [00:28<00:00, 5242.86 examples/s]
Map (num_proc=8):  92%|█████████▏| 164620/179733 [00:25<00:02, 6741.57 examples/s]
Map (num_proc=8):  94%|█████████▍| 168626/179733 [00:26<00:01, 6593.19 examples/s]
Filter:  78%|███████▊  | 141000/179733 [00:00<00:00, 228153.65 examples/s]
Filter:  78%|███████▊  | 140000/179733 [00:00<00:00, 221812.20 examples/s]
Filter:  26%|██▌       | 47000/179733 [00:00<00:00, 223951.00 examples/s]
Map (num_proc=8): 100%|█████████▉| 179240/179733 [00:29<00:00, 1886.35 examples/s]
Map (num_proc=8):  92%|█████████▏| 165319/179733 [00:26<00:02, 6789.56 examples/s]
Map (num_proc=8): 100%|█████████▉| 179292/179733 [00:29<00:00, 1584.72 examples/s]
Map (num_proc=8):  99%|█████████▉| 178454/179733 [00:28<00:00, 4637.31 examples/s]
Map (num_proc=8):  94%|█████████▍| 169286/179733 [00:26<00:01, 6576.50 examples/s]
Filter:  92%|█████████▏| 165000/179733 [00:00<00:00, 229152.52 examples/s]
Filter:  91%|█████████ | 164000/179733 [00:00<00:00, 224724.61 examples/s]
Filter:  40%|███▉      | 71000/179733 [00:00<00:00, 223134.16 examples/s]
Map (num_proc=8):  92%|█████████▏| 166043/179733 [00:26<00:02, 6484.16 examples/s]
Map (num_proc=8): 100%|█████████▉| 179460/179733 [00:29<00:00, 1795.29 examples/s]
Filter: 100%|██████████| 179733/179733 [00:00<00:00, 226817.60 examples/s]
+
Filter: 100%|██████████| 179733/179733 [00:00<00:00, 221211.66 examples/s]
+
Map (num_proc=8):  95%|█████████▍| 169956/179733 [00:27<00:01, 6581.28 examples/s]
Filter:  53%|█████▎    | 95000/179733 [00:00<00:00, 225385.99 examples/s]
Map (num_proc=8):  93%|█████████▎| 166709/179733 [00:26<00:01, 6521.51 examples/s]
Map (num_proc=8): 100%|█████████▉| 179664/179733 [00:29<00:00, 1806.00 examples/s]
Map (num_proc=8):  95%|█████████▍| 170628/179733 [00:27<00:01, 6597.16 examples/s]
Filter:  66%|██████▌   | 119000/179733 [00:00<00:00, 226523.73 examples/s]
Map (num_proc=8):  93%|█████████▎| 167371/179733 [00:26<00:01, 6523.17 examples/s]
Map (num_proc=8): 100%|█████████▉| 179576/179733 [00:29<00:00, 1384.69 examples/s]
Map (num_proc=8):  95%|█████████▌| 171312/179733 [00:27<00:01, 6631.32 examples/s]
Map (num_proc=8): 100%|██████████| 179733/179733 [00:29<00:00, 6008.08 examples/s]
+
Filter:  80%|███████▉  | 143000/179733 [00:00<00:00, 226966.73 examples/s]
Map (num_proc=8):  94%|█████████▎| 168063/179733 [00:26<00:01, 6499.65 examples/s]
Map (num_proc=8): 100%|█████████▉| 178959/179733 [00:28<00:00, 2743.80 examples/s]
Filter:   0%|          | 0/179733 [00:00<?, ? examples/s]
Map (num_proc=8):  96%|█████████▌| 171978/179733 [00:27<00:01, 6613.43 examples/s]
Filter:  93%|█████████▎| 168000/179733 [00:00<00:00, 229444.86 examples/s]
Map (num_proc=8):  94%|█████████▍| 168723/179733 [00:26<00:01, 6472.59 examples/s]
Filter:  13%|█▎        | 23000/179733 [00:00<00:00, 218298.61 examples/s]
Filter: 100%|██████████| 179733/179733 [00:00<00:00, 226290.62 examples/s]
+
Map (num_proc=8):  96%|█████████▌| 172641/179733 [00:27<00:01, 6603.41 examples/s]
Map (num_proc=8): 100%|██████████| 179733/179733 [00:29<00:00, 6007.53 examples/s]
+
Map (num_proc=8):  94%|█████████▍| 169379/179733 [00:26<00:01, 6483.28 examples/s]
Filter:   0%|          | 0/179733 [00:00<?, ? examples/s]
Filter:  26%|██▌       | 47000/179733 [00:00<00:00, 220815.41 examples/s]
Map (num_proc=8):  96%|█████████▋| 173329/179733 [00:27<00:00, 6646.84 examples/s]
Map (num_proc=8):  95%|█████████▍| 170069/179733 [00:26<00:01, 6574.13 examples/s]
Filter:  13%|█▎        | 23000/179733 [00:00<00:00, 216391.34 examples/s]
Map (num_proc=8):  97%|█████████▋| 174026/179733 [00:27<00:00, 6726.43 examples/s]
Filter:  40%|███▉      | 71000/179733 [00:00<00:00, 225367.38 examples/s]
Map (num_proc=8):  95%|█████████▌| 170748/179733 [00:26<00:01, 6610.06 examples/s]
Filter:  26%|██▌       | 47000/179733 [00:00<00:00, 225440.79 examples/s]
Map (num_proc=8):  97%|█████████▋| 174711/179733 [00:27<00:00, 6730.67 examples/s]
Filter:  53%|█████▎    | 95000/179733 [00:00<00:00, 226273.69 examples/s]
Map (num_proc=8):  95%|█████████▌| 171436/179733 [00:27<00:01, 6670.08 examples/s]
Filter:  40%|███▉      | 71000/179733 [00:00<00:00, 226750.77 examples/s]
Map (num_proc=8): 100%|█████████▉| 179352/179733 [00:29<00:00, 1756.17 examples/s]
Map (num_proc=8):  98%|█████████▊| 175406/179733 [00:27<00:00, 6770.04 examples/s]
Filter:  66%|██████▌   | 119000/179733 [00:00<00:00, 228986.56 examples/s]
Map (num_proc=8):  96%|█████████▌| 172140/179733 [00:27<00:01, 6705.78 examples/s]
Filter:  53%|█████▎    | 95000/179733 [00:00<00:00, 228373.48 examples/s]
Map (num_proc=8):  98%|█████████▊| 176102/179733 [00:27<00:00, 6692.71 examples/s]
Filter:  80%|███████▉  | 143000/179733 [00:00<00:00, 229982.27 examples/s]
Map (num_proc=8):  96%|█████████▌| 172823/179733 [00:27<00:01, 6699.99 examples/s]
Filter:  66%|██████▌   | 119000/179733 [00:00<00:00, 228293.57 examples/s]
Filter:  93%|█████████▎| 167000/179733 [00:00<00:00, 228532.85 examples/s]
Map (num_proc=8):  98%|█████████▊| 176777/179733 [00:28<00:00, 6383.26 examples/s]
Filter: 100%|██████████| 179733/179733 [00:00<00:00, 226654.34 examples/s]
+
Map (num_proc=8):  97%|█████████▋| 173516/179733 [00:27<00:00, 6601.73 examples/s]
Filter:  80%|███████▉  | 143000/179733 [00:00<00:00, 226989.47 examples/s]
Map (num_proc=8):  99%|█████████▊| 177435/179733 [00:28<00:00, 6010.42 examples/s]
Map (num_proc=8):  97%|█████████▋| 174181/179733 [00:27<00:00, 6595.41 examples/s]
Map (num_proc=8): 100%|█████████▉| 179658/179733 [00:29<00:00, 1390.22 examples/s]
Filter:  93%|█████████▎| 168000/179733 [00:00<00:00, 229606.52 examples/s]
Filter: 100%|██████████| 179733/179733 [00:00<00:00, 227290.02 examples/s]
+
Map (num_proc=8):  99%|█████████▉| 178085/179733 [00:28<00:00, 6026.61 examples/s]
Map (num_proc=8):  97%|█████████▋| 174851/179733 [00:27<00:00, 6603.64 examples/s]
Map (num_proc=8):  99%|█████████▉| 178707/179733 [00:28<00:00, 5905.12 examples/s]
Map (num_proc=8): 100%|██████████| 179733/179733 [00:29<00:00, 6017.47 examples/s]
+
Map (num_proc=8):  98%|█████████▊| 175547/179733 [00:27<00:00, 6538.41 examples/s]
Filter:   0%|          | 0/179733 [00:00<?, ? examples/s]
Map (num_proc=8):  98%|█████████▊| 176215/179733 [00:27<00:00, 6554.12 examples/s]
Map (num_proc=8): 100%|█████████▉| 179315/179733 [00:28<00:00, 5113.69 examples/s]
Filter:  13%|█▎        | 23000/179733 [00:00<00:00, 217834.26 examples/s]
Map (num_proc=8):  98%|█████████▊| 176878/179733 [00:27<00:00, 6551.61 examples/s]
Filter:  26%|██▌       | 47000/179733 [00:00<00:00, 221293.22 examples/s]
Map (num_proc=8):  99%|█████████▉| 177553/179733 [00:27<00:00, 6490.99 examples/s]
Filter:  40%|███▉      | 71000/179733 [00:00<00:00, 224787.29 examples/s]
Map (num_proc=8):  99%|█████████▉| 178208/179733 [00:28<00:00, 5972.50 examples/s]
Filter:  53%|█████▎    | 95000/179733 [00:00<00:00, 226374.59 examples/s]
Filter:  66%|██████▌   | 119000/179733 [00:00<00:00, 227148.98 examples/s]
Map (num_proc=8): 100%|█████████▉| 178840/179733 [00:28<00:00, 4325.51 examples/s]
Filter:  80%|███████▉  | 143000/179733 [00:00<00:00, 226500.26 examples/s]
Map (num_proc=8): 100%|██████████| 179733/179733 [00:29<00:00, 6173.23 examples/s]
+
Filter:   0%|          | 0/179733 [00:00<?, ? examples/s]
Filter:  93%|█████████▎| 167000/179733 [00:00<00:00, 228516.98 examples/s]
Filter: 100%|██████████| 179733/179733 [00:00<00:00, 226212.12 examples/s]
+
Filter:  13%|█▎        | 23000/179733 [00:00<00:00, 220234.26 examples/s]
Map (num_proc=8): 100%|█████████▉| 179351/179733 [00:28<00:00, 3675.50 examples/s]
Filter:  26%|██▌       | 46000/179733 [00:00<00:00, 221908.44 examples/s]
Filter:  39%|███▉      | 70000/179733 [00:00<00:00, 225502.36 examples/s]
Filter:  52%|█████▏    | 94000/179733 [00:00<00:00, 225874.78 examples/s]
Filter:  70%|███████   | 126000/179733 [00:00<00:00, 216288.29 examples/s]
Map (num_proc=8): 100%|██████████| 179733/179733 [00:28<00:00, 6206.94 examples/s]
+
Map (num_proc=8):   0%|          | 0/5391 [00:00<?, ? examples/s]
Filter:   0%|          | 0/179733 [00:00<?, ? examples/s]
Filter:  83%|████████▎ | 150000/179733 [00:00<00:00, 218934.14 examples/s]
Filter:  13%|█▎        | 23000/179733 [00:00<00:00, 220505.59 examples/s]
Filter:  97%|█████████▋| 174000/179733 [00:00<00:00, 223403.48 examples/s]
Map (num_proc=8):   0%|          | 0/5391 [00:00<?, ? examples/s]
Filter: 100%|██████████| 179733/179733 [00:00<00:00, 221338.97 examples/s]
+
Filter:  26%|██▌       | 47000/179733 [00:00<00:00, 226530.12 examples/s]
Filter:  40%|███▉      | 71000/179733 [00:00<00:00, 227219.83 examples/s]
Map (num_proc=8):   0%|          | 0/5391 [00:00<?, ? examples/s]
Filter:  53%|█████▎    | 95000/179733 [00:00<00:00, 227507.64 examples/s]
Filter:  66%|██████▌   | 119000/179733 [00:00<00:00, 226409.30 examples/s]
Map (num_proc=8):   0%|          | 22/5391 [00:00<02:30, 35.71 examples/s]
Filter:  80%|███████▉  | 143000/179733 [00:00<00:00, 227195.19 examples/s]
Map (num_proc=8):   3%|▎         | 163/5391 [00:00<00:18, 288.12 examples/s]
Filter:  93%|█████████▎| 167000/179733 [00:00<00:00, 228586.32 examples/s]
Map (num_proc=8):   0%|          | 18/5391 [00:00<03:09, 28.35 examples/s]
Filter: 100%|██████████| 179733/179733 [00:00<00:00, 226907.99 examples/s]
+
Map (num_proc=8):   7%|▋         | 379/5391 [00:00<00:07, 676.07 examples/s]
Map (num_proc=8):   3%|▎         | 152/5391 [00:00<00:19, 269.19 examples/s]
Map (num_proc=8):  13%|█▎        | 700/5391 [00:00<00:03, 1249.89 examples/s]
Map (num_proc=8):   7%|▋         | 367/5391 [00:00<00:07, 658.58 examples/s]
Map (num_proc=8):  21%|██        | 1115/5391 [00:01<00:02, 1960.01 examples/s]
Map (num_proc=8):  13%|█▎        | 681/5391 [00:00<00:03, 1220.99 examples/s]
Map (num_proc=8):  30%|███       | 1631/5391 [00:01<00:01, 2780.10 examples/s]
Map (num_proc=8):   0%|          | 0/5391 [00:00<?, ? examples/s]
Map (num_proc=8):   0%|          | 21/5391 [00:00<03:03, 29.24 examples/s]
Map (num_proc=8):  20%|██        | 1099/5391 [00:01<00:02, 1936.04 examples/s]
Map (num_proc=8):  42%|████▏     | 2255/5391 [00:01<00:00, 3692.27 examples/s]
Map (num_proc=8):   0%|          | 0/5391 [00:00<?, ? examples/s]
Map (num_proc=8):   5%|▍         | 257/5391 [00:00<00:12, 409.78 examples/s]
Map (num_proc=8):  31%|███       | 1656/5391 [00:01<00:01, 2862.37 examples/s]
Map (num_proc=8):  54%|█████▎    | 2887/5391 [00:01<00:00, 4399.19 examples/s]
Map (num_proc=8):   9%|▉         | 504/5391 [00:00<00:06, 797.23 examples/s]
Map (num_proc=8):  43%|████▎     | 2293/5391 [00:01<00:00, 3789.16 examples/s]
Map (num_proc=8):  65%|██████▌   | 3516/5391 [00:01<00:00, 4917.22 examples/s]
Map (num_proc=8):  16%|█▋        | 884/5391 [00:01<00:03, 1427.92 examples/s]
Map (num_proc=8):  54%|█████▍    | 2927/5391 [00:01<00:00, 4471.77 examples/s]
Map (num_proc=8):  76%|███████▋  | 4118/5391 [00:01<00:00, 5227.55 examples/s]
Map (num_proc=8):  25%|██▍       | 1341/5391 [00:01<00:01, 2147.89 examples/s]
Map (num_proc=8):  66%|██████▌   | 3570/5391 [00:01<00:00, 5011.58 examples/s]
Map (num_proc=8):  87%|████████▋ | 4679/5391 [00:01<00:00, 5280.17 examples/s]
Map (num_proc=8):  35%|███▍      | 1881/5391 [00:01<00:01, 2946.22 examples/s]
Map (num_proc=8):  78%|███████▊  | 4195/5391 [00:01<00:00, 5210.66 examples/s]
Map (num_proc=8):  47%|████▋     | 2522/5391 [00:01<00:00, 3837.72 examples/s]
Map (num_proc=8):   0%|          | 21/5391 [00:00<02:54, 30.72 examples/s]
Map (num_proc=8):  88%|████████▊ | 4769/5391 [00:01<00:00, 5119.65 examples/s]
Map (num_proc=8):  98%|█████████▊| 5261/5391 [00:01<00:00, 3953.54 examples/s]
Map (num_proc=8):  59%|█████▉    | 3179/5391 [00:01<00:00, 4557.81 examples/s]
Map (num_proc=8):   3%|▎         | 184/5391 [00:00<00:17, 304.16 examples/s]
Map (num_proc=8):  71%|███████   | 3815/5391 [00:01<00:00, 5053.60 examples/s]
Map (num_proc=8):   0%|          | 17/5391 [00:00<03:51, 23.25 examples/s]
Map (num_proc=8):   8%|▊         | 442/5391 [00:00<00:06, 748.49 examples/s]
Map (num_proc=8):  99%|█████████▊| 5321/5391 [00:01<00:00, 3785.36 examples/s]
Map (num_proc=8):  82%|████████▏ | 4408/5391 [00:01<00:00, 3107.32 examples/s]
Map (num_proc=8):  12%|█▏        | 631/5391 [00:01<00:06, 699.34 examples/s]
Map (num_proc=8):   2%|▏         | 88/5391 [00:01<00:54, 96.85 examples/s]
Map (num_proc=8): 100%|██████████| 5391/5391 [00:02<00:00, 2288.33 examples/s]
+
Map (num_proc=8): 100%|██████████| 5391/5391 [00:02<00:00, 2423.04 examples/s]
+
Map (num_proc=8):  90%|█████████ | 4877/5391 [00:01<00:00, 3392.00 examples/s]
Map (num_proc=8):  21%|██▏       | 1151/5391 [00:01<00:02, 1498.83 examples/s]
Filter:   0%|          | 0/5391 [00:00<?, ? examples/s]
Map (num_proc=8):   8%|▊         | 452/5391 [00:01<00:08, 599.21 examples/s]
Map (num_proc=8):   0%|          | 0/5391 [00:00<?, ? examples/s]
Filter: 100%|██████████| 5391/5391 [00:00<00:00, 161857.50 examples/s]
+
Filter:   0%|          | 0/5391 [00:00<?, ? examples/s]
Filter: 100%|██████████| 5391/5391 [00:00<00:00, 157668.07 examples/s]
+
Map (num_proc=8):  32%|███▏      | 1743/5391 [00:01<00:01, 2390.52 examples/s]
Map (num_proc=8):  19%|█▉        | 1046/5391 [00:01<00:02, 1496.47 examples/s][2025-06-30 06:37:57,517] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+[2025-06-30 06:37:57,585] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+
Map (num_proc=8):  44%|████▍     | 2373/5391 [00:01<00:00, 3262.85 examples/s]
Map (num_proc=8):  99%|█████████▉| 5328/5391 [00:02<00:00, 2967.43 examples/s]
Map (num_proc=8):  31%|███       | 1678/5391 [00:01<00:01, 2427.52 examples/s]
Map (num_proc=8):  56%|█████▌    | 3004/5391 [00:01<00:00, 3995.07 examples/s]
Map (num_proc=8):  43%|████▎     | 2322/5391 [00:01<00:00, 3289.11 examples/s]
Map (num_proc=8): 100%|██████████| 5391/5391 [00:02<00:00, 2275.57 examples/s]
+
Map (num_proc=8):  67%|██████▋   | 3621/5391 [00:01<00:00, 4546.51 examples/s]
Map (num_proc=8):  55%|█████▍    | 2965/5391 [00:01<00:00, 4023.51 examples/s]
Filter:   0%|          | 0/5391 [00:00<?, ? examples/s]
Filter: 100%|██████████| 5391/5391 [00:00<00:00, 164028.50 examples/s]
+
Map (num_proc=8):  80%|███████▉  | 4288/5391 [00:01<00:00, 5097.83 examples/s]
Map (num_proc=8):  67%|██████▋   | 3610/5391 [00:01<00:00, 4623.13 examples/s][2025-06-30 06:37:57,968] [INFO] [logging.py:128:log_dist] [Rank 0] DeepSpeed info: version=0.16.3, git-hash=unknown, git-branch=unknown
+[2025-06-30 06:37:57,969] [INFO] [comm.py:677:init_distributed] Distributed backend already initialized
+[2025-06-30 06:37:57,969] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+[2025-06-30 06:37:57,978] [INFO] [logging.py:128:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False
+[2025-06-30 06:37:57,979] [INFO] [logging.py:128:log_dist] [Rank 0] Using client Optimizer as basic optimizer
+[2025-06-30 06:37:57,979] [INFO] [logging.py:128:log_dist] [Rank 0] Removing param_group that has no 'params' in the basic Optimizer
+[2025-06-30 06:37:57,993] [INFO] [logging.py:128:log_dist] [Rank 0] DeepSpeed Basic Optimizer = FusedAdam
+[2025-06-30 06:37:57,993] [INFO] [utils.py:59:is_zero_supported_optimizer] Checking ZeRO support for optimizer=FusedAdam type=<class 'deepspeed.ops.adam.fused_adam.FusedAdam'>
+[2025-06-30 06:37:57,993] [INFO] [logging.py:128:log_dist] [Rank 0] Creating fp16 ZeRO stage 3 optimizer, MiCS is enabled False, Hierarchical params gather False
+[2025-06-30 06:37:57,993] [INFO] [logging.py:128:log_dist] [Rank 0] Creating torch.bfloat16 ZeRO stage 3 optimizer
+
Map (num_proc=8):  79%|███████▉  | 4255/5391 [00:01<00:00, 5101.35 examples/s]
Map (num_proc=8):  90%|█████████ | 4878/5391 [00:01<00:00, 5197.53 examples/s]
Map (num_proc=8):   0%|          | 21/5391 [00:00<02:35, 34.43 examples/s]
Map (num_proc=8):   0%|          | 0/5391 [00:00<?, ? examples/s]
Map (num_proc=8):  90%|█████████ | 4866/5391 [00:01<00:00, 5371.56 examples/s]
Map (num_proc=8):   3%|▎         | 159/5391 [00:00<00:18, 278.27 examples/s][2025-06-30 06:37:58,205] [INFO] [utils.py:781:see_memory_usage] Stage 3 initialize beginning
+[2025-06-30 06:37:58,206] [INFO] [utils.py:782:see_memory_usage] MA 1.77 GB         Max_MA 3.8 GB         CA 3.88 GB         Max_CA 4 GB 
+[2025-06-30 06:37:58,206] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 140.79 GB, percent = 14.0%
+[2025-06-30 06:37:58,208] [INFO] [stage3.py:169:__init__] Reduce bucket size 500000000
+[2025-06-30 06:37:58,208] [INFO] [stage3.py:170:__init__] Prefetch bucket size 50000000
+
Map (num_proc=8):   7%|▋         | 403/5391 [00:00<00:06, 724.21 examples/s]
Map (num_proc=8): 100%|██████████| 5391/5391 [00:02<00:00, 2486.01 examples/s]
+
Map (num_proc=8): 100%|██████████| 5391/5391 [00:02<00:00, 2539.97 examples/s]
+
Filter:   0%|          | 0/5391 [00:00<?, ? examples/s]
Map (num_proc=8):  14%|█▍        | 749/5391 [00:00<00:03, 1339.50 examples/s]
Filter: 100%|██████████| 5391/5391 [00:00<00:00, 149609.25 examples/s]
+[2025-06-30 06:37:58,398] [INFO] [utils.py:781:see_memory_usage] DeepSpeedZeRoOffload initialize [begin]
+[2025-06-30 06:37:58,399] [INFO] [utils.py:782:see_memory_usage] MA 1.77 GB         Max_MA 1.77 GB         CA 3.88 GB         Max_CA 4 GB 
+[2025-06-30 06:37:58,399] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 136.81 GB, percent = 13.6%
+Parameter Offload: Total persistent parameters: 333312 in 141 params
+
Filter:   0%|          | 0/5391 [00:00<?, ? examples/s]
Filter: 100%|██████████| 5391/5391 [00:00<00:00, 161930.53 examples/s]
+
Map (num_proc=8):  22%|██▏       | 1176/5391 [00:01<00:02, 2051.55 examples/s][2025-06-30 06:37:58,474] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+[2025-06-30 06:37:58,533] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+
Map (num_proc=8):  32%|███▏      | 1729/5391 [00:01<00:01, 2946.09 examples/s][2025-06-30 06:37:58,608] [INFO] [utils.py:781:see_memory_usage] DeepSpeedZeRoOffload initialize [end]
+[2025-06-30 06:37:58,609] [INFO] [utils.py:782:see_memory_usage] MA 1.77 GB         Max_MA 1.77 GB         CA 3.88 GB         Max_CA 4 GB 
+[2025-06-30 06:37:58,609] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 137.55 GB, percent = 13.7%
+
Map (num_proc=8):  44%|████▍     | 2359/5391 [00:01<00:00, 3816.40 examples/s]
Map (num_proc=8):   0%|          | 21/5391 [00:00<02:50, 31.58 examples/s]
Map (num_proc=8):  56%|█████▌    | 2998/5391 [00:01<00:00, 4490.71 examples/s][2025-06-30 06:37:58,798] [INFO] [utils.py:781:see_memory_usage] Before creating fp16 partitions
+[2025-06-30 06:37:58,799] [INFO] [utils.py:782:see_memory_usage] MA 1.77 GB         Max_MA 1.77 GB         CA 3.88 GB         Max_CA 4 GB 
+[2025-06-30 06:37:58,800] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 138.21 GB, percent = 13.7%
+
Map (num_proc=8):   3%|▎         | 159/5391 [00:00<00:19, 264.88 examples/s]
Map (num_proc=8):  68%|██████▊   | 3640/5391 [00:01<00:00, 5014.39 examples/s]
Map (num_proc=8):   5%|▌         | 272/5391 [00:00<00:12, 394.74 examples/s]
Map (num_proc=8):  78%|███████▊  | 4190/5391 [00:01<00:00, 4484.16 examples/s]
Map (num_proc=8):  11%|█         | 598/5391 [00:01<00:04, 974.13 examples/s]
Map (num_proc=8):   0%|          | 0/5391 [00:00<?, ? examples/s]
Map (num_proc=8):  87%|████████▋ | 4712/5391 [00:01<00:00, 4665.79 examples/s]
Map (num_proc=8):  20%|██        | 1097/5391 [00:01<00:02, 1871.98 examples/s]
Map (num_proc=8):  31%|███▏      | 1690/5391 [00:01<00:01, 2845.58 examples/s]
Map (num_proc=8):  97%|█████████▋| 5234/5391 [00:01<00:00, 3769.29 examples/s]
Map (num_proc=8):  43%|████▎     | 2336/5391 [00:01<00:00, 3765.65 examples/s]
Map (num_proc=8):  56%|█████▌    | 3000/5391 [00:01<00:00, 4523.83 examples/s]
Map (num_proc=8): 100%|██████████| 5391/5391 [00:02<00:00, 2572.95 examples/s]
+
Map (num_proc=8):  68%|██████▊   | 3667/5391 [00:01<00:00, 5101.26 examples/s]
Filter:   0%|          | 0/5391 [00:00<?, ? examples/s]
Filter: 100%|██████████| 5391/5391 [00:00<00:00, 163800.36 examples/s]
+
Map (num_proc=8):  80%|████████  | 4331/5391 [00:01<00:00, 5458.94 examples/s][2025-06-30 06:37:59,705] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+
Map (num_proc=8):   0%|          | 20/5391 [00:00<02:45, 32.50 examples/s]
Map (num_proc=8):  92%|█████████▏| 4938/5391 [00:01<00:00, 5590.40 examples/s]
Map (num_proc=8):   4%|▎         | 197/5391 [00:00<00:15, 334.29 examples/s]
Map (num_proc=8):   8%|▊         | 444/5391 [00:00<00:06, 758.93 examples/s]
Map (num_proc=8):  15%|█▌        | 818/5391 [00:00<00:03, 1415.67 examples/s]
Map (num_proc=8): 100%|██████████| 5391/5391 [00:02<00:00, 2614.34 examples/s]
+
Map (num_proc=8):  24%|██▍       | 1296/5391 [00:01<00:01, 2212.71 examples/s]
Filter:   0%|          | 0/5391 [00:00<?, ? examples/s]
Filter: 100%|██████████| 5391/5391 [00:00<00:00, 159969.25 examples/s]
+
Map (num_proc=8):  35%|███▌      | 1894/5391 [00:01<00:01, 3160.98 examples/s][2025-06-30 06:38:00,286] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+
Map (num_proc=8):  47%|████▋     | 2548/5391 [00:01<00:00, 4041.82 examples/s]
Map (num_proc=8):  59%|█████▉    | 3205/5391 [00:01<00:00, 4712.01 examples/s]
Map (num_proc=8):  73%|███████▎  | 3930/5391 [00:01<00:00, 5342.26 examples/s]
Map (num_proc=8):  84%|████████▍ | 4523/5391 [00:01<00:00, 5224.73 examples/s]
Map (num_proc=8):  94%|█████████▍| 5092/5391 [00:01<00:00, 4798.10 examples/s]
Map (num_proc=8): 100%|██████████| 5391/5391 [00:02<00:00, 2687.47 examples/s]
+
Filter:   0%|          | 0/5391 [00:00<?, ? examples/s]
Filter: 100%|██████████| 5391/5391 [00:00<00:00, 158674.92 examples/s]
+[2025-06-30 06:38:01,297] [INFO] [config.py:733:__init__] Config mesh_device None world_size = 8
+[2025-06-30 06:38:02,923] [INFO] [utils.py:781:see_memory_usage] After creating fp16 partitions: 2
+[2025-06-30 06:38:02,924] [INFO] [utils.py:782:see_memory_usage] MA 1.77 GB         Max_MA 1.77 GB         CA 1.78 GB         Max_CA 4 GB 
+[2025-06-30 06:38:02,924] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 134.1 GB, percent = 13.3%
+[2025-06-30 06:38:03,108] [INFO] [utils.py:781:see_memory_usage] Before creating fp32 partitions
+[2025-06-30 06:38:03,109] [INFO] [utils.py:782:see_memory_usage] MA 1.77 GB         Max_MA 1.77 GB         CA 1.78 GB         Max_CA 2 GB 
+[2025-06-30 06:38:03,109] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 134.01 GB, percent = 13.3%
+[2025-06-30 06:38:03,577] [INFO] [utils.py:781:see_memory_usage] After creating fp32 partitions
+[2025-06-30 06:38:03,577] [INFO] [utils.py:782:see_memory_usage] MA 5.32 GB         Max_MA 7.09 GB         CA 7.1 GB         Max_CA 7 GB 
+[2025-06-30 06:38:03,578] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 134.09 GB, percent = 13.3%
+[2025-06-30 06:38:03,743] [INFO] [utils.py:781:see_memory_usage] Before initializing optimizer states
+[2025-06-30 06:38:03,744] [INFO] [utils.py:782:see_memory_usage] MA 5.32 GB         Max_MA 5.32 GB         CA 7.1 GB         Max_CA 7 GB 
+[2025-06-30 06:38:03,744] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 134.09 GB, percent = 13.3%
+[2025-06-30 06:38:03,913] [INFO] [utils.py:781:see_memory_usage] After initializing optimizer states
+[2025-06-30 06:38:03,914] [INFO] [utils.py:782:see_memory_usage] MA 5.32 GB         Max_MA 8.87 GB         CA 10.64 GB         Max_CA 11 GB 
+[2025-06-30 06:38:03,914] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 134.05 GB, percent = 13.3%
+[2025-06-30 06:38:03,915] [INFO] [stage3.py:529:_setup_for_real_optimizer] optimizer state initialized
+[2025-06-30 06:38:04,354] [INFO] [utils.py:781:see_memory_usage] After initializing ZeRO optimizer
+[2025-06-30 06:38:04,355] [INFO] [utils.py:782:see_memory_usage] MA 8.02 GB         Max_MA 10.06 GB         CA 11.66 GB         Max_CA 12 GB 
+[2025-06-30 06:38:04,356] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory:  used = 134.33 GB, percent = 13.3%
+[2025-06-30 06:38:04,356] [INFO] [logging.py:128:log_dist] [Rank 0] DeepSpeed Final Optimizer = DeepSpeedZeroOptimizer_Stage3
+[2025-06-30 06:38:04,356] [INFO] [logging.py:128:log_dist] [Rank 0] DeepSpeed using client LR scheduler
+[2025-06-30 06:38:04,356] [INFO] [logging.py:128:log_dist] [Rank 0] DeepSpeed LR Scheduler = <torch.optim.lr_scheduler.LambdaLR object at 0x7f11d0a63e50>
+[2025-06-30 06:38:04,356] [INFO] [logging.py:128:log_dist] [Rank 0] step=0, skipped=0, lr=[0.0, 0.0], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 06:38:04,357] [INFO] [config.py:999:print] DeepSpeedEngine configuration:
+[2025-06-30 06:38:04,358] [INFO] [config.py:1003:print]   activation_checkpointing_config  {
+    "partition_activations": false, 
+    "contiguous_memory_optimization": false, 
+    "cpu_checkpointing": false, 
+    "number_checkpoints": null, 
+    "synchronize_checkpoint_boundary": false, 
+    "profile": false
+}
+[2025-06-30 06:38:04,358] [INFO] [config.py:1003:print]   aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'thread_count': 1, 'single_submit': False, 'overlap_events': True, 'use_gds': False}
+[2025-06-30 06:38:04,358] [INFO] [config.py:1003:print]   amp_enabled .................. False
+[2025-06-30 06:38:04,358] [INFO] [config.py:1003:print]   amp_params ................... False
+[2025-06-30 06:38:04,358] [INFO] [config.py:1003:print]   autotuning_config ............ {
+    "enabled": false, 
+    "start_step": null, 
+    "end_step": null, 
+    "metric_path": null, 
+    "arg_mappings": null, 
+    "metric": "throughput", 
+    "model_info": null, 
+    "results_dir": "autotuning_results", 
+    "exps_dir": "autotuning_exps", 
+    "overwrite": true, 
+    "fast": true, 
+    "start_profile_step": 3, 
+    "end_profile_step": 5, 
+    "tuner_type": "gridsearch", 
+    "tuner_early_stopping": 5, 
+    "tuner_num_trials": 50, 
+    "model_info_path": null, 
+    "mp_size": 1, 
+    "max_train_batch_size": null, 
+    "min_train_batch_size": 1, 
+    "max_train_micro_batch_size_per_gpu": 1.024000e+03, 
+    "min_train_micro_batch_size_per_gpu": 1, 
+    "num_tuning_micro_batch_sizes": 3
+}
+[2025-06-30 06:38:04,358] [INFO] [config.py:1003:print]   bfloat16_enabled ............. True
+[2025-06-30 06:38:04,358] [INFO] [config.py:1003:print]   bfloat16_immediate_grad_update  False
+[2025-06-30 06:38:04,358] [INFO] [config.py:1003:print]   checkpoint_parallel_write_pipeline  False
+[2025-06-30 06:38:04,358] [INFO] [config.py:1003:print]   checkpoint_tag_validation_enabled  True
+[2025-06-30 06:38:04,358] [INFO] [config.py:1003:print]   checkpoint_tag_validation_fail  False
+[2025-06-30 06:38:04,358] [INFO] [config.py:1003:print]   comms_config ................. <deepspeed.comm.config.DeepSpeedCommsConfig object at 0x7f11d17d9d20>
+[2025-06-30 06:38:04,358] [INFO] [config.py:1003:print]   communication_data_type ...... None
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}}
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   curriculum_enabled_legacy .... False
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   curriculum_params_legacy ..... False
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'curriculum_learning': {'enabled': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}}
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   data_efficiency_enabled ...... False
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   dataloader_drop_last ......... False
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   disable_allgather ............ False
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   dump_state ................... False
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   dynamic_loss_scale_args ...... None
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   eigenvalue_enabled ........... False
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   eigenvalue_gas_boundary_resolution  1
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   eigenvalue_layer_name ........ bert.encoder.layer
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   eigenvalue_layer_num ......... 0
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   eigenvalue_max_iter .......... 100
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   eigenvalue_stability ......... 1e-06
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   eigenvalue_tol ............... 0.01
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   eigenvalue_verbose ........... False
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   elasticity_enabled ........... False
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   flops_profiler_config ........ {
+    "enabled": false, 
+    "recompute_fwd_factor": 0.0, 
+    "profile_step": 1, 
+    "module_depth": -1, 
+    "top_modules": 1, 
+    "detailed": true, 
+    "output_file": null
+}
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   fp16_auto_cast ............... None
+[2025-06-30 06:38:04,359] [INFO] [config.py:1003:print]   fp16_enabled ................. False
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   fp16_master_weights_and_gradients  False
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   global_rank .................. 0
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   grad_accum_dtype ............. None
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   gradient_accumulation_steps .. 4
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   gradient_clipping ............ 1.0
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   gradient_predivide_factor .... 1.0
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   graph_harvesting ............. False
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   hybrid_engine ................ enabled=False max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   initial_dynamic_scale ........ 1
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   load_universal_checkpoint .... False
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   loss_scale ................... 1.0
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   memory_breakdown ............. False
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   mics_hierarchial_params_gather  False
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   mics_shard_size .............. -1
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') comet=CometConfig(enabled=False, samples_log_interval=100, project=None, workspace=None, api_key=None, experiment_name=None, experiment_key=None, online=None, mode=None) wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName')
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   nebula_config ................ {
+    "enabled": false, 
+    "persistent_storage_path": null, 
+    "persistent_time_interval": 100, 
+    "num_of_version_in_retention": 2, 
+    "enable_nebula_load": true, 
+    "load_path": null
+}
+[2025-06-30 06:38:04,360] [INFO] [config.py:1003:print]   optimizer_legacy_fusion ...... False
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   optimizer_name ............... None
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   optimizer_params ............. None
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0, 'pipe_partitioned': True, 'grad_partitioned': True}
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   pld_enabled .................. False
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   pld_params ................... False
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   prescale_gradients ........... False
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   scheduler_name ............... None
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   scheduler_params ............. None
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   seq_parallel_communication_data_type  torch.float32
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   sparse_attention ............. None
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   sparse_gradients_enabled ..... False
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   steps_per_print .............. 100
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   timers_config ................ enabled=True synchronized=True
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   train_batch_size ............. 128
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   train_micro_batch_size_per_gpu  4
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   use_data_before_expert_parallel_  False
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   use_node_local_storage ....... False
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   wall_clock_breakdown ......... False
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   weight_quantization_config ... None
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   world_size ................... 8
+[2025-06-30 06:38:04,361] [INFO] [config.py:1003:print]   zero_allow_untested_optimizer  False
+[2025-06-30 06:38:04,362] [INFO] [config.py:1003:print]   zero_config .................. stage=3 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=500000000 use_multi_rank_bucket_allreduce=True allgather_partitions=True allgather_bucket_size=500000000 overlap_comm=True load_from_fp32_weights=True elastic_checkpoint=False offload_param=DeepSpeedZeroOffloadParamConfig(device='none', nvme_path=None, buffer_count=5, buffer_size=100000000, max_in_cpu=1000000000, pin_memory=False) offload_optimizer=DeepSpeedZeroOffloadOptimizerConfig(device='none', nvme_path=None, buffer_count=4, pin_memory=True, pipeline_read=False, pipeline_write=False, fast_init=False, ratio=1.0) sub_group_size=1000000000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=50000000 param_persistence_threshold=100000 model_persistence_threshold=9223372036854775807 max_live_parameters=1000000000 max_reuse_distance=1000000000 gather_16bit_weights_on_model_save=False module_granularity_threshold=0 use_all_reduce_for_fetch_params=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False zero_hpz_partition_size=1 zero_quantized_weights=False zero_quantized_nontrainable_weights=False zero_quantized_gradients=False zeropp_loco_param=None mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=True pipeline_loading_checkpoint=False override_module_apply=True
+[2025-06-30 06:38:04,362] [INFO] [config.py:1003:print]   zero_enabled ................. True
+[2025-06-30 06:38:04,362] [INFO] [config.py:1003:print]   zero_force_ds_cpu_optimizer .. True
+[2025-06-30 06:38:04,362] [INFO] [config.py:1003:print]   zero_optimization_stage ...... 3
+[2025-06-30 06:38:04,362] [INFO] [config.py:989:print_user_config]   json = {
+    "steps_per_print": 100, 
+    "zero_optimization": {
+        "stage": 3, 
+        "offload_param": {
+            "device": "none"
+        }, 
+        "offload_optimizer": {
+            "device": "none", 
+            "pin_memory": true
+        }, 
+        "sub_group_size": "auto", 
+        "stage3_max_live_parameters": "auto", 
+        "stage3_max_reuse_distance": "auto", 
+        "stage3_param_persistence_threshold": "auto", 
+        "stage3_prefetch_bucket_size": "auto", 
+        "reduce_bucket_size": "auto", 
+        "zero_hpz_partition_size": 1, 
+        "zero_quantized_weights": false, 
+        "zero_quantized_gradients": false
+    }, 
+    "bf16": {
+        "enabled": true
+    }, 
+    "gradient_clipping": 1.0, 
+    "prescale_gradients": false, 
+    "wall_clock_breakdown": false, 
+    "data_types": {
+        "grad_accum_dtype": null
+    }, 
+    "train_micro_batch_size_per_gpu": 4, 
+    "train_batch_size": 128
+}
+wandb: Currently logged in as: sarosavo to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
+wandb: Using wandb-core as the SDK backend.  Please refer to https://wandb.me/wandb-core for more information.
+wandb: - Waiting for wandb.init()...
wandb: \ Waiting for wandb.init()...
wandb: | Waiting for wandb.init()...
wandb: Tracking run with wandb version 0.19.6
+wandb: Run data is saved locally in /apdcephfs/share_300000800/user/yulaizhao/codes/rl_grounding/0304/make_reward_sft/wandb/run-20250630_063805-afb04fb4-d0ce-4a7e-9a3c-547fe30cc0d8
+wandb: Run `wandb offline` to turn off syncing.
+wandb: Syncing run RLVR_sft_augmented_data_180k_20250630-0636
+wandb: ⭐️ View project at https://wandb.ai/sarosavo/magic_spell
+wandb: 🚀 View run at https://wandb.ai/sarosavo/magic_spell/runs/afb04fb4-d0ce-4a7e-9a3c-547fe30cc0d8
+
Train epoch:   0%|          | 0/1 [00:00<?, ?it/s]
+
Train step of epoch 0:   0%|          | 0/5616 [00:00<?, ?it/s]TENCENT64:856599:861126 [2] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856599:861126 [2] NCCL INFO Using network IB
+TENCENT64:856598:861130 [1] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856601:861124 [4] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856598:861130 [1] NCCL INFO Using network IB
+TENCENT64:856600:861127 [3] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856601:861124 [4] NCCL INFO Using network IB
+TENCENT64:856604:861125 [7] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856603:861123 [6] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856602:861129 [5] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856603:861123 [6] NCCL INFO Using network IB
+TENCENT64:856600:861127 [3] NCCL INFO Using network IB
+TENCENT64:856602:861129 [5] NCCL INFO Using network IB
+TENCENT64:856604:861125 [7] NCCL INFO Using network IB
+TENCENT64:856597:861128 [0] NCCL INFO Using non-device net plugin version 0
+TENCENT64:856597:861128 [0] NCCL INFO Using network IB
+TENCENT64:856598:861130 [1] NCCL INFO bootstrapSplit: comm 0x7f625c066fc0 parent 0xac3d6d0 rank 1 nranks 8 color -934961569 key 1 prev 0 next 2 - DONE
+TENCENT64:856600:861127 [3] NCCL INFO bootstrapSplit: comm 0x7f18e8068360 parent 0xb785090 rank 3 nranks 8 color -934961569 key 3 prev 2 next 4 - DONE
+TENCENT64:856599:861126 [2] NCCL INFO bootstrapSplit: comm 0x7fcd04067390 parent 0xb7cd040 rank 2 nranks 8 color -934961569 key 2 prev 1 next 3 - DONE
+TENCENT64:856604:861125 [7] NCCL INFO bootstrapSplit: comm 0x7f7a18066fa0 parent 0xa8a21a0 rank 7 nranks 8 color -934961569 key 7 prev 6 next 0 - DONE
+TENCENT64:856598:861130 [1] NCCL INFO ncclCommSplit comm 0x7f625c066fc0 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 2b000 parent 0xac3d6d0 color -934961569 key 1 commId 0xde142fdc93a738ae - Init START
+TENCENT64:856601:861124 [4] NCCL INFO bootstrapSplit: comm 0x7fae74067050 parent 0xbc11380 rank 4 nranks 8 color -934961569 key 4 prev 3 next 5 - DONE
+TENCENT64:856602:861129 [5] NCCL INFO bootstrapSplit: comm 0x7f99a0067130 parent 0xaf5f740 rank 5 nranks 8 color -934961569 key 5 prev 4 next 6 - DONE
+TENCENT64:856600:861127 [3] NCCL INFO ncclCommSplit comm 0x7f18e8068360 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 69000 parent 0xb785090 color -934961569 key 3 commId 0xde142fdc93a738ae - Init START
+TENCENT64:856597:861128 [0] NCCL INFO bootstrapSplit: comm 0x7f0bec068160 parent 0xa71bd30 rank 0 nranks 8 color -934961569 key 0 prev 7 next 1 - DONE
+TENCENT64:856601:861124 [4] NCCL INFO ncclCommSplit comm 0x7fae74067050 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId a2000 parent 0xbc11380 color -934961569 key 4 commId 0xde142fdc93a738ae - Init START
+TENCENT64:856599:861126 [2] NCCL INFO ncclCommSplit comm 0x7fcd04067390 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 64000 parent 0xb7cd040 color -934961569 key 2 commId 0xde142fdc93a738ae - Init START
+TENCENT64:856604:861125 [7] NCCL INFO ncclCommSplit comm 0x7f7a18066fa0 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId e7000 parent 0xa8a21a0 color -934961569 key 7 commId 0xde142fdc93a738ae - Init START
+TENCENT64:856603:861123 [6] NCCL INFO bootstrapSplit: comm 0x7f66fc066f50 parent 0xada66d0 rank 6 nranks 8 color -934961569 key 6 prev 5 next 7 - DONE
+TENCENT64:856602:861129 [5] NCCL INFO ncclCommSplit comm 0x7f99a0067130 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId a7000 parent 0xaf5f740 color -934961569 key 5 commId 0xde142fdc93a738ae - Init START
+TENCENT64:856597:861128 [0] NCCL INFO ncclCommSplit comm 0x7f0bec068160 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 25000 parent 0xa71bd30 color -934961569 key 0 commId 0xde142fdc93a738ae - Init START
+TENCENT64:856603:861123 [6] NCCL INFO ncclCommSplit comm 0x7f66fc066f50 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId e1000 parent 0xada66d0 color -934961569 key 6 commId 0xde142fdc93a738ae - Init START
+TENCENT64:856601:861124 [4] NCCL INFO Setting affinity for GPU 4 to ff,00000000,00000000,00000000,000000ff,00000000,00000000,00000000
+TENCENT64:856601:861124 [4] NCCL INFO NVLS multicast support is not available on dev 4
+TENCENT64:856597:861128 [0] NCCL INFO Setting affinity for GPU 0 to ff,00000000,00000000,00000000,000000ff,00000000
+TENCENT64:856597:861128 [0] NCCL INFO NVLS multicast support is not available on dev 0
+TENCENT64:856604:861125 [7] NCCL INFO Setting affinity for GPU 7 to ff,00000000,00000000,00000000,000000ff,00000000,00000000
+TENCENT64:856604:861125 [7] NCCL INFO NVLS multicast support is not available on dev 7
+TENCENT64:856602:861129 [5] NCCL INFO Setting affinity for GPU 5 to ff,00000000,00000000,00000000,000000ff,00000000,00000000,00000000
+TENCENT64:856602:861129 [5] NCCL INFO NVLS multicast support is not available on dev 5
+TENCENT64:856603:861123 [6] NCCL INFO Setting affinity for GPU 6 to ff,00000000,00000000,00000000,000000ff,00000000,00000000
+TENCENT64:856603:861123 [6] NCCL INFO NVLS multicast support is not available on dev 6
+TENCENT64:856600:861127 [3] NCCL INFO Setting affinity for GPU 3 to ff,00000000,00000000,00000000,000000ff
+TENCENT64:856600:861127 [3] NCCL INFO NVLS multicast support is not available on dev 3
+TENCENT64:856599:861126 [2] NCCL INFO Setting affinity for GPU 2 to ff,00000000,00000000,00000000,000000ff
+TENCENT64:856598:861130 [1] NCCL INFO Setting affinity for GPU 1 to ff,00000000,00000000,00000000,000000ff,00000000
+TENCENT64:856598:861130 [1] NCCL INFO NVLS multicast support is not available on dev 1
+TENCENT64:856599:861126 [2] NCCL INFO NVLS multicast support is not available on dev 2
+TENCENT64:856603:861123 [6] NCCL INFO comm 0x7f66fc066f50 rank 6 nRanks 8 nNodes 1 localRanks 8 localRank 6 MNNVL 0
+TENCENT64:856602:861129 [5] NCCL INFO comm 0x7f99a0067130 rank 5 nRanks 8 nNodes 1 localRanks 8 localRank 5 MNNVL 0
+TENCENT64:856604:861125 [7] NCCL INFO comm 0x7f7a18066fa0 rank 7 nRanks 8 nNodes 1 localRanks 8 localRank 7 MNNVL 0
+TENCENT64:856598:861130 [1] NCCL INFO comm 0x7f625c066fc0 rank 1 nRanks 8 nNodes 1 localRanks 8 localRank 1 MNNVL 0
+TENCENT64:856602:861129 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4
+TENCENT64:856604:861125 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6
+TENCENT64:856599:861126 [2] NCCL INFO comm 0x7fcd04067390 rank 2 nRanks 8 nNodes 1 localRanks 8 localRank 2 MNNVL 0
+TENCENT64:856602:861129 [5] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856601:861124 [4] NCCL INFO comm 0x7fae74067050 rank 4 nRanks 8 nNodes 1 localRanks 8 localRank 4 MNNVL 0
+TENCENT64:856600:861127 [3] NCCL INFO comm 0x7f18e8068360 rank 3 nRanks 8 nNodes 1 localRanks 8 localRank 3 MNNVL 0
+TENCENT64:856597:861128 [0] NCCL INFO comm 0x7f0bec068160 rank 0 nRanks 8 nNodes 1 localRanks 8 localRank 0 MNNVL 0
+TENCENT64:856604:861125 [7] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856603:861123 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5
+TENCENT64:856603:861123 [6] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856597:861128 [0] NCCL INFO Channel 00/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856601:861124 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3
+TENCENT64:856599:861126 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1
+TENCENT64:856597:861128 [0] NCCL INFO Channel 01/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856601:861124 [4] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856598:861130 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0
+TENCENT64:856599:861126 [2] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856600:861127 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2
+TENCENT64:856597:861128 [0] NCCL INFO Channel 02/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856598:861130 [1] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856600:861127 [3] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856597:861128 [0] NCCL INFO Channel 03/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:861128 [0] NCCL INFO Channel 04/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:861128 [0] NCCL INFO Channel 05/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:861128 [0] NCCL INFO Channel 06/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:861128 [0] NCCL INFO Channel 07/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:861128 [0] NCCL INFO Channel 08/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:861128 [0] NCCL INFO Channel 09/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:861128 [0] NCCL INFO Channel 10/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:861128 [0] NCCL INFO Channel 11/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:861128 [0] NCCL INFO Channel 12/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:861128 [0] NCCL INFO Channel 13/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:861128 [0] NCCL INFO Channel 14/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:861128 [0] NCCL INFO Channel 15/16 :    0   1   2   3   4   5   6   7
+TENCENT64:856597:861128 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1
+TENCENT64:856597:861128 [0] NCCL INFO P2P Chunksize set to 524288
+TENCENT64:856602:861129 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 00/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 00/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 01/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 01/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 02/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 02/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 03/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 03/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 04/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 05/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 06/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 07/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 08/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 09/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 04/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 10/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 05/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 11/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 06/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 12/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 07/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 13/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 08/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 14/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 09/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 15/0 : 6[6] -> 7[7] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 10/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 11/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 12/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 13/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 14/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 15/0 : 7[7] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Connected all rings
+TENCENT64:856598:861130 [1] NCCL INFO Connected all rings
+TENCENT64:856597:861128 [0] NCCL INFO Connected all rings
+TENCENT64:856599:861126 [2] NCCL INFO Channel 00/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Connected all rings
+TENCENT64:856598:861130 [1] NCCL INFO Channel 00/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 01/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Connected all rings
+TENCENT64:856598:861130 [1] NCCL INFO Channel 01/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 02/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 03/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 02/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 04/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 03/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 05/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 04/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 06/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 05/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 07/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 06/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 08/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 07/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 08/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 09/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 10/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Connected all rings
+TENCENT64:856604:861125 [7] NCCL INFO Channel 00/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 09/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Connected all rings
+TENCENT64:856603:861123 [6] NCCL INFO Connected all rings
+TENCENT64:856599:861126 [2] NCCL INFO Channel 11/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 01/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 10/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 12/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 02/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 11/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 13/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 03/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 12/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 04/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 13/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 05/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 14/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 14/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 06/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856599:861126 [2] NCCL INFO Channel 15/0 : 2[2] -> 1[1] via P2P/CUMEM/read
+TENCENT64:856598:861130 [1] NCCL INFO Channel 15/0 : 1[1] -> 0[0] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 07/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 08/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 00/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 09/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 01/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 10/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 00/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 02/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 01/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 03/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 02/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 04/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 03/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 05/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 04/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 11/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 06/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 00/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 05/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 12/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 01/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 06/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 13/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 02/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 07/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 14/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 00/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 03/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 08/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856604:861125 [7] NCCL INFO Channel 15/0 : 7[7] -> 6[6] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 01/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 04/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 09/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 02/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 05/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 10/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 07/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 03/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 06/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 11/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 08/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 04/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 09/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 07/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 12/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 05/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 10/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 08/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 13/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 06/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 11/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 09/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 14/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 07/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 12/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856601:861124 [4] NCCL INFO Channel 15/0 : 4[4] -> 3[3] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 08/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 13/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 09/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 14/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856600:861127 [3] NCCL INFO Channel 15/0 : 3[3] -> 2[2] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 10/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 10/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 11/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 11/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 12/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 12/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 13/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 13/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 14/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856602:861129 [5] NCCL INFO Channel 15/0 : 5[5] -> 4[4] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 14/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856603:861123 [6] NCCL INFO Channel 15/0 : 6[6] -> 5[5] via P2P/CUMEM/read
+TENCENT64:856597:861128 [0] NCCL INFO Connected all trees
+TENCENT64:856597:861128 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856597:861128 [0] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856598:861130 [1] NCCL INFO Connected all trees
+TENCENT64:856598:861130 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856598:861130 [1] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856599:861126 [2] NCCL INFO Connected all trees
+TENCENT64:856599:861126 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856599:861126 [2] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856600:861127 [3] NCCL INFO Connected all trees
+TENCENT64:856600:861127 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856600:861127 [3] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856604:861125 [7] NCCL INFO Connected all trees
+TENCENT64:856604:861125 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856604:861125 [7] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856601:861124 [4] NCCL INFO Connected all trees
+TENCENT64:856601:861124 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856601:861124 [4] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856603:861123 [6] NCCL INFO Connected all trees
+TENCENT64:856602:861129 [5] NCCL INFO Connected all trees
+TENCENT64:856603:861123 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856603:861123 [6] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856602:861129 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
+TENCENT64:856602:861129 [5] NCCL INFO 16 coll channels, 16 collnet channels, 0 nvls channels, 16 p2p channels, 16 p2p channels per peer
+TENCENT64:856603:861123 [6] NCCL INFO ncclCommSplit comm 0x7f66fc066f50 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId e1000 parent 0xada66d0 color -934961569 key 6 commId 0xde142fdc93a738ae - Init COMPLETE
+TENCENT64:856597:861128 [0] NCCL INFO ncclCommSplit comm 0x7f0bec068160 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 25000 parent 0xa71bd30 color -934961569 key 0 commId 0xde142fdc93a738ae - Init COMPLETE
+TENCENT64:856601:861124 [4] NCCL INFO ncclCommSplit comm 0x7fae74067050 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId a2000 parent 0xbc11380 color -934961569 key 4 commId 0xde142fdc93a738ae - Init COMPLETE
+TENCENT64:856602:861129 [5] NCCL INFO ncclCommSplit comm 0x7f99a0067130 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId a7000 parent 0xaf5f740 color -934961569 key 5 commId 0xde142fdc93a738ae - Init COMPLETE
+TENCENT64:856598:861130 [1] NCCL INFO ncclCommSplit comm 0x7f625c066fc0 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 2b000 parent 0xac3d6d0 color -934961569 key 1 commId 0xde142fdc93a738ae - Init COMPLETE
+TENCENT64:856600:861127 [3] NCCL INFO ncclCommSplit comm 0x7f18e8068360 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 69000 parent 0xb785090 color -934961569 key 3 commId 0xde142fdc93a738ae - Init COMPLETE
+TENCENT64:856599:861126 [2] NCCL INFO ncclCommSplit comm 0x7fcd04067390 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 64000 parent 0xb7cd040 color -934961569 key 2 commId 0xde142fdc93a738ae - Init COMPLETE
+TENCENT64:856604:861125 [7] NCCL INFO ncclCommSplit comm 0x7f7a18066fa0 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId e7000 parent 0xa8a21a0 color -934961569 key 7 commId 0xde142fdc93a738ae - Init COMPLETE
+
+
Train step of epoch 0:   0%|          | 0/5616 [00:03<?, ?it/s, gpt_loss=0.121, lr=0]
+
Train step of epoch 0:   0%|          | 1/5616 [00:03<4:43:24,  3.03s/it, gpt_loss=0.121, lr=0]
+
Train step of epoch 0:   0%|          | 1/5616 [00:03<4:43:24,  3.03s/it, gpt_loss=1.12, lr=0] 
+
Train step of epoch 0:   0%|          | 2/5616 [00:03<2:49:45,  1.81s/it, gpt_loss=1.12, lr=0]
+
Train step of epoch 0:   0%|          | 2/5616 [00:04<2:49:45,  1.81s/it, gpt_loss=0.466, lr=0]
+
Train step of epoch 0:   0%|          | 3/5616 [00:04<2:08:27,  1.37s/it, gpt_loss=0.466, lr=0]
+
Train step of epoch 0:   0%|          | 3/5616 [00:05<2:08:27,  1.37s/it, gpt_loss=0.104, lr=1.16e-7]
+
Train step of epoch 0:   0%|          | 4/5616 [00:05<1:54:11,  1.22s/it, gpt_loss=0.104, lr=1.16e-7]
+
Train step of epoch 0:   0%|          | 4/5616 [00:06<1:54:11,  1.22s/it, gpt_loss=0.412, lr=1.16e-7]
+
Train step of epoch 0:   0%|          | 5/5616 [00:06<1:35:32,  1.02s/it, gpt_loss=0.412, lr=1.16e-7]
+
Train step of epoch 0:   0%|          | 5/5616 [00:07<1:35:32,  1.02s/it, gpt_loss=0.634, lr=1.16e-7]
+
Train step of epoch 0:   0%|          | 6/5616 [00:07<1:33:01,  1.01it/s, gpt_loss=0.634, lr=1.16e-7]
+
Train step of epoch 0:   0%|          | 6/5616 [00:08<1:33:01,  1.01it/s, gpt_loss=0.557, lr=1.16e-7]
+
Train step of epoch 0:   0%|          | 7/5616 [00:08<1:23:49,  1.12it/s, gpt_loss=0.557, lr=1.16e-7]
+
Train step of epoch 0:   0%|          | 7/5616 [00:08<1:23:49,  1.12it/s, gpt_loss=0.148, lr=2.33e-7]
+
Train step of epoch 0:   0%|          | 8/5616 [00:08<1:21:21,  1.15it/s, gpt_loss=0.148, lr=2.33e-7]
+
Train step of epoch 0:   0%|          | 8/5616 [00:09<1:21:21,  1.15it/s, gpt_loss=0.513, lr=2.33e-7]
+
Train step of epoch 0:   0%|          | 9/5616 [00:09<1:19:06,  1.18it/s, gpt_loss=0.513, lr=2.33e-7]
+
Train step of epoch 0:   0%|          | 9/5616 [00:10<1:19:06,  1.18it/s, gpt_loss=0.388, lr=2.33e-7]
+
Train step of epoch 0:   0%|          | 10/5616 [00:10<1:16:38,  1.22it/s, gpt_loss=0.388, lr=2.33e-7]
+
Train step of epoch 0:   0%|          | 10/5616 [00:11<1:16:38,  1.22it/s, gpt_loss=0.384, lr=2.33e-7]
+
Train step of epoch 0:   0%|          | 11/5616 [00:11<1:15:48,  1.23it/s, gpt_loss=0.384, lr=2.33e-7]
+
Train step of epoch 0:   0%|          | 11/5616 [00:12<1:15:48,  1.23it/s, gpt_loss=0.591, lr=3.49e-7]
+
Train step of epoch 0:   0%|          | 12/5616 [00:12<1:18:44,  1.19it/s, gpt_loss=0.591, lr=3.49e-7]
+
Train step of epoch 0:   0%|          | 12/5616 [00:13<1:18:44,  1.19it/s, gpt_loss=0.85, lr=3.49e-7] 
+
Train step of epoch 0:   0%|          | 13/5616 [00:13<1:21:58,  1.14it/s, gpt_loss=0.85, lr=3.49e-7]
+
Train step of epoch 0:   0%|          | 13/5616 [00:13<1:21:58,  1.14it/s, gpt_loss=1.05, lr=3.49e-7]
+
Train step of epoch 0:   0%|          | 14/5616 [00:13<1:20:30,  1.16it/s, gpt_loss=1.05, lr=3.49e-7]
+
Train step of epoch 0:   0%|          | 14/5616 [00:14<1:20:30,  1.16it/s, gpt_loss=0.768, lr=3.49e-7]
+
Train step of epoch 0:   0%|          | 15/5616 [00:14<1:14:59,  1.24it/s, gpt_loss=0.768, lr=3.49e-7]
+
Train step of epoch 0:   0%|          | 15/5616 [00:15<1:14:59,  1.24it/s, gpt_loss=0.684, lr=4.65e-7]
+
Train step of epoch 0:   0%|          | 16/5616 [00:15<1:14:26,  1.25it/s, gpt_loss=0.684, lr=4.65e-7]
+
Train step of epoch 0:   0%|          | 16/5616 [00:16<1:14:26,  1.25it/s, gpt_loss=0.598, lr=4.65e-7]
+
Train step of epoch 0:   0%|          | 17/5616 [00:16<1:12:35,  1.29it/s, gpt_loss=0.598, lr=4.65e-7]
+
Train step of epoch 0:   0%|          | 17/5616 [00:17<1:12:35,  1.29it/s, gpt_loss=0.318, lr=4.65e-7]
+
Train step of epoch 0:   0%|          | 18/5616 [00:17<1:15:57,  1.23it/s, gpt_loss=0.318, lr=4.65e-7]
+
Train step of epoch 0:   0%|          | 18/5616 [00:17<1:15:57,  1.23it/s, gpt_loss=0.672, lr=4.65e-7]
+
Train step of epoch 0:   0%|          | 19/5616 [00:17<1:14:11,  1.26it/s, gpt_loss=0.672, lr=4.65e-7]
+
Train step of epoch 0:   0%|          | 19/5616 [00:18<1:14:11,  1.26it/s, gpt_loss=0.413, lr=5.81e-7]
+
Train step of epoch 0:   0%|          | 20/5616 [00:18<1:14:41,  1.25it/s, gpt_loss=0.413, lr=5.81e-7]
+
Train step of epoch 0:   0%|          | 20/5616 [00:19<1:14:41,  1.25it/s, gpt_loss=1.01, lr=5.81e-7] 
+
Train step of epoch 0:   0%|          | 21/5616 [00:19<1:14:43,  1.25it/s, gpt_loss=1.01, lr=5.81e-7]
+
Train step of epoch 0:   0%|          | 21/5616 [00:20<1:14:43,  1.25it/s, gpt_loss=1.02, lr=5.81e-7]
+
Train step of epoch 0:   0%|          | 22/5616 [00:20<1:12:24,  1.29it/s, gpt_loss=1.02, lr=5.81e-7]
+
Train step of epoch 0:   0%|          | 22/5616 [00:20<1:12:24,  1.29it/s, gpt_loss=0.284, lr=5.81e-7]
+
Train step of epoch 0:   0%|          | 23/5616 [00:20<1:08:51,  1.35it/s, gpt_loss=0.284, lr=5.81e-7]
+
Train step of epoch 0:   0%|          | 23/5616 [00:21<1:08:51,  1.35it/s, gpt_loss=0.345, lr=6.98e-7]
+
Train step of epoch 0:   0%|          | 24/5616 [00:21<1:19:02,  1.18it/s, gpt_loss=0.345, lr=6.98e-7]
+
Train step of epoch 0:   0%|          | 24/5616 [00:22<1:19:02,  1.18it/s, gpt_loss=0.327, lr=6.98e-7]
+
Train step of epoch 0:   0%|          | 25/5616 [00:22<1:18:12,  1.19it/s, gpt_loss=0.327, lr=6.98e-7]
+
Train step of epoch 0:   0%|          | 25/5616 [00:23<1:18:12,  1.19it/s, gpt_loss=0.398, lr=6.98e-7]
+
Train step of epoch 0:   0%|          | 26/5616 [00:23<1:15:25,  1.24it/s, gpt_loss=0.398, lr=6.98e-7]
+
Train step of epoch 0:   0%|          | 26/5616 [00:24<1:15:25,  1.24it/s, gpt_loss=1.13, lr=6.98e-7] 
+
Train step of epoch 0:   0%|          | 27/5616 [00:24<1:10:31,  1.32it/s, gpt_loss=1.13, lr=6.98e-7]
+
Train step of epoch 0:   0%|          | 27/5616 [00:25<1:10:31,  1.32it/s, gpt_loss=0.4, lr=8.14e-7] 
+
Train step of epoch 0:   0%|          | 28/5616 [00:25<1:15:35,  1.23it/s, gpt_loss=0.4, lr=8.14e-7]
+
Train step of epoch 0:   0%|          | 28/5616 [00:25<1:15:35,  1.23it/s, gpt_loss=0.571, lr=8.14e-7]
+
Train step of epoch 0:   1%|          | 29/5616 [00:25<1:11:27,  1.30it/s, gpt_loss=0.571, lr=8.14e-7]
+
Train step of epoch 0:   1%|          | 29/5616 [00:26<1:11:27,  1.30it/s, gpt_loss=0.154, lr=8.14e-7]
+
Train step of epoch 0:   1%|          | 30/5616 [00:26<1:10:05,  1.33it/s, gpt_loss=0.154, lr=8.14e-7]
+
Train step of epoch 0:   1%|          | 30/5616 [00:27<1:10:05,  1.33it/s, gpt_loss=1.05, lr=8.14e-7] 
+
Train step of epoch 0:   1%|          | 31/5616 [00:27<1:10:24,  1.32it/s, gpt_loss=1.05, lr=8.14e-7]
+
Train step of epoch 0:   1%|          | 31/5616 [00:27<1:10:24,  1.32it/s, gpt_loss=0.602, lr=9.3e-7]
+
Train step of epoch 0:   1%|          | 32/5616 [00:27<1:10:07,  1.33it/s, gpt_loss=0.602, lr=9.3e-7]
+
Train step of epoch 0:   1%|          | 32/5616 [00:28<1:10:07,  1.33it/s, gpt_loss=0.401, lr=9.3e-7]
+
Train step of epoch 0:   1%|          | 33/5616 [00:28<1:09:20,  1.34it/s, gpt_loss=0.401, lr=9.3e-7]
+
Train step of epoch 0:   1%|          | 33/5616 [00:29<1:09:20,  1.34it/s, gpt_loss=0.651, lr=9.3e-7]
+
Train step of epoch 0:   1%|          | 34/5616 [00:29<1:09:41,  1.34it/s, gpt_loss=0.651, lr=9.3e-7]
+
Train step of epoch 0:   1%|          | 34/5616 [00:30<1:09:41,  1.34it/s, gpt_loss=0.217, lr=9.3e-7]
+
Train step of epoch 0:   1%|          | 35/5616 [00:30<1:07:52,  1.37it/s, gpt_loss=0.217, lr=9.3e-7]
+
Train step of epoch 0:   1%|          | 35/5616 [00:30<1:07:52,  1.37it/s, gpt_loss=0.98, lr=1.05e-6]
+
Train step of epoch 0:   1%|          | 36/5616 [00:30<1:10:33,  1.32it/s, gpt_loss=0.98, lr=1.05e-6]
+
Train step of epoch 0:   1%|          | 36/5616 [00:31<1:10:33,  1.32it/s, gpt_loss=0.511, lr=1.05e-6]
+
Train step of epoch 0:   1%|          | 37/5616 [00:31<1:14:31,  1.25it/s, gpt_loss=0.511, lr=1.05e-6]
+
Train step of epoch 0:   1%|          | 37/5616 [00:32<1:14:31,  1.25it/s, gpt_loss=0.224, lr=1.05e-6]
+
Train step of epoch 0:   1%|          | 38/5616 [00:32<1:12:42,  1.28it/s, gpt_loss=0.224, lr=1.05e-6]
+
Train step of epoch 0:   1%|          | 38/5616 [00:33<1:12:42,  1.28it/s, gpt_loss=0.436, lr=1.05e-6]
+
Train step of epoch 0:   1%|          | 39/5616 [00:33<1:19:22,  1.17it/s, gpt_loss=0.436, lr=1.05e-6]
+
Train step of epoch 0:   1%|          | 39/5616 [00:34<1:19:22,  1.17it/s, gpt_loss=0.525, lr=1.16e-6]
+
Train step of epoch 0:   1%|          | 40/5616 [00:34<1:19:08,  1.17it/s, gpt_loss=0.525, lr=1.16e-6]
+
Train step of epoch 0:   1%|          | 40/5616 [00:35<1:19:08,  1.17it/s, gpt_loss=0.14, lr=1.16e-6] 
+
Train step of epoch 0:   1%|          | 41/5616 [00:35<1:14:34,  1.25it/s, gpt_loss=0.14, lr=1.16e-6]
+
Train step of epoch 0:   1%|          | 41/5616 [00:35<1:14:34,  1.25it/s, gpt_loss=0.264, lr=1.16e-6]
+
Train step of epoch 0:   1%|          | 42/5616 [00:35<1:12:37,  1.28it/s, gpt_loss=0.264, lr=1.16e-6]
+
Train step of epoch 0:   1%|          | 42/5616 [00:36<1:12:37,  1.28it/s, gpt_loss=0.27, lr=1.16e-6] 
+
Train step of epoch 0:   1%|          | 43/5616 [00:36<1:09:38,  1.33it/s, gpt_loss=0.27, lr=1.16e-6]
+
Train step of epoch 0:   1%|          | 43/5616 [00:37<1:09:38,  1.33it/s, gpt_loss=0.185, lr=1.28e-6]
+
Train step of epoch 0:   1%|          | 44/5616 [00:37<1:11:41,  1.30it/s, gpt_loss=0.185, lr=1.28e-6]
+
Train step of epoch 0:   1%|          | 44/5616 [00:38<1:11:41,  1.30it/s, gpt_loss=0.0749, lr=1.28e-6]
+
Train step of epoch 0:   1%|          | 45/5616 [00:38<1:11:21,  1.30it/s, gpt_loss=0.0749, lr=1.28e-6]
+
Train step of epoch 0:   1%|          | 45/5616 [00:38<1:11:21,  1.30it/s, gpt_loss=0.157, lr=1.28e-6] 
+
Train step of epoch 0:   1%|          | 46/5616 [00:38<1:09:55,  1.33it/s, gpt_loss=0.157, lr=1.28e-6]
+
Train step of epoch 0:   1%|          | 46/5616 [00:39<1:09:55,  1.33it/s, gpt_loss=0.338, lr=1.28e-6]
+
Train step of epoch 0:   1%|          | 47/5616 [00:39<1:19:35,  1.17it/s, gpt_loss=0.338, lr=1.28e-6]
+
Train step of epoch 0:   1%|          | 47/5616 [00:40<1:19:35,  1.17it/s, gpt_loss=0.278, lr=1.4e-6] 
+
Train step of epoch 0:   1%|          | 48/5616 [00:40<1:16:37,  1.21it/s, gpt_loss=0.278, lr=1.4e-6]
+
Train step of epoch 0:   1%|          | 48/5616 [00:41<1:16:37,  1.21it/s, gpt_loss=0.317, lr=1.4e-6]
+
Train step of epoch 0:   1%|          | 49/5616 [00:41<1:15:37,  1.23it/s, gpt_loss=0.317, lr=1.4e-6]
+
Train step of epoch 0:   1%|          | 49/5616 [00:42<1:15:37,  1.23it/s, gpt_loss=0.249, lr=1.4e-6]
+
Train step of epoch 0:   1%|          | 50/5616 [00:42<1:11:37,  1.30it/s, gpt_loss=0.249, lr=1.4e-6]
+
Train step of epoch 0:   1%|          | 50/5616 [00:42<1:11:37,  1.30it/s, gpt_loss=0.0706, lr=1.4e-6]
+
Train step of epoch 0:   1%|          | 51/5616 [00:42<1:09:15,  1.34it/s, gpt_loss=0.0706, lr=1.4e-6]
+
Train step of epoch 0:   1%|          | 51/5616 [00:43<1:09:15,  1.34it/s, gpt_loss=0.122, lr=1.51e-6]
+
Train step of epoch 0:   1%|          | 52/5616 [00:43<1:10:03,  1.32it/s, gpt_loss=0.122, lr=1.51e-6]
+
Train step of epoch 0:   1%|          | 52/5616 [00:44<1:10:03,  1.32it/s, gpt_loss=0.144, lr=1.51e-6]
+
Train step of epoch 0:   1%|          | 53/5616 [00:44<1:15:30,  1.23it/s, gpt_loss=0.144, lr=1.51e-6]
+
Train step of epoch 0:   1%|          | 53/5616 [00:45<1:15:30,  1.23it/s, gpt_loss=0.13, lr=1.51e-6] 
+
Train step of epoch 0:   1%|          | 54/5616 [00:45<1:13:38,  1.26it/s, gpt_loss=0.13, lr=1.51e-6]
+
Train step of epoch 0:   1%|          | 54/5616 [00:46<1:13:38,  1.26it/s, gpt_loss=0.102, lr=1.51e-6]
+
Train step of epoch 0:   1%|          | 55/5616 [00:46<1:14:54,  1.24it/s, gpt_loss=0.102, lr=1.51e-6]
+
Train step of epoch 0:   1%|          | 55/5616 [00:46<1:14:54,  1.24it/s, gpt_loss=0.139, lr=1.63e-6]
+
Train step of epoch 0:   1%|          | 56/5616 [00:46<1:15:26,  1.23it/s, gpt_loss=0.139, lr=1.63e-6]
+
Train step of epoch 0:   1%|          | 56/5616 [00:47<1:15:26,  1.23it/s, gpt_loss=0.0711, lr=1.63e-6]
+
Train step of epoch 0:   1%|          | 57/5616 [00:47<1:12:52,  1.27it/s, gpt_loss=0.0711, lr=1.63e-6]
+
Train step of epoch 0:   1%|          | 57/5616 [00:48<1:12:52,  1.27it/s, gpt_loss=0.128, lr=1.63e-6] 
+
Train step of epoch 0:   1%|          | 58/5616 [00:48<1:09:27,  1.33it/s, gpt_loss=0.128, lr=1.63e-6]
+
Train step of epoch 0:   1%|          | 58/5616 [00:49<1:09:27,  1.33it/s, gpt_loss=0.129, lr=1.63e-6]
+
Train step of epoch 0:   1%|          | 59/5616 [00:49<1:07:15,  1.38it/s, gpt_loss=0.129, lr=1.63e-6]
+
Train step of epoch 0:   1%|          | 59/5616 [00:49<1:07:15,  1.38it/s, gpt_loss=0.12, lr=1.74e-6] 
+
Train step of epoch 0:   1%|          | 60/5616 [00:49<1:09:21,  1.33it/s, gpt_loss=0.12, lr=1.74e-6]
+
Train step of epoch 0:   1%|          | 60/5616 [00:50<1:09:21,  1.33it/s, gpt_loss=0.17, lr=1.74e-6]
+
Train step of epoch 0:   1%|          | 61/5616 [00:50<1:07:09,  1.38it/s, gpt_loss=0.17, lr=1.74e-6]
+
Train step of epoch 0:   1%|          | 61/5616 [00:51<1:07:09,  1.38it/s, gpt_loss=0.172, lr=1.74e-6]
+
Train step of epoch 0:   1%|          | 62/5616 [00:51<1:06:17,  1.40it/s, gpt_loss=0.172, lr=1.74e-6]
+
Train step of epoch 0:   1%|          | 62/5616 [00:52<1:06:17,  1.40it/s, gpt_loss=0.151, lr=1.74e-6]
+
Train step of epoch 0:   1%|          | 63/5616 [00:52<1:09:10,  1.34it/s, gpt_loss=0.151, lr=1.74e-6]
+
Train step of epoch 0:   1%|          | 63/5616 [00:52<1:09:10,  1.34it/s, gpt_loss=0.219, lr=1.86e-6]
+
Train step of epoch 0:   1%|          | 64/5616 [00:52<1:08:04,  1.36it/s, gpt_loss=0.219, lr=1.86e-6]
+
Train step of epoch 0:   1%|          | 64/5616 [00:53<1:08:04,  1.36it/s, gpt_loss=0.138, lr=1.86e-6]
+
Train step of epoch 0:   1%|          | 65/5616 [00:53<1:05:48,  1.41it/s, gpt_loss=0.138, lr=1.86e-6]
+
Train step of epoch 0:   1%|          | 65/5616 [00:54<1:05:48,  1.41it/s, gpt_loss=0.153, lr=1.86e-6]
+
Train step of epoch 0:   1%|          | 66/5616 [00:54<1:06:46,  1.39it/s, gpt_loss=0.153, lr=1.86e-6]
+
Train step of epoch 0:   1%|          | 66/5616 [00:54<1:06:46,  1.39it/s, gpt_loss=0.122, lr=1.86e-6]
+
Train step of epoch 0:   1%|          | 67/5616 [00:54<1:04:47,  1.43it/s, gpt_loss=0.122, lr=1.86e-6]
+
Train step of epoch 0:   1%|          | 67/5616 [00:55<1:04:47,  1.43it/s, gpt_loss=0.122, lr=1.98e-6]
+
Train step of epoch 0:   1%|          | 68/5616 [00:55<1:06:27,  1.39it/s, gpt_loss=0.122, lr=1.98e-6]
+
Train step of epoch 0:   1%|          | 68/5616 [00:56<1:06:27,  1.39it/s, gpt_loss=0.118, lr=1.98e-6]
+
Train step of epoch 0:   1%|          | 69/5616 [00:56<1:08:29,  1.35it/s, gpt_loss=0.118, lr=1.98e-6]
+
Train step of epoch 0:   1%|          | 69/5616 [00:57<1:08:29,  1.35it/s, gpt_loss=0.136, lr=1.98e-6]
+
Train step of epoch 0:   1%|          | 70/5616 [00:57<1:10:41,  1.31it/s, gpt_loss=0.136, lr=1.98e-6]
+
Train step of epoch 0:   1%|          | 70/5616 [00:58<1:10:41,  1.31it/s, gpt_loss=0.116, lr=1.98e-6]
+
Train step of epoch 0:   1%|▏         | 71/5616 [00:58<1:15:42,  1.22it/s, gpt_loss=0.116, lr=1.98e-6]
+
Train step of epoch 0:   1%|▏         | 71/5616 [00:58<1:15:42,  1.22it/s, gpt_loss=0.16, lr=2.09e-6] 
+
Train step of epoch 0:   1%|▏         | 72/5616 [00:58<1:14:37,  1.24it/s, gpt_loss=0.16, lr=2.09e-6]
+
Train step of epoch 0:   1%|▏         | 72/5616 [00:59<1:14:37,  1.24it/s, gpt_loss=0.12, lr=2.09e-6]
+
Train step of epoch 0:   1%|▏         | 73/5616 [00:59<1:11:34,  1.29it/s, gpt_loss=0.12, lr=2.09e-6]
+
Train step of epoch 0:   1%|▏         | 73/5616 [01:00<1:11:34,  1.29it/s, gpt_loss=0.11, lr=2.09e-6]
+
Train step of epoch 0:   1%|▏         | 74/5616 [01:00<1:09:48,  1.32it/s, gpt_loss=0.11, lr=2.09e-6]
+
Train step of epoch 0:   1%|▏         | 74/5616 [01:01<1:09:48,  1.32it/s, gpt_loss=0.0782, lr=2.09e-6]
+
Train step of epoch 0:   1%|▏         | 75/5616 [01:01<1:10:21,  1.31it/s, gpt_loss=0.0782, lr=2.09e-6]
+
Train step of epoch 0:   1%|▏         | 75/5616 [01:01<1:10:21,  1.31it/s, gpt_loss=0.109, lr=2.21e-6] 
+
Train step of epoch 0:   1%|▏         | 76/5616 [01:01<1:10:36,  1.31it/s, gpt_loss=0.109, lr=2.21e-6]
+
Train step of epoch 0:   1%|▏         | 76/5616 [01:02<1:10:36,  1.31it/s, gpt_loss=0.0916, lr=2.21e-6]
+
Train step of epoch 0:   1%|▏         | 77/5616 [01:02<1:21:00,  1.14it/s, gpt_loss=0.0916, lr=2.21e-6]
+
Train step of epoch 0:   1%|▏         | 77/5616 [01:03<1:21:00,  1.14it/s, gpt_loss=0.117, lr=2.21e-6] 
+
Train step of epoch 0:   1%|▏         | 78/5616 [01:03<1:15:17,  1.23it/s, gpt_loss=0.117, lr=2.21e-6]
+
Train step of epoch 0:   1%|▏         | 78/5616 [01:04<1:15:17,  1.23it/s, gpt_loss=0.0531, lr=2.21e-6]
+
Train step of epoch 0:   1%|▏         | 79/5616 [01:04<1:11:53,  1.28it/s, gpt_loss=0.0531, lr=2.21e-6]
+
Train step of epoch 0:   1%|▏         | 79/5616 [01:05<1:11:53,  1.28it/s, gpt_loss=0.106, lr=2.33e-6] 
+
Train step of epoch 0:   1%|▏         | 80/5616 [01:05<1:14:22,  1.24it/s, gpt_loss=0.106, lr=2.33e-6]
+
Train step of epoch 0:   1%|▏         | 80/5616 [01:05<1:14:22,  1.24it/s, gpt_loss=0.0547, lr=2.33e-6]
+
Train step of epoch 0:   1%|▏         | 81/5616 [01:05<1:11:13,  1.30it/s, gpt_loss=0.0547, lr=2.33e-6]
+
Train step of epoch 0:   1%|▏         | 81/5616 [01:06<1:11:13,  1.30it/s, gpt_loss=0.0693, lr=2.33e-6]
+
Train step of epoch 0:   1%|▏         | 82/5616 [01:06<1:08:39,  1.34it/s, gpt_loss=0.0693, lr=2.33e-6]
+
Train step of epoch 0:   1%|▏         | 82/5616 [01:07<1:08:39,  1.34it/s, gpt_loss=0.0707, lr=2.33e-6]
+
Train step of epoch 0:   1%|▏         | 83/5616 [01:07<1:11:00,  1.30it/s, gpt_loss=0.0707, lr=2.33e-6]
+
Train step of epoch 0:   1%|▏         | 83/5616 [01:08<1:11:00,  1.30it/s, gpt_loss=0.0657, lr=2.44e-6]
+
Train step of epoch 0:   1%|▏         | 84/5616 [01:08<1:12:20,  1.27it/s, gpt_loss=0.0657, lr=2.44e-6]
+
Train step of epoch 0:   1%|▏         | 84/5616 [01:08<1:12:20,  1.27it/s, gpt_loss=0.141, lr=2.44e-6] 
+
Train step of epoch 0:   2%|▏         | 85/5616 [01:08<1:11:19,  1.29it/s, gpt_loss=0.141, lr=2.44e-6]
+
Train step of epoch 0:   2%|▏         | 85/5616 [01:09<1:11:19,  1.29it/s, gpt_loss=0.028, lr=2.44e-6]
+
Train step of epoch 0:   2%|▏         | 86/5616 [01:09<1:12:55,  1.26it/s, gpt_loss=0.028, lr=2.44e-6]
+
Train step of epoch 0:   2%|▏         | 86/5616 [01:10<1:12:55,  1.26it/s, gpt_loss=0.106, lr=2.44e-6]
+
Train step of epoch 0:   2%|▏         | 87/5616 [01:10<1:11:54,  1.28it/s, gpt_loss=0.106, lr=2.44e-6]
+
Train step of epoch 0:   2%|▏         | 87/5616 [01:11<1:11:54,  1.28it/s, gpt_loss=0.117, lr=2.56e-6]
+
Train step of epoch 0:   2%|▏         | 88/5616 [01:11<1:13:55,  1.25it/s, gpt_loss=0.117, lr=2.56e-6]
+
Train step of epoch 0:   2%|▏         | 88/5616 [01:12<1:13:55,  1.25it/s, gpt_loss=0.0723, lr=2.56e-6]
+
Train step of epoch 0:   2%|▏         | 89/5616 [01:12<1:18:05,  1.18it/s, gpt_loss=0.0723, lr=2.56e-6]
+
Train step of epoch 0:   2%|▏         | 89/5616 [01:13<1:18:05,  1.18it/s, gpt_loss=0.0602, lr=2.56e-6]
+
Train step of epoch 0:   2%|▏         | 90/5616 [01:13<1:13:47,  1.25it/s, gpt_loss=0.0602, lr=2.56e-6]
+
Train step of epoch 0:   2%|▏         | 90/5616 [01:14<1:13:47,  1.25it/s, gpt_loss=0.112, lr=2.56e-6] 
+
Train step of epoch 0:   2%|▏         | 91/5616 [01:14<1:18:37,  1.17it/s, gpt_loss=0.112, lr=2.56e-6]
+
Train step of epoch 0:   2%|▏         | 91/5616 [01:14<1:18:37,  1.17it/s, gpt_loss=0.114, lr=2.67e-6]
+
Train step of epoch 0:   2%|▏         | 92/5616 [01:14<1:19:32,  1.16it/s, gpt_loss=0.114, lr=2.67e-6]
+
Train step of epoch 0:   2%|▏         | 92/5616 [01:15<1:19:32,  1.16it/s, gpt_loss=0.0668, lr=2.67e-6]
+
Train step of epoch 0:   2%|▏         | 93/5616 [01:15<1:16:05,  1.21it/s, gpt_loss=0.0668, lr=2.67e-6]
+
Train step of epoch 0:   2%|▏         | 93/5616 [01:16<1:16:05,  1.21it/s, gpt_loss=0.0891, lr=2.67e-6]
+
Train step of epoch 0:   2%|▏         | 94/5616 [01:16<1:14:21,  1.24it/s, gpt_loss=0.0891, lr=2.67e-6]
+
Train step of epoch 0:   2%|▏         | 94/5616 [01:17<1:14:21,  1.24it/s, gpt_loss=0.0296, lr=2.67e-6]
+
Train step of epoch 0:   2%|▏         | 95/5616 [01:17<1:10:27,  1.31it/s, gpt_loss=0.0296, lr=2.67e-6]
+
Train step of epoch 0:   2%|▏         | 95/5616 [01:18<1:10:27,  1.31it/s, gpt_loss=0.0923, lr=2.79e-6]
+
Train step of epoch 0:   2%|▏         | 96/5616 [01:18<1:16:41,  1.20it/s, gpt_loss=0.0923, lr=2.79e-6]
+
Train step of epoch 0:   2%|▏         | 96/5616 [01:18<1:16:41,  1.20it/s, gpt_loss=0.0576, lr=2.79e-6]
+
Train step of epoch 0:   2%|▏         | 97/5616 [01:18<1:14:00,  1.24it/s, gpt_loss=0.0576, lr=2.79e-6]
+
Train step of epoch 0:   2%|▏         | 97/5616 [01:19<1:14:00,  1.24it/s, gpt_loss=0.0818, lr=2.79e-6]
+
Train step of epoch 0:   2%|▏         | 98/5616 [01:19<1:10:31,  1.30it/s, gpt_loss=0.0818, lr=2.79e-6]
+
Train step of epoch 0:   2%|▏         | 98/5616 [01:20<1:10:31,  1.30it/s, gpt_loss=0.0432, lr=2.79e-6]
+
Train step of epoch 0:   2%|▏         | 99/5616 [01:20<1:18:12,  1.18it/s, gpt_loss=0.0432, lr=2.79e-6]
+
Train step of epoch 0:   2%|▏         | 99/5616 [01:21<1:18:12,  1.18it/s, gpt_loss=0.093, lr=2.91e-6] 
+
Train step of epoch 0:   2%|▏         | 100/5616 [01:21<1:18:03,  1.18it/s, gpt_loss=0.093, lr=2.91e-6]
+
Train step of epoch 0:   2%|▏         | 100/5616 [01:22<1:18:03,  1.18it/s, gpt_loss=0.166, lr=2.91e-6]
+
Train step of epoch 0:   2%|▏         | 101/5616 [01:22<1:13:06,  1.26it/s, gpt_loss=0.166, lr=2.91e-6]
+
Train step of epoch 0:   2%|▏         | 101/5616 [01:22<1:13:06,  1.26it/s, gpt_loss=0.0376, lr=2.91e-6]
+
Train step of epoch 0:   2%|▏         | 102/5616 [01:22<1:11:17,  1.29it/s, gpt_loss=0.0376, lr=2.91e-6]
+
Train step of epoch 0:   2%|▏         | 102/5616 [01:23<1:11:17,  1.29it/s, gpt_loss=0.162, lr=2.91e-6] 
+
Train step of epoch 0:   2%|▏         | 103/5616 [01:23<1:11:55,  1.28it/s, gpt_loss=0.162, lr=2.91e-6]
+
Train step of epoch 0:   2%|▏         | 103/5616 [01:24<1:11:55,  1.28it/s, gpt_loss=0.0261, lr=3.02e-6]
+
Train step of epoch 0:   2%|▏         | 104/5616 [01:24<1:13:24,  1.25it/s, gpt_loss=0.0261, lr=3.02e-6]
+
Train step of epoch 0:   2%|▏         | 104/5616 [01:25<1:13:24,  1.25it/s, gpt_loss=0.136, lr=3.02e-6] 
+
Train step of epoch 0:   2%|▏         | 105/5616 [01:25<1:09:07,  1.33it/s, gpt_loss=0.136, lr=3.02e-6]
+
Train step of epoch 0:   2%|▏         | 105/5616 [01:25<1:09:07,  1.33it/s, gpt_loss=0.0584, lr=3.02e-6]
+
Train step of epoch 0:   2%|▏         | 106/5616 [01:25<1:06:16,  1.39it/s, gpt_loss=0.0584, lr=3.02e-6]
+
Train step of epoch 0:   2%|▏         | 106/5616 [01:26<1:06:16,  1.39it/s, gpt_loss=0.0287, lr=3.02e-6]
+
Train step of epoch 0:   2%|▏         | 107/5616 [01:26<1:06:30,  1.38it/s, gpt_loss=0.0287, lr=3.02e-6]
+
Train step of epoch 0:   2%|▏         | 107/5616 [01:27<1:06:30,  1.38it/s, gpt_loss=0.0294, lr=3.14e-6]
+
Train step of epoch 0:   2%|▏         | 108/5616 [01:27<1:07:34,  1.36it/s, gpt_loss=0.0294, lr=3.14e-6]
+
Train step of epoch 0:   2%|▏         | 108/5616 [01:28<1:07:34,  1.36it/s, gpt_loss=0.0916, lr=3.14e-6]
+
Train step of epoch 0:   2%|▏         | 109/5616 [01:28<1:08:25,  1.34it/s, gpt_loss=0.0916, lr=3.14e-6]
+
Train step of epoch 0:   2%|▏         | 109/5616 [01:28<1:08:25,  1.34it/s, gpt_loss=0.178, lr=3.14e-6] 
+
Train step of epoch 0:   2%|▏         | 110/5616 [01:28<1:09:12,  1.33it/s, gpt_loss=0.178, lr=3.14e-6]
+
Train step of epoch 0:   2%|▏         | 110/5616 [01:29<1:09:12,  1.33it/s, gpt_loss=0.085, lr=3.14e-6]
+
Train step of epoch 0:   2%|▏         | 111/5616 [01:29<1:08:29,  1.34it/s, gpt_loss=0.085, lr=3.14e-6]
+
Train step of epoch 0:   2%|▏         | 111/5616 [01:30<1:08:29,  1.34it/s, gpt_loss=0.0993, lr=3.26e-6]
+
Train step of epoch 0:   2%|▏         | 112/5616 [01:30<1:09:57,  1.31it/s, gpt_loss=0.0993, lr=3.26e-6]
+
Train step of epoch 0:   2%|▏         | 112/5616 [01:31<1:09:57,  1.31it/s, gpt_loss=0.0726, lr=3.26e-6]
+
Train step of epoch 0:   2%|▏         | 113/5616 [01:31<1:09:24,  1.32it/s, gpt_loss=0.0726, lr=3.26e-6]
+
Train step of epoch 0:   2%|▏         | 113/5616 [01:31<1:09:24,  1.32it/s, gpt_loss=0.209, lr=3.26e-6] 
+
Train step of epoch 0:   2%|▏         | 114/5616 [01:31<1:07:36,  1.36it/s, gpt_loss=0.209, lr=3.26e-6]
+
Train step of epoch 0:   2%|▏         | 114/5616 [01:32<1:07:36,  1.36it/s, gpt_loss=0.104, lr=3.26e-6]
+
Train step of epoch 0:   2%|▏         | 115/5616 [01:32<1:08:18,  1.34it/s, gpt_loss=0.104, lr=3.26e-6]
+
Train step of epoch 0:   2%|▏         | 115/5616 [01:33<1:08:18,  1.34it/s, gpt_loss=0.138, lr=3.37e-6]
+
Train step of epoch 0:   2%|▏         | 116/5616 [01:33<1:09:28,  1.32it/s, gpt_loss=0.138, lr=3.37e-6]
+
Train step of epoch 0:   2%|▏         | 116/5616 [01:33<1:09:28,  1.32it/s, gpt_loss=0.0964, lr=3.37e-6]
+
Train step of epoch 0:   2%|▏         | 117/5616 [01:33<1:07:42,  1.35it/s, gpt_loss=0.0964, lr=3.37e-6]
+
Train step of epoch 0:   2%|▏         | 117/5616 [01:34<1:07:42,  1.35it/s, gpt_loss=0.134, lr=3.37e-6] 
+
Train step of epoch 0:   2%|▏         | 118/5616 [01:34<1:07:01,  1.37it/s, gpt_loss=0.134, lr=3.37e-6]
+
Train step of epoch 0:   2%|▏         | 118/5616 [01:35<1:07:01,  1.37it/s, gpt_loss=0.0506, lr=3.37e-6]
+
Train step of epoch 0:   2%|▏         | 119/5616 [01:35<1:07:21,  1.36it/s, gpt_loss=0.0506, lr=3.37e-6]
+
Train step of epoch 0:   2%|▏         | 119/5616 [01:36<1:07:21,  1.36it/s, gpt_loss=0.0542, lr=3.49e-6]
+
Train step of epoch 0:   2%|▏         | 120/5616 [01:36<1:10:29,  1.30it/s, gpt_loss=0.0542, lr=3.49e-6]
+
Train step of epoch 0:   2%|▏         | 120/5616 [01:37<1:10:29,  1.30it/s, gpt_loss=0.0625, lr=3.49e-6]
+
Train step of epoch 0:   2%|▏         | 121/5616 [01:37<1:10:16,  1.30it/s, gpt_loss=0.0625, lr=3.49e-6]
+
Train step of epoch 0:   2%|▏         | 121/5616 [01:37<1:10:16,  1.30it/s, gpt_loss=0.0897, lr=3.49e-6]
+
Train step of epoch 0:   2%|▏         | 122/5616 [01:37<1:10:30,  1.30it/s, gpt_loss=0.0897, lr=3.49e-6]
+
Train step of epoch 0:   2%|▏         | 122/5616 [01:38<1:10:30,  1.30it/s, gpt_loss=0.072, lr=3.49e-6] 
+
Train step of epoch 0:   2%|▏         | 123/5616 [01:38<1:09:58,  1.31it/s, gpt_loss=0.072, lr=3.49e-6]
+
Train step of epoch 0:   2%|▏         | 123/5616 [01:39<1:09:58,  1.31it/s, gpt_loss=0.0658, lr=3.6e-6]
+
Train step of epoch 0:   2%|▏         | 124/5616 [01:39<1:13:12,  1.25it/s, gpt_loss=0.0658, lr=3.6e-6]
+
Train step of epoch 0:   2%|▏         | 124/5616 [01:40<1:13:12,  1.25it/s, gpt_loss=0.0414, lr=3.6e-6]
+
Train step of epoch 0:   2%|▏         | 125/5616 [01:40<1:11:43,  1.28it/s, gpt_loss=0.0414, lr=3.6e-6]
+
Train step of epoch 0:   2%|▏         | 125/5616 [01:40<1:11:43,  1.28it/s, gpt_loss=0.0665, lr=3.6e-6]
+
Train step of epoch 0:   2%|▏         | 126/5616 [01:40<1:10:22,  1.30it/s, gpt_loss=0.0665, lr=3.6e-6]
+
Train step of epoch 0:   2%|▏         | 126/5616 [01:41<1:10:22,  1.30it/s, gpt_loss=0.0942, lr=3.6e-6]
+
Train step of epoch 0:   2%|▏         | 127/5616 [01:41<1:10:39,  1.29it/s, gpt_loss=0.0942, lr=3.6e-6]
+
Train step of epoch 0:   2%|▏         | 127/5616 [01:42<1:10:39,  1.29it/s, gpt_loss=0.142, lr=3.72e-6]
+
Train step of epoch 0:   2%|▏         | 128/5616 [01:42<1:11:26,  1.28it/s, gpt_loss=0.142, lr=3.72e-6]
+
Train step of epoch 0:   2%|▏         | 128/5616 [01:43<1:11:26,  1.28it/s, gpt_loss=0.0819, lr=3.72e-6]
+
Train step of epoch 0:   2%|▏         | 129/5616 [01:43<1:08:28,  1.34it/s, gpt_loss=0.0819, lr=3.72e-6]
+
Train step of epoch 0:   2%|▏         | 129/5616 [01:43<1:08:28,  1.34it/s, gpt_loss=0.0874, lr=3.72e-6]
+
Train step of epoch 0:   2%|▏         | 130/5616 [01:43<1:06:39,  1.37it/s, gpt_loss=0.0874, lr=3.72e-6]
+
Train step of epoch 0:   2%|▏         | 130/5616 [01:44<1:06:39,  1.37it/s, gpt_loss=0.0931, lr=3.72e-6]
+
Train step of epoch 0:   2%|▏         | 131/5616 [01:44<1:08:45,  1.33it/s, gpt_loss=0.0931, lr=3.72e-6]
+
Train step of epoch 0:   2%|▏         | 131/5616 [01:45<1:08:45,  1.33it/s, gpt_loss=0.0298, lr=3.84e-6]
+
Train step of epoch 0:   2%|▏         | 132/5616 [01:45<1:09:46,  1.31it/s, gpt_loss=0.0298, lr=3.84e-6]
+
Train step of epoch 0:   2%|▏         | 132/5616 [01:46<1:09:46,  1.31it/s, gpt_loss=0.131, lr=3.84e-6] 
+
Train step of epoch 0:   2%|▏         | 133/5616 [01:46<1:08:52,  1.33it/s, gpt_loss=0.131, lr=3.84e-6]
+
Train step of epoch 0:   2%|▏         | 133/5616 [01:46<1:08:52,  1.33it/s, gpt_loss=0.0403, lr=3.84e-6]
+
Train step of epoch 0:   2%|▏         | 134/5616 [01:46<1:07:27,  1.35it/s, gpt_loss=0.0403, lr=3.84e-6]
+
Train step of epoch 0:   2%|▏         | 134/5616 [01:47<1:07:27,  1.35it/s, gpt_loss=0.0502, lr=3.84e-6]
+
Train step of epoch 0:   2%|▏         | 135/5616 [01:47<1:05:45,  1.39it/s, gpt_loss=0.0502, lr=3.84e-6]
+
Train step of epoch 0:   2%|▏         | 135/5616 [01:48<1:05:45,  1.39it/s, gpt_loss=0.0648, lr=3.95e-6]
+
Train step of epoch 0:   2%|▏         | 136/5616 [01:48<1:14:57,  1.22it/s, gpt_loss=0.0648, lr=3.95e-6]
+
Train step of epoch 0:   2%|▏         | 136/5616 [01:49<1:14:57,  1.22it/s, gpt_loss=0.0609, lr=3.95e-6]
+
Train step of epoch 0:   2%|▏         | 137/5616 [01:49<1:11:17,  1.28it/s, gpt_loss=0.0609, lr=3.95e-6]
+
Train step of epoch 0:   2%|▏         | 137/5616 [01:50<1:11:17,  1.28it/s, gpt_loss=0.0611, lr=3.95e-6]
+
Train step of epoch 0:   2%|▏         | 138/5616 [01:50<1:10:46,  1.29it/s, gpt_loss=0.0611, lr=3.95e-6]
+
Train step of epoch 0:   2%|▏         | 138/5616 [01:50<1:10:46,  1.29it/s, gpt_loss=0.115, lr=3.95e-6] 
+
Train step of epoch 0:   2%|▏         | 139/5616 [01:50<1:07:21,  1.36it/s, gpt_loss=0.115, lr=3.95e-6]
+
Train step of epoch 0:   2%|▏         | 139/5616 [01:51<1:07:21,  1.36it/s, gpt_loss=0.0671, lr=4.07e-6]
+
Train step of epoch 0:   2%|▏         | 140/5616 [01:51<1:11:49,  1.27it/s, gpt_loss=0.0671, lr=4.07e-6]
+
Train step of epoch 0:   2%|▏         | 140/5616 [01:52<1:11:49,  1.27it/s, gpt_loss=0.0232, lr=4.07e-6]
+
Train step of epoch 0:   3%|▎         | 141/5616 [01:52<1:09:24,  1.31it/s, gpt_loss=0.0232, lr=4.07e-6]
+
Train step of epoch 0:   3%|▎         | 141/5616 [01:53<1:09:24,  1.31it/s, gpt_loss=0.167, lr=4.07e-6] 
+
Train step of epoch 0:   3%|▎         | 142/5616 [01:53<1:10:33,  1.29it/s, gpt_loss=0.167, lr=4.07e-6]
+
Train step of epoch 0:   3%|▎         | 142/5616 [01:53<1:10:33,  1.29it/s, gpt_loss=0.0572, lr=4.07e-6]
+
Train step of epoch 0:   3%|▎         | 143/5616 [01:53<1:08:07,  1.34it/s, gpt_loss=0.0572, lr=4.07e-6]
+
Train step of epoch 0:   3%|▎         | 143/5616 [01:55<1:08:07,  1.34it/s, gpt_loss=0.0965, lr=4.19e-6]
+
Train step of epoch 0:   3%|▎         | 144/5616 [01:55<1:20:17,  1.14it/s, gpt_loss=0.0965, lr=4.19e-6]
+
Train step of epoch 0:   3%|▎         | 144/5616 [01:55<1:20:17,  1.14it/s, gpt_loss=0.116, lr=4.19e-6] 
+
Train step of epoch 0:   3%|▎         | 145/5616 [01:55<1:19:47,  1.14it/s, gpt_loss=0.116, lr=4.19e-6]
+
Train step of epoch 0:   3%|▎         | 145/5616 [01:56<1:19:47,  1.14it/s, gpt_loss=0.0615, lr=4.19e-6]
+
Train step of epoch 0:   3%|▎         | 146/5616 [01:56<1:14:52,  1.22it/s, gpt_loss=0.0615, lr=4.19e-6]
+
Train step of epoch 0:   3%|▎         | 146/5616 [01:57<1:14:52,  1.22it/s, gpt_loss=0.141, lr=4.19e-6] 
+
Train step of epoch 0:   3%|▎         | 147/5616 [01:57<1:11:35,  1.27it/s, gpt_loss=0.141, lr=4.19e-6]
+
Train step of epoch 0:   3%|▎         | 147/5616 [01:58<1:11:35,  1.27it/s, gpt_loss=0.0811, lr=4.3e-6]
+
Train step of epoch 0:   3%|▎         | 148/5616 [01:58<1:14:01,  1.23it/s, gpt_loss=0.0811, lr=4.3e-6]
+
Train step of epoch 0:   3%|▎         | 148/5616 [01:58<1:14:01,  1.23it/s, gpt_loss=0.0316, lr=4.3e-6]
+
Train step of epoch 0:   3%|▎         | 149/5616 [01:58<1:10:23,  1.29it/s, gpt_loss=0.0316, lr=4.3e-6]
+
Train step of epoch 0:   3%|▎         | 149/5616 [01:59<1:10:23,  1.29it/s, gpt_loss=0.0264, lr=4.3e-6]
+
Train step of epoch 0:   3%|▎         | 150/5616 [01:59<1:12:56,  1.25it/s, gpt_loss=0.0264, lr=4.3e-6]
+
Train step of epoch 0:   3%|▎         | 150/5616 [02:00<1:12:56,  1.25it/s, gpt_loss=0.0529, lr=4.3e-6]
+
Train step of epoch 0:   3%|▎         | 151/5616 [02:00<1:13:19,  1.24it/s, gpt_loss=0.0529, lr=4.3e-6]
+
Train step of epoch 0:   3%|▎         | 151/5616 [02:01<1:13:19,  1.24it/s, gpt_loss=0.0124, lr=4.42e-6]
+
Train step of epoch 0:   3%|▎         | 152/5616 [02:01<1:13:46,  1.23it/s, gpt_loss=0.0124, lr=4.42e-6]
+
Train step of epoch 0:   3%|▎         | 152/5616 [02:02<1:13:46,  1.23it/s, gpt_loss=0.0937, lr=4.42e-6]
+
Train step of epoch 0:   3%|▎         | 153/5616 [02:02<1:09:32,  1.31it/s, gpt_loss=0.0937, lr=4.42e-6]
+
Train step of epoch 0:   3%|▎         | 153/5616 [02:02<1:09:32,  1.31it/s, gpt_loss=0.0347, lr=4.42e-6]
+
Train step of epoch 0:   3%|▎         | 154/5616 [02:02<1:06:58,  1.36it/s, gpt_loss=0.0347, lr=4.42e-6]
+
Train step of epoch 0:   3%|▎         | 154/5616 [02:03<1:06:58,  1.36it/s, gpt_loss=0.0306, lr=4.42e-6]
+
Train step of epoch 0:   3%|▎         | 155/5616 [02:03<1:09:54,  1.30it/s, gpt_loss=0.0306, lr=4.42e-6]
+
Train step of epoch 0:   3%|▎         | 155/5616 [02:04<1:09:54,  1.30it/s, gpt_loss=0.0443, lr=4.53e-6]
+
Train step of epoch 0:   3%|▎         | 156/5616 [02:04<1:09:32,  1.31it/s, gpt_loss=0.0443, lr=4.53e-6]
+
Train step of epoch 0:   3%|▎         | 156/5616 [02:04<1:09:32,  1.31it/s, gpt_loss=0.0267, lr=4.53e-6]
+
Train step of epoch 0:   3%|▎         | 157/5616 [02:04<1:06:23,  1.37it/s, gpt_loss=0.0267, lr=4.53e-6]
+
Train step of epoch 0:   3%|▎         | 157/5616 [02:05<1:06:23,  1.37it/s, gpt_loss=0.0347, lr=4.53e-6]
+
Train step of epoch 0:   3%|▎         | 158/5616 [02:05<1:05:44,  1.38it/s, gpt_loss=0.0347, lr=4.53e-6]
+
Train step of epoch 0:   3%|▎         | 158/5616 [02:06<1:05:44,  1.38it/s, gpt_loss=0.0552, lr=4.53e-6]
+
Train step of epoch 0:   3%|▎         | 159/5616 [02:06<1:05:56,  1.38it/s, gpt_loss=0.0552, lr=4.53e-6]
+
Train step of epoch 0:   3%|▎         | 159/5616 [02:07<1:05:56,  1.38it/s, gpt_loss=0.0258, lr=4.65e-6]
+
Train step of epoch 0:   3%|▎         | 160/5616 [02:07<1:07:54,  1.34it/s, gpt_loss=0.0258, lr=4.65e-6]
+
Train step of epoch 0:   3%|▎         | 160/5616 [02:07<1:07:54,  1.34it/s, gpt_loss=0.053, lr=4.65e-6] 
+
Train step of epoch 0:   3%|▎         | 161/5616 [02:07<1:06:03,  1.38it/s, gpt_loss=0.053, lr=4.65e-6]
+
Train step of epoch 0:   3%|▎         | 161/5616 [02:08<1:06:03,  1.38it/s, gpt_loss=0.0642, lr=4.65e-6]
+
Train step of epoch 0:   3%|▎         | 162/5616 [02:08<1:04:33,  1.41it/s, gpt_loss=0.0642, lr=4.65e-6]
+
Train step of epoch 0:   3%|▎         | 162/5616 [02:09<1:04:33,  1.41it/s, gpt_loss=0.0269, lr=4.65e-6]
+
Train step of epoch 0:   3%|▎         | 163/5616 [02:09<1:02:14,  1.46it/s, gpt_loss=0.0269, lr=4.65e-6]
+
Train step of epoch 0:   3%|▎         | 163/5616 [02:10<1:02:14,  1.46it/s, gpt_loss=0.0646, lr=4.77e-6]
+
Train step of epoch 0:   3%|▎         | 164/5616 [02:10<1:08:10,  1.33it/s, gpt_loss=0.0646, lr=4.77e-6]
+
Train step of epoch 0:   3%|▎         | 164/5616 [02:10<1:08:10,  1.33it/s, gpt_loss=0.0607, lr=4.77e-6]
+
Train step of epoch 0:   3%|▎         | 165/5616 [02:10<1:06:01,  1.38it/s, gpt_loss=0.0607, lr=4.77e-6]
+
Train step of epoch 0:   3%|▎         | 165/5616 [02:11<1:06:01,  1.38it/s, gpt_loss=0.0108, lr=4.77e-6]
+
Train step of epoch 0:   3%|▎         | 166/5616 [02:11<1:05:55,  1.38it/s, gpt_loss=0.0108, lr=4.77e-6]
+
Train step of epoch 0:   3%|▎         | 166/5616 [02:12<1:05:55,  1.38it/s, gpt_loss=0.0148, lr=4.77e-6]
+
Train step of epoch 0:   3%|▎         | 167/5616 [02:12<1:06:27,  1.37it/s, gpt_loss=0.0148, lr=4.77e-6]
+
Train step of epoch 0:   3%|▎         | 167/5616 [02:13<1:06:27,  1.37it/s, gpt_loss=0.0362, lr=4.88e-6]
+
Train step of epoch 0:   3%|▎         | 168/5616 [02:13<1:09:07,  1.31it/s, gpt_loss=0.0362, lr=4.88e-6]
+
Train step of epoch 0:   3%|▎         | 168/5616 [02:14<1:09:07,  1.31it/s, gpt_loss=0.176, lr=4.88e-6] 
+
Train step of epoch 0:   3%|▎         | 169/5616 [02:14<1:18:50,  1.15it/s, gpt_loss=0.176, lr=4.88e-6]
+
Train step of epoch 0:   3%|▎         | 169/5616 [02:14<1:18:50,  1.15it/s, gpt_loss=0.0564, lr=4.88e-6]
+
Train step of epoch 0:   3%|▎         | 170/5616 [02:14<1:15:05,  1.21it/s, gpt_loss=0.0564, lr=4.88e-6]
+
Train step of epoch 0:   3%|▎         | 170/5616 [02:15<1:15:05,  1.21it/s, gpt_loss=0.0744, lr=4.88e-6]
+
Train step of epoch 0:   3%|▎         | 171/5616 [02:15<1:12:55,  1.24it/s, gpt_loss=0.0744, lr=4.88e-6]
+
Train step of epoch 0:   3%|▎         | 171/5616 [02:16<1:12:55,  1.24it/s, gpt_loss=0.0696, lr=5e-6]   
+
Train step of epoch 0:   3%|▎         | 172/5616 [02:16<1:15:50,  1.20it/s, gpt_loss=0.0696, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 172/5616 [02:17<1:15:50,  1.20it/s, gpt_loss=0.0575, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 173/5616 [02:17<1:14:11,  1.22it/s, gpt_loss=0.0575, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 173/5616 [02:17<1:14:11,  1.22it/s, gpt_loss=0.0168, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 174/5616 [02:17<1:09:28,  1.31it/s, gpt_loss=0.0168, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 174/5616 [02:18<1:09:28,  1.31it/s, gpt_loss=0.105, lr=5e-6] 
+
Train step of epoch 0:   3%|▎         | 175/5616 [02:18<1:08:56,  1.32it/s, gpt_loss=0.105, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 175/5616 [02:19<1:08:56,  1.32it/s, gpt_loss=0.0998, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 176/5616 [02:19<1:10:51,  1.28it/s, gpt_loss=0.0998, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 176/5616 [02:20<1:10:51,  1.28it/s, gpt_loss=0.137, lr=5e-6] 
+
Train step of epoch 0:   3%|▎         | 177/5616 [02:20<1:10:07,  1.29it/s, gpt_loss=0.137, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 177/5616 [02:21<1:10:07,  1.29it/s, gpt_loss=0.103, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 178/5616 [02:21<1:09:00,  1.31it/s, gpt_loss=0.103, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 178/5616 [02:21<1:09:00,  1.31it/s, gpt_loss=0.00588, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 179/5616 [02:21<1:12:57,  1.24it/s, gpt_loss=0.00588, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 179/5616 [02:22<1:12:57,  1.24it/s, gpt_loss=0.157, lr=5e-6]  
+
Train step of epoch 0:   3%|▎         | 180/5616 [02:22<1:12:56,  1.24it/s, gpt_loss=0.157, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 180/5616 [02:23<1:12:56,  1.24it/s, gpt_loss=0.202, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 181/5616 [02:23<1:11:32,  1.27it/s, gpt_loss=0.202, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 181/5616 [02:24<1:11:32,  1.27it/s, gpt_loss=0.01, lr=5e-6] 
+
Train step of epoch 0:   3%|▎         | 182/5616 [02:24<1:08:20,  1.33it/s, gpt_loss=0.01, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 182/5616 [02:24<1:08:20,  1.33it/s, gpt_loss=0.0175, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 183/5616 [02:24<1:07:00,  1.35it/s, gpt_loss=0.0175, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 183/5616 [02:25<1:07:00,  1.35it/s, gpt_loss=0.0851, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 184/5616 [02:25<1:08:14,  1.33it/s, gpt_loss=0.0851, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 184/5616 [02:26<1:08:14,  1.33it/s, gpt_loss=0.0108, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 185/5616 [02:26<1:07:36,  1.34it/s, gpt_loss=0.0108, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 185/5616 [02:27<1:07:36,  1.34it/s, gpt_loss=0.128, lr=5e-6] 
+
Train step of epoch 0:   3%|▎         | 186/5616 [02:27<1:06:20,  1.36it/s, gpt_loss=0.128, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 186/5616 [02:27<1:06:20,  1.36it/s, gpt_loss=0.00361, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 187/5616 [02:27<1:08:03,  1.33it/s, gpt_loss=0.00361, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 187/5616 [02:28<1:08:03,  1.33it/s, gpt_loss=0.0634, lr=5e-6] 
+
Train step of epoch 0:   3%|▎         | 188/5616 [02:28<1:08:50,  1.31it/s, gpt_loss=0.0634, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 188/5616 [02:29<1:08:50,  1.31it/s, gpt_loss=0.105, lr=5e-6] 
+
Train step of epoch 0:   3%|▎         | 189/5616 [02:29<1:05:30,  1.38it/s, gpt_loss=0.105, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 189/5616 [02:30<1:05:30,  1.38it/s, gpt_loss=0.0949, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 190/5616 [02:30<1:09:16,  1.31it/s, gpt_loss=0.0949, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 190/5616 [02:30<1:09:16,  1.31it/s, gpt_loss=0.0636, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 191/5616 [02:30<1:07:15,  1.34it/s, gpt_loss=0.0636, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 191/5616 [02:31<1:07:15,  1.34it/s, gpt_loss=0.117, lr=5e-6] 
+
Train step of epoch 0:   3%|▎         | 192/5616 [02:31<1:11:29,  1.26it/s, gpt_loss=0.117, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 192/5616 [02:32<1:11:29,  1.26it/s, gpt_loss=0.00909, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 193/5616 [02:32<1:11:44,  1.26it/s, gpt_loss=0.00909, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 193/5616 [02:33<1:11:44,  1.26it/s, gpt_loss=0.0807, lr=5e-6] 
+
Train step of epoch 0:   3%|▎         | 194/5616 [02:33<1:08:46,  1.31it/s, gpt_loss=0.0807, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 194/5616 [02:33<1:08:46,  1.31it/s, gpt_loss=0.0742, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 195/5616 [02:33<1:07:10,  1.35it/s, gpt_loss=0.0742, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 195/5616 [02:34<1:07:10,  1.35it/s, gpt_loss=0.0125, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 196/5616 [02:34<1:11:14,  1.27it/s, gpt_loss=0.0125, lr=5e-6]
+
Train step of epoch 0:   3%|▎         | 196/5616 [02:35<1:11:14,  1.27it/s, gpt_loss=0.0236, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 197/5616 [02:35<1:07:57,  1.33it/s, gpt_loss=0.0236, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 197/5616 [02:36<1:07:57,  1.33it/s, gpt_loss=0.0143, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 198/5616 [02:36<1:07:06,  1.35it/s, gpt_loss=0.0143, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 198/5616 [02:36<1:07:06,  1.35it/s, gpt_loss=0.124, lr=5e-6] 
+
Train step of epoch 0:   4%|▎         | 199/5616 [02:36<1:05:11,  1.38it/s, gpt_loss=0.124, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 199/5616 [02:37<1:05:11,  1.38it/s, gpt_loss=0.0346, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 200/5616 [02:37<1:07:16,  1.34it/s, gpt_loss=0.0346, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 200/5616 [02:38<1:07:16,  1.34it/s, gpt_loss=0.0533, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 201/5616 [02:38<1:04:41,  1.40it/s, gpt_loss=0.0533, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 201/5616 [02:39<1:04:41,  1.40it/s, gpt_loss=0.0412, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 202/5616 [02:39<1:14:25,  1.21it/s, gpt_loss=0.0412, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 202/5616 [02:40<1:14:25,  1.21it/s, gpt_loss=0.04, lr=5e-6]  
+
Train step of epoch 0:   4%|▎         | 203/5616 [02:40<1:11:26,  1.26it/s, gpt_loss=0.04, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 203/5616 [02:40<1:11:26,  1.26it/s, gpt_loss=0.0638, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 204/5616 [02:40<1:11:15,  1.27it/s, gpt_loss=0.0638, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 204/5616 [02:41<1:11:15,  1.27it/s, gpt_loss=0.02, lr=5e-6]  
+
Train step of epoch 0:   4%|▎         | 205/5616 [02:41<1:14:07,  1.22it/s, gpt_loss=0.02, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 205/5616 [02:42<1:14:07,  1.22it/s, gpt_loss=0.0363, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 206/5616 [02:42<1:13:30,  1.23it/s, gpt_loss=0.0363, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 206/5616 [02:43<1:13:30,  1.23it/s, gpt_loss=0.0166, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 207/5616 [02:43<1:17:40,  1.16it/s, gpt_loss=0.0166, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 207/5616 [02:44<1:17:40,  1.16it/s, gpt_loss=0.0282, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 208/5616 [02:44<1:15:59,  1.19it/s, gpt_loss=0.0282, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 208/5616 [02:45<1:15:59,  1.19it/s, gpt_loss=0.0336, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 209/5616 [02:45<1:11:31,  1.26it/s, gpt_loss=0.0336, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 209/5616 [02:45<1:11:31,  1.26it/s, gpt_loss=0.0422, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 210/5616 [02:45<1:07:43,  1.33it/s, gpt_loss=0.0422, lr=5e-6]
+
Train step of epoch 0:   4%|▎         | 210/5616 [02:46<1:07:43,  1.33it/s, gpt_loss=0.0353, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 211/5616 [02:46<1:04:49,  1.39it/s, gpt_loss=0.0353, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 211/5616 [02:47<1:04:49,  1.39it/s, gpt_loss=0.0247, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 212/5616 [02:47<1:05:52,  1.37it/s, gpt_loss=0.0247, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 212/5616 [02:47<1:05:52,  1.37it/s, gpt_loss=0.0901, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 213/5616 [02:47<1:09:55,  1.29it/s, gpt_loss=0.0901, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 213/5616 [02:48<1:09:55,  1.29it/s, gpt_loss=0.106, lr=5e-6] 
+
Train step of epoch 0:   4%|▍         | 214/5616 [02:48<1:09:26,  1.30it/s, gpt_loss=0.106, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 214/5616 [02:49<1:09:26,  1.30it/s, gpt_loss=0.0624, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 215/5616 [02:49<1:09:28,  1.30it/s, gpt_loss=0.0624, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 215/5616 [02:50<1:09:28,  1.30it/s, gpt_loss=0.0422, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 216/5616 [02:50<1:09:42,  1.29it/s, gpt_loss=0.0422, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 216/5616 [02:51<1:09:42,  1.29it/s, gpt_loss=0.0784, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 217/5616 [02:51<1:07:42,  1.33it/s, gpt_loss=0.0784, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 217/5616 [02:51<1:07:42,  1.33it/s, gpt_loss=0.0333, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 218/5616 [02:51<1:05:29,  1.37it/s, gpt_loss=0.0333, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 218/5616 [02:52<1:05:29,  1.37it/s, gpt_loss=0.0231, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 219/5616 [02:52<1:07:23,  1.33it/s, gpt_loss=0.0231, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 219/5616 [02:53<1:07:23,  1.33it/s, gpt_loss=0.00752, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 220/5616 [02:53<1:07:49,  1.33it/s, gpt_loss=0.00752, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 220/5616 [02:53<1:07:49,  1.33it/s, gpt_loss=0.019, lr=5e-6]  
+
Train step of epoch 0:   4%|▍         | 221/5616 [02:53<1:06:23,  1.35it/s, gpt_loss=0.019, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 221/5616 [02:54<1:06:23,  1.35it/s, gpt_loss=0.0117, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 222/5616 [02:54<1:10:02,  1.28it/s, gpt_loss=0.0117, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 222/5616 [02:55<1:10:02,  1.28it/s, gpt_loss=0.253, lr=5e-6] 
+
Train step of epoch 0:   4%|▍         | 223/5616 [02:55<1:13:23,  1.22it/s, gpt_loss=0.253, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 223/5616 [02:56<1:13:23,  1.22it/s, gpt_loss=0.0948, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 224/5616 [02:56<1:14:18,  1.21it/s, gpt_loss=0.0948, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 224/5616 [02:57<1:14:18,  1.21it/s, gpt_loss=0.00461, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 225/5616 [02:57<1:10:39,  1.27it/s, gpt_loss=0.00461, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 225/5616 [02:57<1:10:39,  1.27it/s, gpt_loss=0.0274, lr=5e-6] 
+
Train step of epoch 0:   4%|▍         | 226/5616 [02:57<1:09:00,  1.30it/s, gpt_loss=0.0274, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 226/5616 [02:58<1:09:00,  1.30it/s, gpt_loss=0.0652, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 227/5616 [02:58<1:06:16,  1.36it/s, gpt_loss=0.0652, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 227/5616 [02:59<1:06:16,  1.36it/s, gpt_loss=0.125, lr=5e-6] 
+
Train step of epoch 0:   4%|▍         | 228/5616 [02:59<1:07:28,  1.33it/s, gpt_loss=0.125, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 228/5616 [03:00<1:07:28,  1.33it/s, gpt_loss=0.0185, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 229/5616 [03:00<1:08:45,  1.31it/s, gpt_loss=0.0185, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 229/5616 [03:00<1:08:45,  1.31it/s, gpt_loss=0.0341, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 230/5616 [03:00<1:06:39,  1.35it/s, gpt_loss=0.0341, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 230/5616 [03:01<1:06:39,  1.35it/s, gpt_loss=0.0956, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 231/5616 [03:01<1:04:58,  1.38it/s, gpt_loss=0.0956, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 231/5616 [03:02<1:04:58,  1.38it/s, gpt_loss=0.0448, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 232/5616 [03:02<1:08:52,  1.30it/s, gpt_loss=0.0448, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 232/5616 [03:03<1:08:52,  1.30it/s, gpt_loss=0.0498, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 233/5616 [03:03<1:06:48,  1.34it/s, gpt_loss=0.0498, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 233/5616 [03:04<1:06:48,  1.34it/s, gpt_loss=0.0279, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 234/5616 [03:04<1:09:54,  1.28it/s, gpt_loss=0.0279, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 234/5616 [03:04<1:09:54,  1.28it/s, gpt_loss=0.058, lr=5e-6] 
+
Train step of epoch 0:   4%|▍         | 235/5616 [03:04<1:11:47,  1.25it/s, gpt_loss=0.058, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 235/5616 [03:05<1:11:47,  1.25it/s, gpt_loss=0.106, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 236/5616 [03:05<1:10:12,  1.28it/s, gpt_loss=0.106, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 236/5616 [03:06<1:10:12,  1.28it/s, gpt_loss=0.0184, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 237/5616 [03:06<1:07:07,  1.34it/s, gpt_loss=0.0184, lr=5e-6]
+
Train step of epoch 0:   4%|��         | 237/5616 [03:07<1:07:07,  1.34it/s, gpt_loss=0.0387, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 238/5616 [03:07<1:06:24,  1.35it/s, gpt_loss=0.0387, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 238/5616 [03:07<1:06:24,  1.35it/s, gpt_loss=0.0221, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 239/5616 [03:07<1:05:26,  1.37it/s, gpt_loss=0.0221, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 239/5616 [03:08<1:05:26,  1.37it/s, gpt_loss=0.121, lr=5e-6] 
+
Train step of epoch 0:   4%|▍         | 240/5616 [03:08<1:06:52,  1.34it/s, gpt_loss=0.121, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 240/5616 [03:09<1:06:52,  1.34it/s, gpt_loss=0.082, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 241/5616 [03:09<1:03:49,  1.40it/s, gpt_loss=0.082, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 241/5616 [03:09<1:03:49,  1.40it/s, gpt_loss=0.0418, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 242/5616 [03:09<1:04:48,  1.38it/s, gpt_loss=0.0418, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 242/5616 [03:10<1:04:48,  1.38it/s, gpt_loss=0.0708, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 243/5616 [03:10<1:04:47,  1.38it/s, gpt_loss=0.0708, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 243/5616 [03:11<1:04:47,  1.38it/s, gpt_loss=0.0753, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 244/5616 [03:11<1:05:31,  1.37it/s, gpt_loss=0.0753, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 244/5616 [03:12<1:05:31,  1.37it/s, gpt_loss=0.0588, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 245/5616 [03:12<1:04:08,  1.40it/s, gpt_loss=0.0588, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 245/5616 [03:12<1:04:08,  1.40it/s, gpt_loss=0.0498, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 246/5616 [03:12<1:05:00,  1.38it/s, gpt_loss=0.0498, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 246/5616 [03:13<1:05:00,  1.38it/s, gpt_loss=0.0094, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 247/5616 [03:13<1:04:03,  1.40it/s, gpt_loss=0.0094, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 247/5616 [03:14<1:04:03,  1.40it/s, gpt_loss=0.0444, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 248/5616 [03:14<1:13:22,  1.22it/s, gpt_loss=0.0444, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 248/5616 [03:15<1:13:22,  1.22it/s, gpt_loss=0.0867, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 249/5616 [03:15<1:10:47,  1.26it/s, gpt_loss=0.0867, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 249/5616 [03:15<1:10:47,  1.26it/s, gpt_loss=0.101, lr=5e-6] 
+
Train step of epoch 0:   4%|▍         | 250/5616 [03:15<1:07:17,  1.33it/s, gpt_loss=0.101, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 250/5616 [03:16<1:07:17,  1.33it/s, gpt_loss=0.032, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 251/5616 [03:16<1:06:07,  1.35it/s, gpt_loss=0.032, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 251/5616 [03:17<1:06:07,  1.35it/s, gpt_loss=0.0181, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 252/5616 [03:17<1:11:30,  1.25it/s, gpt_loss=0.0181, lr=5e-6]
+
Train step of epoch 0:   4%|▍         | 252/5616 [03:18<1:11:30,  1.25it/s, gpt_loss=0.0516, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 253/5616 [03:18<1:07:06,  1.33it/s, gpt_loss=0.0516, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 253/5616 [03:18<1:07:06,  1.33it/s, gpt_loss=0.0306, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 254/5616 [03:18<1:06:18,  1.35it/s, gpt_loss=0.0306, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 254/5616 [03:19<1:06:18,  1.35it/s, gpt_loss=0.0583, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 255/5616 [03:19<1:06:21,  1.35it/s, gpt_loss=0.0583, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 255/5616 [03:20<1:06:21,  1.35it/s, gpt_loss=0.0736, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 256/5616 [03:20<1:07:22,  1.33it/s, gpt_loss=0.0736, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 256/5616 [03:21<1:07:22,  1.33it/s, gpt_loss=0.0395, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 257/5616 [03:21<1:08:49,  1.30it/s, gpt_loss=0.0395, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 257/5616 [03:22<1:08:49,  1.30it/s, gpt_loss=0.105, lr=5e-6] 
+
Train step of epoch 0:   5%|▍         | 258/5616 [03:22<1:10:12,  1.27it/s, gpt_loss=0.105, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 258/5616 [03:22<1:10:12,  1.27it/s, gpt_loss=0.0239, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 259/5616 [03:22<1:12:17,  1.24it/s, gpt_loss=0.0239, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 259/5616 [03:23<1:12:17,  1.24it/s, gpt_loss=0.134, lr=5e-6] 
+
Train step of epoch 0:   5%|▍         | 260/5616 [03:23<1:09:51,  1.28it/s, gpt_loss=0.134, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 260/5616 [03:24<1:09:51,  1.28it/s, gpt_loss=0.0472, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 261/5616 [03:24<1:07:03,  1.33it/s, gpt_loss=0.0472, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 261/5616 [03:25<1:07:03,  1.33it/s, gpt_loss=0.0562, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 262/5616 [03:25<1:09:42,  1.28it/s, gpt_loss=0.0562, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 262/5616 [03:25<1:09:42,  1.28it/s, gpt_loss=0.0303, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 263/5616 [03:25<1:07:57,  1.31it/s, gpt_loss=0.0303, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 263/5616 [03:26<1:07:57,  1.31it/s, gpt_loss=0.0925, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 264/5616 [03:26<1:06:57,  1.33it/s, gpt_loss=0.0925, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 264/5616 [03:27<1:06:57,  1.33it/s, gpt_loss=0.0866, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 265/5616 [03:27<1:05:13,  1.37it/s, gpt_loss=0.0866, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 265/5616 [03:28<1:05:13,  1.37it/s, gpt_loss=0.0477, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 266/5616 [03:28<1:04:40,  1.38it/s, gpt_loss=0.0477, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 266/5616 [03:28<1:04:40,  1.38it/s, gpt_loss=0.0448, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 267/5616 [03:28<1:04:48,  1.38it/s, gpt_loss=0.0448, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 267/5616 [03:29<1:04:48,  1.38it/s, gpt_loss=0.0238, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 268/5616 [03:29<1:06:46,  1.33it/s, gpt_loss=0.0238, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 268/5616 [03:30<1:06:46,  1.33it/s, gpt_loss=0.0116, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 269/5616 [03:30<1:15:21,  1.18it/s, gpt_loss=0.0116, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 269/5616 [03:31<1:15:21,  1.18it/s, gpt_loss=0.116, lr=5e-6] 
+
Train step of epoch 0:   5%|▍         | 270/5616 [03:31<1:10:01,  1.27it/s, gpt_loss=0.116, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 270/5616 [03:32<1:10:01,  1.27it/s, gpt_loss=0.0447, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 271/5616 [03:32<1:08:23,  1.30it/s, gpt_loss=0.0447, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 271/5616 [03:32<1:08:23,  1.30it/s, gpt_loss=0.0813, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 272/5616 [03:32<1:09:21,  1.28it/s, gpt_loss=0.0813, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 272/5616 [03:33<1:09:21,  1.28it/s, gpt_loss=0.175, lr=5e-6] 
+
Train step of epoch 0:   5%|▍         | 273/5616 [03:33<1:07:48,  1.31it/s, gpt_loss=0.175, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 273/5616 [03:34<1:07:48,  1.31it/s, gpt_loss=0.028, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 274/5616 [03:34<1:09:17,  1.28it/s, gpt_loss=0.028, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 274/5616 [03:35<1:09:17,  1.28it/s, gpt_loss=0.0291, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 275/5616 [03:35<1:07:39,  1.32it/s, gpt_loss=0.0291, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 275/5616 [03:35<1:07:39,  1.32it/s, gpt_loss=0.0425, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 276/5616 [03:35<1:10:26,  1.26it/s, gpt_loss=0.0425, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 276/5616 [03:36<1:10:26,  1.26it/s, gpt_loss=0.0534, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 277/5616 [03:36<1:08:09,  1.31it/s, gpt_loss=0.0534, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 277/5616 [03:37<1:08:09,  1.31it/s, gpt_loss=0.00616, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 278/5616 [03:37<1:06:01,  1.35it/s, gpt_loss=0.00616, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 278/5616 [03:38<1:06:01,  1.35it/s, gpt_loss=0.0145, lr=5e-6] 
+
Train step of epoch 0:   5%|▍         | 279/5616 [03:38<1:05:07,  1.37it/s, gpt_loss=0.0145, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 279/5616 [03:38<1:05:07,  1.37it/s, gpt_loss=0.0387, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 280/5616 [03:38<1:04:22,  1.38it/s, gpt_loss=0.0387, lr=5e-6]
+
Train step of epoch 0:   5%|▍         | 280/5616 [03:39<1:04:22,  1.38it/s, gpt_loss=0.0907, lr=5e-6]
+
Train step of epoch 0:   5%|▌         | 281/5616 [03:39<1:03:17,  1.40it/s, gpt_loss=0.0907, lr=5e-6]
+
Train step of epoch 0:   5%|▌         | 281/5616 [03:40<1:03:17,  1.40it/s, gpt_loss=0.0689, lr=5e-6]
+
Train step of epoch 0:   5%|▌         | 282/5616 [03:40<1:04:56,  1.37it/s, gpt_loss=0.0689, lr=5e-6]
+
Train step of epoch 0:   5%|▌         | 282/5616 [03:40<1:04:56,  1.37it/s, gpt_loss=0.121, lr=5e-6] 
+
Train step of epoch 0:   5%|▌         | 283/5616 [03:40<1:05:05,  1.37it/s, gpt_loss=0.121, lr=5e-6]
+
Train step of epoch 0:   5%|▌         | 283/5616 [03:41<1:05:05,  1.37it/s, gpt_loss=0.0938, lr=5e-6]
+
Train step of epoch 0:   5%|▌         | 284/5616 [03:41<1:09:08,  1.29it/s, gpt_loss=0.0938, lr=5e-6]
+
Train step of epoch 0:   5%|▌         | 284/5616 [03:42<1:09:08,  1.29it/s, gpt_loss=0.0337, lr=5e-6]
+
Train step of epoch 0:   5%|▌         | 285/5616 [03:42<1:13:06,  1.22it/s, gpt_loss=0.0337, lr=5e-6]
+
Train step of epoch 0:   5%|▌         | 285/5616 [03:43<1:13:06,  1.22it/s, gpt_loss=0.0705, lr=5e-6]
+
Train step of epoch 0:   5%|▌         | 286/5616 [03:43<1:11:08,  1.25it/s, gpt_loss=0.0705, lr=5e-6]
+
Train step of epoch 0:   5%|▌         | 286/5616 [03:44<1:11:08,  1.25it/s, gpt_loss=0.0383, lr=5e-6]
+
Train step of epoch 0:   5%|▌         | 287/5616 [03:44<1:09:54,  1.27it/s, gpt_loss=0.0383, lr=5e-6]
+
Train step of epoch 0:   5%|▌         | 287/5616 [03:44<1:09:54,  1.27it/s, gpt_loss=0.0663, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 288/5616 [03:44<1:08:18,  1.30it/s, gpt_loss=0.0663, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 288/5616 [03:45<1:08:18,  1.30it/s, gpt_loss=0.0176, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 289/5616 [03:45<1:05:42,  1.35it/s, gpt_loss=0.0176, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 289/5616 [03:46<1:05:42,  1.35it/s, gpt_loss=0.0969, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 290/5616 [03:46<1:06:44,  1.33it/s, gpt_loss=0.0969, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 290/5616 [03:47<1:06:44,  1.33it/s, gpt_loss=0.0937, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 291/5616 [03:47<1:06:26,  1.34it/s, gpt_loss=0.0937, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 291/5616 [03:48<1:06:26,  1.34it/s, gpt_loss=0.0583, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 292/5616 [03:48<1:19:11,  1.12it/s, gpt_loss=0.0583, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 292/5616 [03:49<1:19:11,  1.12it/s, gpt_loss=0.00707, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 293/5616 [03:49<1:14:46,  1.19it/s, gpt_loss=0.00707, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 293/5616 [03:49<1:14:46,  1.19it/s, gpt_loss=0.0087, lr=4.99e-6] 
+
Train step of epoch 0:   5%|▌         | 294/5616 [03:49<1:09:27,  1.28it/s, gpt_loss=0.0087, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 294/5616 [03:50<1:09:27,  1.28it/s, gpt_loss=0.0309, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 295/5616 [03:50<1:07:46,  1.31it/s, gpt_loss=0.0309, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 295/5616 [03:51<1:07:46,  1.31it/s, gpt_loss=0.0391, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 296/5616 [03:51<1:07:14,  1.32it/s, gpt_loss=0.0391, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 296/5616 [03:52<1:07:14,  1.32it/s, gpt_loss=0.0342, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 297/5616 [03:52<1:08:44,  1.29it/s, gpt_loss=0.0342, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 297/5616 [03:52<1:08:44,  1.29it/s, gpt_loss=0.0434, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 298/5616 [03:52<1:08:35,  1.29it/s, gpt_loss=0.0434, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 298/5616 [03:53<1:08:35,  1.29it/s, gpt_loss=0.0234, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 299/5616 [03:53<1:07:50,  1.31it/s, gpt_loss=0.0234, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 299/5616 [03:54<1:07:50,  1.31it/s, gpt_loss=0.0339, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 300/5616 [03:54<1:10:00,  1.27it/s, gpt_loss=0.0339, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 300/5616 [03:55<1:10:00,  1.27it/s, gpt_loss=0.0304, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 301/5616 [03:55<1:07:31,  1.31it/s, gpt_loss=0.0304, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 301/5616 [03:55<1:07:31,  1.31it/s, gpt_loss=0.0838, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 302/5616 [03:55<1:04:14,  1.38it/s, gpt_loss=0.0838, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 302/5616 [03:56<1:04:14,  1.38it/s, gpt_loss=0.0143, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 303/5616 [03:56<1:02:16,  1.42it/s, gpt_loss=0.0143, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 303/5616 [03:57<1:02:16,  1.42it/s, gpt_loss=0.1, lr=4.99e-6]   
+
Train step of epoch 0:   5%|▌         | 304/5616 [03:57<1:02:17,  1.42it/s, gpt_loss=0.1, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 304/5616 [03:57<1:02:17,  1.42it/s, gpt_loss=0.095, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 305/5616 [03:57<1:03:04,  1.40it/s, gpt_loss=0.095, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 305/5616 [03:58<1:03:04,  1.40it/s, gpt_loss=0.0364, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 306/5616 [03:58<1:01:19,  1.44it/s, gpt_loss=0.0364, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 306/5616 [03:59<1:01:19,  1.44it/s, gpt_loss=0.00292, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 307/5616 [03:59<1:02:18,  1.42it/s, gpt_loss=0.00292, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 307/5616 [04:00<1:02:18,  1.42it/s, gpt_loss=0.172, lr=4.99e-6]  
+
Train step of epoch 0:   5%|▌         | 308/5616 [04:00<1:06:17,  1.33it/s, gpt_loss=0.172, lr=4.99e-6]
+
Train step of epoch 0:   5%|▌         | 308/5616 [04:00<1:06:17,  1.33it/s, gpt_loss=0.0951, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 309/5616 [04:00<1:04:40,  1.37it/s, gpt_loss=0.0951, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 309/5616 [04:01<1:04:40,  1.37it/s, gpt_loss=0.135, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▌         | 310/5616 [04:01<1:06:37,  1.33it/s, gpt_loss=0.135, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 310/5616 [04:02<1:06:37,  1.33it/s, gpt_loss=0.0197, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 311/5616 [04:02<1:10:05,  1.26it/s, gpt_loss=0.0197, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 311/5616 [04:03<1:10:05,  1.26it/s, gpt_loss=0.0331, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 312/5616 [04:03<1:14:56,  1.18it/s, gpt_loss=0.0331, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 312/5616 [04:04<1:14:56,  1.18it/s, gpt_loss=0.172, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▌         | 313/5616 [04:04<1:12:51,  1.21it/s, gpt_loss=0.172, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 313/5616 [04:04<1:12:51,  1.21it/s, gpt_loss=0.0744, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 314/5616 [04:04<1:09:39,  1.27it/s, gpt_loss=0.0744, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 314/5616 [04:05<1:09:39,  1.27it/s, gpt_loss=0.117, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▌         | 315/5616 [04:05<1:15:47,  1.17it/s, gpt_loss=0.117, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 315/5616 [04:06<1:15:47,  1.17it/s, gpt_loss=0.0258, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 316/5616 [04:06<1:14:42,  1.18it/s, gpt_loss=0.0258, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 316/5616 [04:07<1:14:42,  1.18it/s, gpt_loss=0.116, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▌         | 317/5616 [04:07<1:17:06,  1.15it/s, gpt_loss=0.116, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 317/5616 [04:08<1:17:06,  1.15it/s, gpt_loss=0.106, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 318/5616 [04:08<1:13:10,  1.21it/s, gpt_loss=0.106, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 318/5616 [04:09<1:13:10,  1.21it/s, gpt_loss=0.0531, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 319/5616 [04:09<1:10:58,  1.24it/s, gpt_loss=0.0531, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 319/5616 [04:10<1:10:58,  1.24it/s, gpt_loss=0.013, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▌         | 320/5616 [04:10<1:19:12,  1.11it/s, gpt_loss=0.013, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 320/5616 [04:10<1:19:12,  1.11it/s, gpt_loss=0.214, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 321/5616 [04:10<1:13:31,  1.20it/s, gpt_loss=0.214, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 321/5616 [04:11<1:13:31,  1.20it/s, gpt_loss=0.0619, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 322/5616 [04:11<1:09:27,  1.27it/s, gpt_loss=0.0619, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 322/5616 [04:12<1:09:27,  1.27it/s, gpt_loss=0.00929, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 323/5616 [04:12<1:04:54,  1.36it/s, gpt_loss=0.00929, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 323/5616 [04:13<1:04:54,  1.36it/s, gpt_loss=0.0323, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▌         | 324/5616 [04:13<1:08:51,  1.28it/s, gpt_loss=0.0323, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 324/5616 [04:14<1:08:51,  1.28it/s, gpt_loss=0.137, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▌         | 325/5616 [04:14<1:12:33,  1.22it/s, gpt_loss=0.137, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 325/5616 [04:14<1:12:33,  1.22it/s, gpt_loss=0.078, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 326/5616 [04:14<1:11:18,  1.24it/s, gpt_loss=0.078, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 326/5616 [04:15<1:11:18,  1.24it/s, gpt_loss=0.057, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 327/5616 [04:15<1:12:15,  1.22it/s, gpt_loss=0.057, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 327/5616 [04:16<1:12:15,  1.22it/s, gpt_loss=0.0593, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 328/5616 [04:16<1:16:31,  1.15it/s, gpt_loss=0.0593, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 328/5616 [04:17<1:16:31,  1.15it/s, gpt_loss=0.0807, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 329/5616 [04:17<1:11:17,  1.24it/s, gpt_loss=0.0807, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 329/5616 [04:18<1:11:17,  1.24it/s, gpt_loss=0.027, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▌         | 330/5616 [04:18<1:10:22,  1.25it/s, gpt_loss=0.027, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 330/5616 [04:18<1:10:22,  1.25it/s, gpt_loss=0.0693, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 331/5616 [04:18<1:07:12,  1.31it/s, gpt_loss=0.0693, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 331/5616 [04:19<1:07:12,  1.31it/s, gpt_loss=0.0555, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 332/5616 [04:19<1:07:24,  1.31it/s, gpt_loss=0.0555, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 332/5616 [04:20<1:07:24,  1.31it/s, gpt_loss=0.0722, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 333/5616 [04:20<1:08:10,  1.29it/s, gpt_loss=0.0722, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 333/5616 [04:21<1:08:10,  1.29it/s, gpt_loss=0.142, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▌         | 334/5616 [04:21<1:06:00,  1.33it/s, gpt_loss=0.142, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 334/5616 [04:21<1:06:00,  1.33it/s, gpt_loss=0.225, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 335/5616 [04:21<1:06:22,  1.33it/s, gpt_loss=0.225, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 335/5616 [04:22<1:06:22,  1.33it/s, gpt_loss=0.0181, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 336/5616 [04:22<1:07:34,  1.30it/s, gpt_loss=0.0181, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 336/5616 [04:23<1:07:34,  1.30it/s, gpt_loss=0.176, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▌         | 337/5616 [04:23<1:08:36,  1.28it/s, gpt_loss=0.176, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 337/5616 [04:24<1:08:36,  1.28it/s, gpt_loss=0.145, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 338/5616 [04:24<1:10:14,  1.25it/s, gpt_loss=0.145, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 338/5616 [04:25<1:10:14,  1.25it/s, gpt_loss=0.0878, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 339/5616 [04:25<1:11:39,  1.23it/s, gpt_loss=0.0878, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 339/5616 [04:25<1:11:39,  1.23it/s, gpt_loss=0.0937, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 340/5616 [04:25<1:09:54,  1.26it/s, gpt_loss=0.0937, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 340/5616 [04:26<1:09:54,  1.26it/s, gpt_loss=0.0908, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 341/5616 [04:26<1:07:33,  1.30it/s, gpt_loss=0.0908, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 341/5616 [04:27<1:07:33,  1.30it/s, gpt_loss=0.0676, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 342/5616 [04:27<1:04:56,  1.35it/s, gpt_loss=0.0676, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 342/5616 [04:27<1:04:56,  1.35it/s, gpt_loss=0.0692, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 343/5616 [04:27<1:03:26,  1.39it/s, gpt_loss=0.0692, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 343/5616 [04:28<1:03:26,  1.39it/s, gpt_loss=0.0559, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 344/5616 [04:28<1:08:36,  1.28it/s, gpt_loss=0.0559, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 344/5616 [04:29<1:08:36,  1.28it/s, gpt_loss=0.064, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▌         | 345/5616 [04:29<1:09:17,  1.27it/s, gpt_loss=0.064, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 345/5616 [04:30<1:09:17,  1.27it/s, gpt_loss=0.0605, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 346/5616 [04:30<1:13:08,  1.20it/s, gpt_loss=0.0605, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 346/5616 [04:31<1:13:08,  1.20it/s, gpt_loss=0.086, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▌         | 347/5616 [04:31<1:10:21,  1.25it/s, gpt_loss=0.086, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 347/5616 [04:32<1:10:21,  1.25it/s, gpt_loss=0.0867, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 348/5616 [04:32<1:09:50,  1.26it/s, gpt_loss=0.0867, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 348/5616 [04:32<1:09:50,  1.26it/s, gpt_loss=0.063, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▌         | 349/5616 [04:32<1:06:25,  1.32it/s, gpt_loss=0.063, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 349/5616 [04:33<1:06:25,  1.32it/s, gpt_loss=0.0497, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 350/5616 [04:33<1:05:58,  1.33it/s, gpt_loss=0.0497, lr=4.99e-6]
+
Train step of epoch 0:   6%|▌         | 350/5616 [04:34<1:05:58,  1.33it/s, gpt_loss=0.0127, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 351/5616 [04:34<1:05:48,  1.33it/s, gpt_loss=0.0127, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 351/5616 [04:35<1:05:48,  1.33it/s, gpt_loss=0.0752, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 352/5616 [04:35<1:07:18,  1.30it/s, gpt_loss=0.0752, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 352/5616 [04:35<1:07:18,  1.30it/s, gpt_loss=0.0603, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 353/5616 [04:35<1:04:14,  1.37it/s, gpt_loss=0.0603, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 353/5616 [04:36<1:04:14,  1.37it/s, gpt_loss=0.0842, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 354/5616 [04:36<1:02:42,  1.40it/s, gpt_loss=0.0842, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 354/5616 [04:37<1:02:42,  1.40it/s, gpt_loss=0.086, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▋         | 355/5616 [04:37<1:03:00,  1.39it/s, gpt_loss=0.086, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 355/5616 [04:38<1:03:00,  1.39it/s, gpt_loss=0.0735, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 356/5616 [04:38<1:07:49,  1.29it/s, gpt_loss=0.0735, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 356/5616 [04:38<1:07:49,  1.29it/s, gpt_loss=0.0225, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 357/5616 [04:38<1:06:15,  1.32it/s, gpt_loss=0.0225, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 357/5616 [04:39<1:06:15,  1.32it/s, gpt_loss=0.0345, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 358/5616 [04:39<1:03:58,  1.37it/s, gpt_loss=0.0345, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 358/5616 [04:40<1:03:58,  1.37it/s, gpt_loss=0.0502, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 359/5616 [04:40<1:01:53,  1.42it/s, gpt_loss=0.0502, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 359/5616 [04:40<1:01:53,  1.42it/s, gpt_loss=0.0733, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 360/5616 [04:40<1:05:04,  1.35it/s, gpt_loss=0.0733, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 360/5616 [04:41<1:05:04,  1.35it/s, gpt_loss=0.104, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▋         | 361/5616 [04:41<1:05:10,  1.34it/s, gpt_loss=0.104, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 361/5616 [04:42<1:05:10,  1.34it/s, gpt_loss=0.0351, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 362/5616 [04:42<1:04:06,  1.37it/s, gpt_loss=0.0351, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 362/5616 [04:42<1:04:06,  1.37it/s, gpt_loss=0.149, lr=4.99e-6] 
+
Train step of epoch 0:   6%|▋         | 363/5616 [04:42<1:01:58,  1.41it/s, gpt_loss=0.149, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 363/5616 [04:43<1:01:58,  1.41it/s, gpt_loss=0.0513, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 364/5616 [04:43<1:03:53,  1.37it/s, gpt_loss=0.0513, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 364/5616 [04:44<1:03:53,  1.37it/s, gpt_loss=0.0691, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 365/5616 [04:44<1:04:27,  1.36it/s, gpt_loss=0.0691, lr=4.99e-6]
+
Train step of epoch 0:   6%|▋         | 365/5616 [04:45<1:04:27,  1.36it/s, gpt_loss=0.0121, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 366/5616 [04:45<1:03:32,  1.38it/s, gpt_loss=0.0121, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 366/5616 [04:45<1:03:32,  1.38it/s, gpt_loss=0.00413, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 367/5616 [04:45<1:02:57,  1.39it/s, gpt_loss=0.00413, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 367/5616 [04:46<1:02:57,  1.39it/s, gpt_loss=0.0361, lr=4.99e-6] 
+
Train step of epoch 0:   7%|▋         | 368/5616 [04:46<1:06:17,  1.32it/s, gpt_loss=0.0361, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 368/5616 [04:47<1:06:17,  1.32it/s, gpt_loss=0.0221, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 369/5616 [04:47<1:07:45,  1.29it/s, gpt_loss=0.0221, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 369/5616 [04:48<1:07:45,  1.29it/s, gpt_loss=0.0382, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 370/5616 [04:48<1:05:15,  1.34it/s, gpt_loss=0.0382, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 370/5616 [04:48<1:05:15,  1.34it/s, gpt_loss=0.0395, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 371/5616 [04:48<1:03:34,  1.38it/s, gpt_loss=0.0395, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 371/5616 [04:49<1:03:34,  1.38it/s, gpt_loss=0.0416, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 372/5616 [04:49<1:05:30,  1.33it/s, gpt_loss=0.0416, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 372/5616 [04:50<1:05:30,  1.33it/s, gpt_loss=0.00299, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 373/5616 [04:50<1:07:59,  1.29it/s, gpt_loss=0.00299, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 373/5616 [04:51<1:07:59,  1.29it/s, gpt_loss=0.0284, lr=4.99e-6] 
+
Train step of epoch 0:   7%|▋         | 374/5616 [04:51<1:07:55,  1.29it/s, gpt_loss=0.0284, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 374/5616 [04:52<1:07:55,  1.29it/s, gpt_loss=0.0239, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 375/5616 [04:52<1:09:57,  1.25it/s, gpt_loss=0.0239, lr=4.99e-6]
+
Train step of epoch 0:   7%|▋         | 375/5616 [04:53<1:09:57,  1.25it/s, gpt_loss=0.161, lr=4.98e-6] 
+
Train step of epoch 0:   7%|▋         | 376/5616 [04:53<1:14:53,  1.17it/s, gpt_loss=0.161, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 376/5616 [04:54<1:14:53,  1.17it/s, gpt_loss=0.0472, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 377/5616 [04:54<1:14:42,  1.17it/s, gpt_loss=0.0472, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 377/5616 [04:54<1:14:42,  1.17it/s, gpt_loss=0.162, lr=4.98e-6] 
+
Train step of epoch 0:   7%|▋         | 378/5616 [04:54<1:12:42,  1.20it/s, gpt_loss=0.162, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 378/5616 [04:55<1:12:42,  1.20it/s, gpt_loss=0.0916, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 379/5616 [04:55<1:08:20,  1.28it/s, gpt_loss=0.0916, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 379/5616 [04:56<1:08:20,  1.28it/s, gpt_loss=0.0161, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 380/5616 [04:56<1:11:56,  1.21it/s, gpt_loss=0.0161, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 380/5616 [04:57<1:11:56,  1.21it/s, gpt_loss=0.178, lr=4.98e-6] 
+
Train step of epoch 0:   7%|▋         | 381/5616 [04:57<1:09:43,  1.25it/s, gpt_loss=0.178, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 381/5616 [04:57<1:09:43,  1.25it/s, gpt_loss=0.0114, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 382/5616 [04:57<1:07:10,  1.30it/s, gpt_loss=0.0114, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 382/5616 [04:58<1:07:10,  1.30it/s, gpt_loss=0.0194, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 383/5616 [04:58<1:10:49,  1.23it/s, gpt_loss=0.0194, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 383/5616 [04:59<1:10:49,  1.23it/s, gpt_loss=0.0587, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 384/5616 [04:59<1:13:26,  1.19it/s, gpt_loss=0.0587, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 384/5616 [05:00<1:13:26,  1.19it/s, gpt_loss=0.069, lr=4.98e-6] 
+
Train step of epoch 0:   7%|▋         | 385/5616 [05:00<1:07:42,  1.29it/s, gpt_loss=0.069, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 385/5616 [05:01<1:07:42,  1.29it/s, gpt_loss=0.111, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 386/5616 [05:01<1:07:24,  1.29it/s, gpt_loss=0.111, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 386/5616 [05:01<1:07:24,  1.29it/s, gpt_loss=0.0304, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 387/5616 [05:01<1:07:18,  1.29it/s, gpt_loss=0.0304, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 387/5616 [05:02<1:07:18,  1.29it/s, gpt_loss=0.00967, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 388/5616 [05:02<1:07:50,  1.28it/s, gpt_loss=0.00967, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 388/5616 [05:03<1:07:50,  1.28it/s, gpt_loss=0.0186, lr=4.98e-6] 
+
Train step of epoch 0:   7%|▋         | 389/5616 [05:03<1:05:47,  1.32it/s, gpt_loss=0.0186, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 389/5616 [05:04<1:05:47,  1.32it/s, gpt_loss=0.11, lr=4.98e-6]  
+
Train step of epoch 0:   7%|▋         | 390/5616 [05:04<1:03:32,  1.37it/s, gpt_loss=0.11, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 390/5616 [05:04<1:03:32,  1.37it/s, gpt_loss=0.0103, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 391/5616 [05:04<1:02:35,  1.39it/s, gpt_loss=0.0103, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 391/5616 [05:05<1:02:35,  1.39it/s, gpt_loss=0.0334, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 392/5616 [05:05<1:04:55,  1.34it/s, gpt_loss=0.0334, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 392/5616 [05:06<1:04:55,  1.34it/s, gpt_loss=0.0164, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 393/5616 [05:06<1:05:40,  1.33it/s, gpt_loss=0.0164, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 393/5616 [05:07<1:05:40,  1.33it/s, gpt_loss=0.0811, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 394/5616 [05:07<1:06:14,  1.31it/s, gpt_loss=0.0811, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 394/5616 [05:07<1:06:14,  1.31it/s, gpt_loss=0.0599, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 395/5616 [05:07<1:04:32,  1.35it/s, gpt_loss=0.0599, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 395/5616 [05:08<1:04:32,  1.35it/s, gpt_loss=0.0444, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 396/5616 [05:08<1:06:03,  1.32it/s, gpt_loss=0.0444, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 396/5616 [05:09<1:06:03,  1.32it/s, gpt_loss=0.0176, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 397/5616 [05:09<1:05:06,  1.34it/s, gpt_loss=0.0176, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 397/5616 [05:09<1:05:06,  1.34it/s, gpt_loss=0.0309, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 398/5616 [05:09<1:03:34,  1.37it/s, gpt_loss=0.0309, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 398/5616 [05:10<1:03:34,  1.37it/s, gpt_loss=0.029, lr=4.98e-6] 
+
Train step of epoch 0:   7%|▋         | 399/5616 [05:10<1:02:50,  1.38it/s, gpt_loss=0.029, lr=4.98e-6][2025-06-30 06:43:19,989] [INFO] [logging.py:128:log_dist] [Rank 0] step=100, skipped=0, lr=[4.9805527088379165e-06, 4.9805527088379165e-06], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 06:43:19,990] [INFO] [timer.py:264:stop] epoch=0/micro_step=400/global_step=100, RunningAvgSamplesPerSec=41.97609346309232, CurrSamplesPerSec=44.93966911829873, MemAllocated=15.92GB, MaxMemAllocated=51.79GB
+
+
Train step of epoch 0:   7%|▋         | 399/5616 [05:11<1:02:50,  1.38it/s, gpt_loss=0.106, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 400/5616 [05:11<1:03:58,  1.36it/s, gpt_loss=0.106, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 400/5616 [05:12<1:03:58,  1.36it/s, gpt_loss=0.0204, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 401/5616 [05:12<1:04:23,  1.35it/s, gpt_loss=0.0204, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 401/5616 [05:12<1:04:23,  1.35it/s, gpt_loss=0.0583, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 402/5616 [05:12<1:03:25,  1.37it/s, gpt_loss=0.0583, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 402/5616 [05:13<1:03:25,  1.37it/s, gpt_loss=0.118, lr=4.98e-6] 
+
Train step of epoch 0:   7%|▋         | 403/5616 [05:13<1:04:46,  1.34it/s, gpt_loss=0.118, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 403/5616 [05:14<1:04:46,  1.34it/s, gpt_loss=0.0546, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 404/5616 [05:14<1:06:25,  1.31it/s, gpt_loss=0.0546, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 404/5616 [05:15<1:06:25,  1.31it/s, gpt_loss=0.0823, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 405/5616 [05:15<1:04:52,  1.34it/s, gpt_loss=0.0823, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 405/5616 [05:15<1:04:52,  1.34it/s, gpt_loss=0.135, lr=4.98e-6] 
+
Train step of epoch 0:   7%|▋         | 406/5616 [05:15<1:01:32,  1.41it/s, gpt_loss=0.135, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 406/5616 [05:16<1:01:32,  1.41it/s, gpt_loss=0.0856, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 407/5616 [05:16<1:01:27,  1.41it/s, gpt_loss=0.0856, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 407/5616 [05:17<1:01:27,  1.41it/s, gpt_loss=0.0412, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 408/5616 [05:17<1:02:57,  1.38it/s, gpt_loss=0.0412, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 408/5616 [05:17<1:02:57,  1.38it/s, gpt_loss=0.0195, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 409/5616 [05:17<1:02:03,  1.40it/s, gpt_loss=0.0195, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 409/5616 [05:18<1:02:03,  1.40it/s, gpt_loss=0.157, lr=4.98e-6] 
+
Train step of epoch 0:   7%|▋         | 410/5616 [05:18<1:02:17,  1.39it/s, gpt_loss=0.157, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 410/5616 [05:19<1:02:17,  1.39it/s, gpt_loss=0.024, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 411/5616 [05:19<1:00:57,  1.42it/s, gpt_loss=0.024, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 411/5616 [05:20<1:00:57,  1.42it/s, gpt_loss=0.0687, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 412/5616 [05:20<1:02:26,  1.39it/s, gpt_loss=0.0687, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 412/5616 [05:20<1:02:26,  1.39it/s, gpt_loss=0.0172, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 413/5616 [05:20<1:03:47,  1.36it/s, gpt_loss=0.0172, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 413/5616 [05:21<1:03:47,  1.36it/s, gpt_loss=0.0266, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 414/5616 [05:21<1:02:06,  1.40it/s, gpt_loss=0.0266, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 414/5616 [05:22<1:02:06,  1.40it/s, gpt_loss=0.0388, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 415/5616 [05:22<1:15:25,  1.15it/s, gpt_loss=0.0388, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 415/5616 [05:23<1:15:25,  1.15it/s, gpt_loss=0.051, lr=4.98e-6] 
+
Train step of epoch 0:   7%|▋         | 416/5616 [05:23<1:14:53,  1.16it/s, gpt_loss=0.051, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 416/5616 [05:24<1:14:53,  1.16it/s, gpt_loss=0.0774, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 417/5616 [05:24<1:10:36,  1.23it/s, gpt_loss=0.0774, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 417/5616 [05:25<1:10:36,  1.23it/s, gpt_loss=0.0219, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 418/5616 [05:25<1:06:46,  1.30it/s, gpt_loss=0.0219, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 418/5616 [05:25<1:06:46,  1.30it/s, gpt_loss=0.0767, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 419/5616 [05:25<1:04:46,  1.34it/s, gpt_loss=0.0767, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 419/5616 [05:26<1:04:46,  1.34it/s, gpt_loss=0.0496, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 420/5616 [05:26<1:05:07,  1.33it/s, gpt_loss=0.0496, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 420/5616 [05:27<1:05:07,  1.33it/s, gpt_loss=0.0626, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 421/5616 [05:27<1:04:27,  1.34it/s, gpt_loss=0.0626, lr=4.98e-6]
+
Train step of epoch 0:   7%|▋         | 421/5616 [05:28<1:04:27,  1.34it/s, gpt_loss=0.0242, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 422/5616 [05:28<1:05:56,  1.31it/s, gpt_loss=0.0242, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 422/5616 [05:28<1:05:56,  1.31it/s, gpt_loss=0.0337, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 423/5616 [05:28<1:05:00,  1.33it/s, gpt_loss=0.0337, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 423/5616 [05:29<1:05:00,  1.33it/s, gpt_loss=0.05, lr=4.98e-6]  
+
Train step of epoch 0:   8%|▊         | 424/5616 [05:29<1:06:18,  1.31it/s, gpt_loss=0.05, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 424/5616 [05:30<1:06:18,  1.31it/s, gpt_loss=0.148, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 425/5616 [05:30<1:02:39,  1.38it/s, gpt_loss=0.148, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 425/5616 [05:30<1:02:39,  1.38it/s, gpt_loss=0.121, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 426/5616 [05:30<1:02:16,  1.39it/s, gpt_loss=0.121, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 426/5616 [05:31<1:02:16,  1.39it/s, gpt_loss=0.0587, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 427/5616 [05:31<1:01:01,  1.42it/s, gpt_loss=0.0587, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 427/5616 [05:32<1:01:01,  1.42it/s, gpt_loss=0.116, lr=4.98e-6] 
+
Train step of epoch 0:   8%|▊         | 428/5616 [05:32<1:11:59,  1.20it/s, gpt_loss=0.116, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 428/5616 [05:33<1:11:59,  1.20it/s, gpt_loss=0.0196, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 429/5616 [05:33<1:11:19,  1.21it/s, gpt_loss=0.0196, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 429/5616 [05:34<1:11:19,  1.21it/s, gpt_loss=0.0257, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 430/5616 [05:34<1:09:11,  1.25it/s, gpt_loss=0.0257, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 430/5616 [05:34<1:09:11,  1.25it/s, gpt_loss=0.0566, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 431/5616 [05:34<1:06:00,  1.31it/s, gpt_loss=0.0566, lr=4.98e-6]
+
Train step of epoch 0:   8%|▊         | 431/5616 [05:35<1:06:00,  1.31it/s, gpt_loss=0.0348, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 432/5616 [05:35<1:06:29,  1.30it/s, gpt_loss=0.0348, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 432/5616 [05:36<1:06:29,  1.30it/s, gpt_loss=0.015, lr=4.97e-6] 
+
Train step of epoch 0:   8%|▊         | 433/5616 [05:36<1:13:19,  1.18it/s, gpt_loss=0.015, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 433/5616 [05:37<1:13:19,  1.18it/s, gpt_loss=0.009, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 434/5616 [05:37<1:08:43,  1.26it/s, gpt_loss=0.009, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 434/5616 [05:38<1:08:43,  1.26it/s, gpt_loss=0.135, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 435/5616 [05:38<1:04:23,  1.34it/s, gpt_loss=0.135, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 435/5616 [05:38<1:04:23,  1.34it/s, gpt_loss=0.00656, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 436/5616 [05:38<1:05:24,  1.32it/s, gpt_loss=0.00656, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 436/5616 [05:39<1:05:24,  1.32it/s, gpt_loss=0.164, lr=4.97e-6]  
+
Train step of epoch 0:   8%|▊         | 437/5616 [05:39<1:03:04,  1.37it/s, gpt_loss=0.164, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 437/5616 [05:40<1:03:04,  1.37it/s, gpt_loss=0.0167, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 438/5616 [05:40<1:02:48,  1.37it/s, gpt_loss=0.0167, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 438/5616 [05:40<1:02:48,  1.37it/s, gpt_loss=0.04, lr=4.97e-6]  
+
Train step of epoch 0:   8%|▊         | 439/5616 [05:40<1:03:58,  1.35it/s, gpt_loss=0.04, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 439/5616 [05:41<1:03:58,  1.35it/s, gpt_loss=0.0724, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 440/5616 [05:41<1:04:32,  1.34it/s, gpt_loss=0.0724, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 440/5616 [05:42<1:04:32,  1.34it/s, gpt_loss=0.136, lr=4.97e-6] 
+
Train step of epoch 0:   8%|▊         | 441/5616 [05:42<1:05:04,  1.33it/s, gpt_loss=0.136, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 441/5616 [05:43<1:05:04,  1.33it/s, gpt_loss=0.133, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 442/5616 [05:43<1:02:46,  1.37it/s, gpt_loss=0.133, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 442/5616 [05:43<1:02:46,  1.37it/s, gpt_loss=0.0801, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 443/5616 [05:43<1:02:39,  1.38it/s, gpt_loss=0.0801, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 443/5616 [05:44<1:02:39,  1.38it/s, gpt_loss=0.0324, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 444/5616 [05:44<1:09:06,  1.25it/s, gpt_loss=0.0324, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 444/5616 [05:45<1:09:06,  1.25it/s, gpt_loss=0.0376, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 445/5616 [05:45<1:05:51,  1.31it/s, gpt_loss=0.0376, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 445/5616 [05:46<1:05:51,  1.31it/s, gpt_loss=0.0561, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 446/5616 [05:46<1:08:16,  1.26it/s, gpt_loss=0.0561, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 446/5616 [05:47<1:08:16,  1.26it/s, gpt_loss=0.0211, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 447/5616 [05:47<1:07:51,  1.27it/s, gpt_loss=0.0211, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 447/5616 [05:47<1:07:51,  1.27it/s, gpt_loss=0.0347, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 448/5616 [05:47<1:07:15,  1.28it/s, gpt_loss=0.0347, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 448/5616 [05:48<1:07:15,  1.28it/s, gpt_loss=0.0161, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 449/5616 [05:48<1:03:50,  1.35it/s, gpt_loss=0.0161, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 449/5616 [05:49<1:03:50,  1.35it/s, gpt_loss=0.0556, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 450/5616 [05:49<1:01:51,  1.39it/s, gpt_loss=0.0556, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 450/5616 [05:50<1:01:51,  1.39it/s, gpt_loss=0.105, lr=4.97e-6] 
+
Train step of epoch 0:   8%|▊         | 451/5616 [05:50<1:03:19,  1.36it/s, gpt_loss=0.105, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 451/5616 [05:50<1:03:19,  1.36it/s, gpt_loss=0.0116, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 452/5616 [05:50<1:06:00,  1.30it/s, gpt_loss=0.0116, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 452/5616 [05:51<1:06:00,  1.30it/s, gpt_loss=0.162, lr=4.97e-6] 
+
Train step of epoch 0:   8%|▊         | 453/5616 [05:51<1:06:45,  1.29it/s, gpt_loss=0.162, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 453/5616 [05:52<1:06:45,  1.29it/s, gpt_loss=0.0673, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 454/5616 [05:52<1:03:29,  1.36it/s, gpt_loss=0.0673, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 454/5616 [05:52<1:03:29,  1.36it/s, gpt_loss=0.0594, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 455/5616 [05:52<1:01:29,  1.40it/s, gpt_loss=0.0594, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 455/5616 [05:53<1:01:29,  1.40it/s, gpt_loss=0.13, lr=4.97e-6]  
+
Train step of epoch 0:   8%|▊         | 456/5616 [05:53<1:07:28,  1.27it/s, gpt_loss=0.13, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 456/5616 [05:54<1:07:28,  1.27it/s, gpt_loss=0.0427, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 457/5616 [05:54<1:07:02,  1.28it/s, gpt_loss=0.0427, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 457/5616 [05:55<1:07:02,  1.28it/s, gpt_loss=0.0967, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 458/5616 [05:55<1:06:36,  1.29it/s, gpt_loss=0.0967, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 458/5616 [05:56<1:06:36,  1.29it/s, gpt_loss=0.0769, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 459/5616 [05:56<1:03:41,  1.35it/s, gpt_loss=0.0769, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 459/5616 [05:56<1:03:41,  1.35it/s, gpt_loss=0.0923, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 460/5616 [05:56<1:04:47,  1.33it/s, gpt_loss=0.0923, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 460/5616 [05:57<1:04:47,  1.33it/s, gpt_loss=0.04, lr=4.97e-6]  
+
Train step of epoch 0:   8%|▊         | 461/5616 [05:57<1:03:23,  1.36it/s, gpt_loss=0.04, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 461/5616 [05:58<1:03:23,  1.36it/s, gpt_loss=0.107, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 462/5616 [05:58<1:05:57,  1.30it/s, gpt_loss=0.107, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 462/5616 [05:59<1:05:57,  1.30it/s, gpt_loss=0.0699, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 463/5616 [05:59<1:08:49,  1.25it/s, gpt_loss=0.0699, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 463/5616 [06:00<1:08:49,  1.25it/s, gpt_loss=0.0585, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 464/5616 [06:00<1:10:56,  1.21it/s, gpt_loss=0.0585, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 464/5616 [06:00<1:10:56,  1.21it/s, gpt_loss=0.0804, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 465/5616 [06:00<1:08:58,  1.24it/s, gpt_loss=0.0804, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 465/5616 [06:01<1:08:58,  1.24it/s, gpt_loss=0.0178, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 466/5616 [06:01<1:08:44,  1.25it/s, gpt_loss=0.0178, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 466/5616 [06:02<1:08:44,  1.25it/s, gpt_loss=0.0472, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 467/5616 [06:02<1:06:37,  1.29it/s, gpt_loss=0.0472, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 467/5616 [06:03<1:06:37,  1.29it/s, gpt_loss=0.0683, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 468/5616 [06:03<1:07:43,  1.27it/s, gpt_loss=0.0683, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 468/5616 [06:04<1:07:43,  1.27it/s, gpt_loss=0.0305, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 469/5616 [06:04<1:06:09,  1.30it/s, gpt_loss=0.0305, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 469/5616 [06:04<1:06:09,  1.30it/s, gpt_loss=0.0307, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 470/5616 [06:04<1:03:45,  1.35it/s, gpt_loss=0.0307, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 470/5616 [06:05<1:03:45,  1.35it/s, gpt_loss=0.121, lr=4.97e-6] 
+
Train step of epoch 0:   8%|▊         | 471/5616 [06:05<1:05:53,  1.30it/s, gpt_loss=0.121, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 471/5616 [06:06<1:05:53,  1.30it/s, gpt_loss=0.0544, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 472/5616 [06:06<1:07:13,  1.28it/s, gpt_loss=0.0544, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 472/5616 [06:07<1:07:13,  1.28it/s, gpt_loss=0.0856, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 473/5616 [06:07<1:05:11,  1.31it/s, gpt_loss=0.0856, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 473/5616 [06:07<1:05:11,  1.31it/s, gpt_loss=0.0237, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 474/5616 [06:07<1:08:29,  1.25it/s, gpt_loss=0.0237, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 474/5616 [06:08<1:08:29,  1.25it/s, gpt_loss=0.0272, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 475/5616 [06:08<1:06:21,  1.29it/s, gpt_loss=0.0272, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 475/5616 [06:09<1:06:21,  1.29it/s, gpt_loss=0.0951, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 476/5616 [06:09<1:07:19,  1.27it/s, gpt_loss=0.0951, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 476/5616 [06:10<1:07:19,  1.27it/s, gpt_loss=0.0298, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 477/5616 [06:10<1:07:35,  1.27it/s, gpt_loss=0.0298, lr=4.97e-6]
+
Train step of epoch 0:   8%|▊         | 477/5616 [06:10<1:07:35,  1.27it/s, gpt_loss=0.0434, lr=4.97e-6]
+
Train step of epoch 0:   9%|▊         | 478/5616 [06:10<1:03:26,  1.35it/s, gpt_loss=0.0434, lr=4.97e-6]
+
Train step of epoch 0:   9%|▊         | 478/5616 [06:11<1:03:26,  1.35it/s, gpt_loss=0.0514, lr=4.97e-6]
+
Train step of epoch 0:   9%|▊         | 479/5616 [06:11<1:02:29,  1.37it/s, gpt_loss=0.0514, lr=4.97e-6]
+
Train step of epoch 0:   9%|▊         | 479/5616 [06:12<1:02:29,  1.37it/s, gpt_loss=0.0498, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 480/5616 [06:12<1:03:59,  1.34it/s, gpt_loss=0.0498, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 480/5616 [06:13<1:03:59,  1.34it/s, gpt_loss=0.141, lr=4.96e-6] 
+
Train step of epoch 0:   9%|▊         | 481/5616 [06:13<1:06:12,  1.29it/s, gpt_loss=0.141, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 481/5616 [06:13<1:06:12,  1.29it/s, gpt_loss=0.084, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 482/5616 [06:13<1:06:03,  1.30it/s, gpt_loss=0.084, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 482/5616 [06:14<1:06:03,  1.30it/s, gpt_loss=0.039, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 483/5616 [06:14<1:06:23,  1.29it/s, gpt_loss=0.039, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 483/5616 [06:15<1:06:23,  1.29it/s, gpt_loss=0.0657, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 484/5616 [06:15<1:05:43,  1.30it/s, gpt_loss=0.0657, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 484/5616 [06:16<1:05:43,  1.30it/s, gpt_loss=0.0491, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 485/5616 [06:16<1:04:05,  1.33it/s, gpt_loss=0.0491, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 485/5616 [06:16<1:04:05,  1.33it/s, gpt_loss=0.0232, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 486/5616 [06:16<1:03:08,  1.35it/s, gpt_loss=0.0232, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 486/5616 [06:17<1:03:08,  1.35it/s, gpt_loss=0.0544, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 487/5616 [06:17<1:02:12,  1.37it/s, gpt_loss=0.0544, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 487/5616 [06:18<1:02:12,  1.37it/s, gpt_loss=0.0781, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 488/5616 [06:18<1:02:08,  1.38it/s, gpt_loss=0.0781, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 488/5616 [06:19<1:02:08,  1.38it/s, gpt_loss=0.0317, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 489/5616 [06:19<1:03:18,  1.35it/s, gpt_loss=0.0317, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 489/5616 [06:19<1:03:18,  1.35it/s, gpt_loss=0.0301, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 490/5616 [06:19<1:02:51,  1.36it/s, gpt_loss=0.0301, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 490/5616 [06:20<1:02:51,  1.36it/s, gpt_loss=0.0284, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 491/5616 [06:20<1:02:39,  1.36it/s, gpt_loss=0.0284, lr=4.96e-6]
+
Train step of epoch 0:   9%|▊         | 491/5616 [06:21<1:02:39,  1.36it/s, gpt_loss=0.113, lr=4.96e-6] 
+
Train step of epoch 0:   9%|▉         | 492/5616 [06:21<1:02:34,  1.36it/s, gpt_loss=0.113, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 492/5616 [06:22<1:02:34,  1.36it/s, gpt_loss=0.1, lr=4.96e-6]  
+
Train step of epoch 0:   9%|▉         | 493/5616 [06:22<1:05:22,  1.31it/s, gpt_loss=0.1, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 493/5616 [06:22<1:05:22,  1.31it/s, gpt_loss=0.0462, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 494/5616 [06:22<1:06:05,  1.29it/s, gpt_loss=0.0462, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 494/5616 [06:23<1:06:05,  1.29it/s, gpt_loss=0.128, lr=4.96e-6] 
+
Train step of epoch 0:   9%|▉         | 495/5616 [06:23<1:04:06,  1.33it/s, gpt_loss=0.128, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 495/5616 [06:24<1:04:06,  1.33it/s, gpt_loss=0.123, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 496/5616 [06:24<1:06:59,  1.27it/s, gpt_loss=0.123, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 496/5616 [06:25<1:06:59,  1.27it/s, gpt_loss=0.0609, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 497/5616 [06:25<1:07:33,  1.26it/s, gpt_loss=0.0609, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 497/5616 [06:26<1:07:33,  1.26it/s, gpt_loss=0.124, lr=4.96e-6] 
+
Train step of epoch 0:   9%|▉         | 498/5616 [06:26<1:07:53,  1.26it/s, gpt_loss=0.124, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 498/5616 [06:26<1:07:53,  1.26it/s, gpt_loss=0.0657, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 499/5616 [06:26<1:08:02,  1.25it/s, gpt_loss=0.0657, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 499/5616 [06:27<1:08:02,  1.25it/s, gpt_loss=0.0776, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 500/5616 [06:27<1:10:19,  1.21it/s, gpt_loss=0.0776, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 500/5616 [06:28<1:10:19,  1.21it/s, gpt_loss=0.0564, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 501/5616 [06:28<1:09:01,  1.23it/s, gpt_loss=0.0564, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 501/5616 [06:29<1:09:01,  1.23it/s, gpt_loss=0.0843, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 502/5616 [06:29<1:07:32,  1.26it/s, gpt_loss=0.0843, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 502/5616 [06:30<1:07:32,  1.26it/s, gpt_loss=0.0205, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 503/5616 [06:30<1:05:02,  1.31it/s, gpt_loss=0.0205, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 503/5616 [06:30<1:05:02,  1.31it/s, gpt_loss=0.112, lr=4.96e-6] 
+
Train step of epoch 0:   9%|▉         | 504/5616 [06:30<1:08:05,  1.25it/s, gpt_loss=0.112, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 504/5616 [06:31<1:08:05,  1.25it/s, gpt_loss=0.0241, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 505/5616 [06:31<1:05:06,  1.31it/s, gpt_loss=0.0241, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 505/5616 [06:32<1:05:06,  1.31it/s, gpt_loss=0.00573, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 506/5616 [06:32<1:02:37,  1.36it/s, gpt_loss=0.00573, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 506/5616 [06:33<1:02:37,  1.36it/s, gpt_loss=0.127, lr=4.96e-6]  
+
Train step of epoch 0:   9%|▉         | 507/5616 [06:33<1:02:55,  1.35it/s, gpt_loss=0.127, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 507/5616 [06:33<1:02:55,  1.35it/s, gpt_loss=0.0459, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 508/5616 [06:33<1:04:12,  1.33it/s, gpt_loss=0.0459, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 508/5616 [06:34<1:04:12,  1.33it/s, gpt_loss=0.0353, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 509/5616 [06:34<1:05:28,  1.30it/s, gpt_loss=0.0353, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 509/5616 [06:35<1:05:28,  1.30it/s, gpt_loss=0.0229, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 510/5616 [06:35<1:03:23,  1.34it/s, gpt_loss=0.0229, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 510/5616 [06:36<1:03:23,  1.34it/s, gpt_loss=0.105, lr=4.96e-6] 
+
Train step of epoch 0:   9%|▉         | 511/5616 [06:36<1:02:30,  1.36it/s, gpt_loss=0.105, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 511/5616 [06:36<1:02:30,  1.36it/s, gpt_loss=0.0225, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 512/5616 [06:36<1:02:19,  1.36it/s, gpt_loss=0.0225, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 512/5616 [06:37<1:02:19,  1.36it/s, gpt_loss=0.127, lr=4.96e-6] 
+
Train step of epoch 0:   9%|▉         | 513/5616 [06:37<1:01:20,  1.39it/s, gpt_loss=0.127, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 513/5616 [06:38<1:01:20,  1.39it/s, gpt_loss=0.0703, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 514/5616 [06:38<59:28,  1.43it/s, gpt_loss=0.0703, lr=4.96e-6]  
+
Train step of epoch 0:   9%|▉         | 514/5616 [06:38<59:28,  1.43it/s, gpt_loss=0.0922, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 515/5616 [06:38<1:01:12,  1.39it/s, gpt_loss=0.0922, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 515/5616 [06:39<1:01:12,  1.39it/s, gpt_loss=0.0458, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 516/5616 [06:39<1:08:26,  1.24it/s, gpt_loss=0.0458, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 516/5616 [06:40<1:08:26,  1.24it/s, gpt_loss=0.117, lr=4.96e-6] 
+
Train step of epoch 0:   9%|▉         | 517/5616 [06:40<1:07:01,  1.27it/s, gpt_loss=0.117, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 517/5616 [06:41<1:07:01,  1.27it/s, gpt_loss=0.0293, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 518/5616 [06:41<1:06:44,  1.27it/s, gpt_loss=0.0293, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 518/5616 [06:42<1:06:44,  1.27it/s, gpt_loss=0.0355, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 519/5616 [06:42<1:03:21,  1.34it/s, gpt_loss=0.0355, lr=4.96e-6]
+
Train step of epoch 0:   9%|▉         | 519/5616 [06:42<1:03:21,  1.34it/s, gpt_loss=0.109, lr=4.95e-6] 
+
Train step of epoch 0:   9%|▉         | 520/5616 [06:42<1:03:55,  1.33it/s, gpt_loss=0.109, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 520/5616 [06:43<1:03:55,  1.33it/s, gpt_loss=0.0645, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 521/5616 [06:43<1:04:36,  1.31it/s, gpt_loss=0.0645, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 521/5616 [06:44<1:04:36,  1.31it/s, gpt_loss=0.0337, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 522/5616 [06:44<1:05:49,  1.29it/s, gpt_loss=0.0337, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 522/5616 [06:45<1:05:49,  1.29it/s, gpt_loss=0.0112, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 523/5616 [06:45<1:03:06,  1.34it/s, gpt_loss=0.0112, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 523/5616 [06:45<1:03:06,  1.34it/s, gpt_loss=0.054, lr=4.95e-6] 
+
Train step of epoch 0:   9%|▉         | 524/5616 [06:45<1:01:54,  1.37it/s, gpt_loss=0.054, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 524/5616 [06:46<1:01:54,  1.37it/s, gpt_loss=0.0341, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 525/5616 [06:46<1:00:54,  1.39it/s, gpt_loss=0.0341, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 525/5616 [06:47<1:00:54,  1.39it/s, gpt_loss=0.0226, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 526/5616 [06:47<1:00:29,  1.40it/s, gpt_loss=0.0226, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 526/5616 [06:47<1:00:29,  1.40it/s, gpt_loss=0.0412, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 527/5616 [06:47<59:25,  1.43it/s, gpt_loss=0.0412, lr=4.95e-6]  
+
Train step of epoch 0:   9%|▉         | 527/5616 [06:48<59:25,  1.43it/s, gpt_loss=0.0305, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 528/5616 [06:48<59:53,  1.42it/s, gpt_loss=0.0305, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 528/5616 [06:49<59:53,  1.42it/s, gpt_loss=0.179, lr=4.95e-6] 
+
Train step of epoch 0:   9%|▉         | 529/5616 [06:49<59:00,  1.44it/s, gpt_loss=0.179, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 529/5616 [06:49<59:00,  1.44it/s, gpt_loss=0.0792, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 530/5616 [06:49<58:50,  1.44it/s, gpt_loss=0.0792, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 530/5616 [06:50<58:50,  1.44it/s, gpt_loss=0.0558, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 531/5616 [06:50<1:00:08,  1.41it/s, gpt_loss=0.0558, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 531/5616 [06:51<1:00:08,  1.41it/s, gpt_loss=0.0463, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 532/5616 [06:51<1:00:29,  1.40it/s, gpt_loss=0.0463, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 532/5616 [06:52<1:00:29,  1.40it/s, gpt_loss=0.0186, lr=4.95e-6]
+
Train step of epoch 0:   9%|▉         | 533/5616 [06:52<59:52,  1.41it/s, gpt_loss=0.0186, lr=4.95e-6]  
+
Train step of epoch 0:   9%|▉         | 533/5616 [06:52<59:52,  1.41it/s, gpt_loss=0.0152, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 534/5616 [06:52<1:01:27,  1.38it/s, gpt_loss=0.0152, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 534/5616 [06:53<1:01:27,  1.38it/s, gpt_loss=0.071, lr=4.95e-6] 
+
Train step of epoch 0:  10%|▉         | 535/5616 [06:53<1:01:42,  1.37it/s, gpt_loss=0.071, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 535/5616 [06:54<1:01:42,  1.37it/s, gpt_loss=0.0666, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 536/5616 [06:54<1:05:42,  1.29it/s, gpt_loss=0.0666, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 536/5616 [06:55<1:05:42,  1.29it/s, gpt_loss=0.0692, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 537/5616 [06:55<1:15:13,  1.13it/s, gpt_loss=0.0692, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 537/5616 [06:56<1:15:13,  1.13it/s, gpt_loss=0.14, lr=4.95e-6]  
+
Train step of epoch 0:  10%|▉         | 538/5616 [06:56<1:09:55,  1.21it/s, gpt_loss=0.14, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 538/5616 [06:57<1:09:55,  1.21it/s, gpt_loss=0.0208, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 539/5616 [06:57<1:07:49,  1.25it/s, gpt_loss=0.0208, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 539/5616 [06:57<1:07:49,  1.25it/s, gpt_loss=0.105, lr=4.95e-6] 
+
Train step of epoch 0:  10%|▉         | 540/5616 [06:57<1:06:33,  1.27it/s, gpt_loss=0.105, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 540/5616 [06:58<1:06:33,  1.27it/s, gpt_loss=0.0175, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 541/5616 [06:58<1:13:44,  1.15it/s, gpt_loss=0.0175, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 541/5616 [06:59<1:13:44,  1.15it/s, gpt_loss=0.052, lr=4.95e-6] 
+
Train step of epoch 0:  10%|▉         | 542/5616 [06:59<1:10:37,  1.20it/s, gpt_loss=0.052, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 542/5616 [07:00<1:10:37,  1.20it/s, gpt_loss=0.0381, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 543/5616 [07:00<1:05:21,  1.29it/s, gpt_loss=0.0381, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 543/5616 [07:01<1:05:21,  1.29it/s, gpt_loss=0.0189, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 544/5616 [07:01<1:04:38,  1.31it/s, gpt_loss=0.0189, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 544/5616 [07:01<1:04:38,  1.31it/s, gpt_loss=0.011, lr=4.95e-6] 
+
Train step of epoch 0:  10%|▉         | 545/5616 [07:01<1:04:09,  1.32it/s, gpt_loss=0.011, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 545/5616 [07:02<1:04:09,  1.32it/s, gpt_loss=0.092, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 546/5616 [07:02<1:14:26,  1.14it/s, gpt_loss=0.092, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 546/5616 [07:03<1:14:26,  1.14it/s, gpt_loss=0.121, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 547/5616 [07:03<1:09:16,  1.22it/s, gpt_loss=0.121, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 547/5616 [07:04<1:09:16,  1.22it/s, gpt_loss=0.132, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 548/5616 [07:04<1:07:15,  1.26it/s, gpt_loss=0.132, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 548/5616 [07:04<1:07:15,  1.26it/s, gpt_loss=0.137, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 549/5616 [07:04<1:03:12,  1.34it/s, gpt_loss=0.137, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 549/5616 [07:05<1:03:12,  1.34it/s, gpt_loss=0.0615, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 550/5616 [07:05<1:01:28,  1.37it/s, gpt_loss=0.0615, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 550/5616 [07:06<1:01:28,  1.37it/s, gpt_loss=0.125, lr=4.95e-6] 
+
Train step of epoch 0:  10%|▉         | 551/5616 [07:06<1:01:27,  1.37it/s, gpt_loss=0.125, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 551/5616 [07:07<1:01:27,  1.37it/s, gpt_loss=0.107, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 552/5616 [07:07<1:07:01,  1.26it/s, gpt_loss=0.107, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 552/5616 [07:07<1:07:01,  1.26it/s, gpt_loss=0.0962, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 553/5616 [07:07<1:03:23,  1.33it/s, gpt_loss=0.0962, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 553/5616 [07:08<1:03:23,  1.33it/s, gpt_loss=0.043, lr=4.95e-6] 
+
Train step of epoch 0:  10%|▉         | 554/5616 [07:08<1:07:03,  1.26it/s, gpt_loss=0.043, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 554/5616 [07:09<1:07:03,  1.26it/s, gpt_loss=0.0625, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 555/5616 [07:09<1:03:46,  1.32it/s, gpt_loss=0.0625, lr=4.95e-6]
+
Train step of epoch 0:  10%|▉         | 555/5616 [07:10<1:03:46,  1.32it/s, gpt_loss=0.0218, lr=4.94e-6]
+
Train step of epoch 0:  10%|▉         | 556/5616 [07:10<1:09:32,  1.21it/s, gpt_loss=0.0218, lr=4.94e-6]
+
Train step of epoch 0:  10%|▉         | 556/5616 [07:11<1:09:32,  1.21it/s, gpt_loss=0.0114, lr=4.94e-6]
+
Train step of epoch 0:  10%|▉         | 557/5616 [07:11<1:06:30,  1.27it/s, gpt_loss=0.0114, lr=4.94e-6]
+
Train step of epoch 0:  10%|▉         | 557/5616 [07:11<1:06:30,  1.27it/s, gpt_loss=0.00975, lr=4.94e-6]
+
Train step of epoch 0:  10%|▉         | 558/5616 [07:11<1:03:59,  1.32it/s, gpt_loss=0.00975, lr=4.94e-6]
+
Train step of epoch 0:  10%|▉         | 558/5616 [07:12<1:03:59,  1.32it/s, gpt_loss=0.162, lr=4.94e-6]  
+
Train step of epoch 0:  10%|▉         | 559/5616 [07:12<1:04:06,  1.31it/s, gpt_loss=0.162, lr=4.94e-6]
+
Train step of epoch 0:  10%|▉         | 559/5616 [07:13<1:04:06,  1.31it/s, gpt_loss=0.032, lr=4.94e-6]
+
Train step of epoch 0:  10%|▉         | 560/5616 [07:13<1:09:50,  1.21it/s, gpt_loss=0.032, lr=4.94e-6]
+
Train step of epoch 0:  10%|▉         | 560/5616 [07:14<1:09:50,  1.21it/s, gpt_loss=0.146, lr=4.94e-6]
+
Train step of epoch 0:  10%|▉         | 561/5616 [07:14<1:08:20,  1.23it/s, gpt_loss=0.146, lr=4.94e-6]
+
Train step of epoch 0:  10%|▉         | 561/5616 [07:15<1:08:20,  1.23it/s, gpt_loss=0.0192, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 562/5616 [07:15<1:05:43,  1.28it/s, gpt_loss=0.0192, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 562/5616 [07:15<1:05:43,  1.28it/s, gpt_loss=0.0658, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 563/5616 [07:15<1:06:45,  1.26it/s, gpt_loss=0.0658, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 563/5616 [07:16<1:06:45,  1.26it/s, gpt_loss=0.0441, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 564/5616 [07:16<1:06:58,  1.26it/s, gpt_loss=0.0441, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 564/5616 [07:17<1:06:58,  1.26it/s, gpt_loss=0.0547, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 565/5616 [07:17<1:04:01,  1.31it/s, gpt_loss=0.0547, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 565/5616 [07:18<1:04:01,  1.31it/s, gpt_loss=0.0698, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 566/5616 [07:18<1:03:09,  1.33it/s, gpt_loss=0.0698, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 566/5616 [07:18<1:03:09,  1.33it/s, gpt_loss=0.112, lr=4.94e-6] 
+
Train step of epoch 0:  10%|█         | 567/5616 [07:18<1:01:37,  1.37it/s, gpt_loss=0.112, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 567/5616 [07:19<1:01:37,  1.37it/s, gpt_loss=0.0327, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 568/5616 [07:19<1:04:31,  1.30it/s, gpt_loss=0.0327, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 568/5616 [07:20<1:04:31,  1.30it/s, gpt_loss=0.0956, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 569/5616 [07:20<1:02:30,  1.35it/s, gpt_loss=0.0956, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 569/5616 [07:21<1:02:30,  1.35it/s, gpt_loss=0.0205, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 570/5616 [07:21<1:01:02,  1.38it/s, gpt_loss=0.0205, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 570/5616 [07:22<1:01:02,  1.38it/s, gpt_loss=0.0759, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 571/5616 [07:22<1:06:37,  1.26it/s, gpt_loss=0.0759, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 571/5616 [07:22<1:06:37,  1.26it/s, gpt_loss=0.0509, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 572/5616 [07:22<1:04:47,  1.30it/s, gpt_loss=0.0509, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 572/5616 [07:23<1:04:47,  1.30it/s, gpt_loss=0.0584, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 573/5616 [07:23<1:02:39,  1.34it/s, gpt_loss=0.0584, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 573/5616 [07:24<1:02:39,  1.34it/s, gpt_loss=0.029, lr=4.94e-6] 
+
Train step of epoch 0:  10%|█         | 574/5616 [07:24<1:04:28,  1.30it/s, gpt_loss=0.029, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 574/5616 [07:25<1:04:28,  1.30it/s, gpt_loss=0.114, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 575/5616 [07:25<1:03:55,  1.31it/s, gpt_loss=0.114, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 575/5616 [07:25<1:03:55,  1.31it/s, gpt_loss=0.0953, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 576/5616 [07:25<1:04:52,  1.29it/s, gpt_loss=0.0953, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 576/5616 [07:26<1:04:52,  1.29it/s, gpt_loss=0.0995, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 577/5616 [07:26<1:02:58,  1.33it/s, gpt_loss=0.0995, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 577/5616 [07:27<1:02:58,  1.33it/s, gpt_loss=0.0177, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 578/5616 [07:27<1:01:49,  1.36it/s, gpt_loss=0.0177, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 578/5616 [07:28<1:01:49,  1.36it/s, gpt_loss=0.0925, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 579/5616 [07:28<1:04:17,  1.31it/s, gpt_loss=0.0925, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 579/5616 [07:28<1:04:17,  1.31it/s, gpt_loss=0.0787, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 580/5616 [07:28<1:03:44,  1.32it/s, gpt_loss=0.0787, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 580/5616 [07:29<1:03:44,  1.32it/s, gpt_loss=0.101, lr=4.94e-6] 
+
Train step of epoch 0:  10%|█         | 581/5616 [07:29<1:02:57,  1.33it/s, gpt_loss=0.101, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 581/5616 [07:30<1:02:57,  1.33it/s, gpt_loss=0.0251, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 582/5616 [07:30<1:01:42,  1.36it/s, gpt_loss=0.0251, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 582/5616 [07:31<1:01:42,  1.36it/s, gpt_loss=0.0619, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 583/5616 [07:31<1:03:37,  1.32it/s, gpt_loss=0.0619, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 583/5616 [07:31<1:03:37,  1.32it/s, gpt_loss=0.0196, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 584/5616 [07:31<1:03:07,  1.33it/s, gpt_loss=0.0196, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 584/5616 [07:32<1:03:07,  1.33it/s, gpt_loss=0.0253, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 585/5616 [07:32<1:02:48,  1.33it/s, gpt_loss=0.0253, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 585/5616 [07:33<1:02:48,  1.33it/s, gpt_loss=0.056, lr=4.94e-6] 
+
Train step of epoch 0:  10%|█         | 586/5616 [07:33<1:00:02,  1.40it/s, gpt_loss=0.056, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 586/5616 [07:33<1:00:02,  1.40it/s, gpt_loss=0.07, lr=4.94e-6] 
+
Train step of epoch 0:  10%|█         | 587/5616 [07:33<1:02:03,  1.35it/s, gpt_loss=0.07, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 587/5616 [07:34<1:02:03,  1.35it/s, gpt_loss=0.0691, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 588/5616 [07:34<1:04:04,  1.31it/s, gpt_loss=0.0691, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 588/5616 [07:35<1:04:04,  1.31it/s, gpt_loss=0.0942, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 589/5616 [07:35<1:03:05,  1.33it/s, gpt_loss=0.0942, lr=4.94e-6]
+
Train step of epoch 0:  10%|█         | 589/5616 [07:36<1:03:05,  1.33it/s, gpt_loss=0.0229, lr=4.94e-6]
+
Train step of epoch 0:  11%|█         | 590/5616 [07:36<1:01:34,  1.36it/s, gpt_loss=0.0229, lr=4.94e-6]
+
Train step of epoch 0:  11%|█         | 590/5616 [07:36<1:01:34,  1.36it/s, gpt_loss=0.0301, lr=4.94e-6]
+
Train step of epoch 0:  11%|█         | 591/5616 [07:36<1:02:08,  1.35it/s, gpt_loss=0.0301, lr=4.94e-6]
+
Train step of epoch 0:  11%|█         | 591/5616 [07:37<1:02:08,  1.35it/s, gpt_loss=0.101, lr=4.93e-6] 
+
Train step of epoch 0:  11%|█         | 592/5616 [07:37<1:02:59,  1.33it/s, gpt_loss=0.101, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 592/5616 [07:38<1:02:59,  1.33it/s, gpt_loss=0.167, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 593/5616 [07:38<1:04:44,  1.29it/s, gpt_loss=0.167, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 593/5616 [07:39<1:04:44,  1.29it/s, gpt_loss=0.0704, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 594/5616 [07:39<1:03:40,  1.31it/s, gpt_loss=0.0704, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 594/5616 [07:40<1:03:40,  1.31it/s, gpt_loss=0.0234, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 595/5616 [07:40<1:03:00,  1.33it/s, gpt_loss=0.0234, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 595/5616 [07:40<1:03:00,  1.33it/s, gpt_loss=0.0214, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 596/5616 [07:40<1:05:16,  1.28it/s, gpt_loss=0.0214, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 596/5616 [07:41<1:05:16,  1.28it/s, gpt_loss=0.0381, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 597/5616 [07:41<1:04:51,  1.29it/s, gpt_loss=0.0381, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 597/5616 [07:42<1:04:51,  1.29it/s, gpt_loss=0.0105, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 598/5616 [07:42<1:03:01,  1.33it/s, gpt_loss=0.0105, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 598/5616 [07:43<1:03:01,  1.33it/s, gpt_loss=0.0607, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 599/5616 [07:43<1:02:08,  1.35it/s, gpt_loss=0.0607, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 599/5616 [07:43<1:02:08,  1.35it/s, gpt_loss=0.0249, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 600/5616 [07:43<1:04:38,  1.29it/s, gpt_loss=0.0249, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 600/5616 [07:44<1:04:38,  1.29it/s, gpt_loss=0.0076, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 601/5616 [07:44<1:01:30,  1.36it/s, gpt_loss=0.0076, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 601/5616 [07:45<1:01:30,  1.36it/s, gpt_loss=0.0415, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 602/5616 [07:45<1:06:36,  1.25it/s, gpt_loss=0.0415, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 602/5616 [07:46<1:06:36,  1.25it/s, gpt_loss=0.0635, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 603/5616 [07:46<1:04:11,  1.30it/s, gpt_loss=0.0635, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 603/5616 [07:46<1:04:11,  1.30it/s, gpt_loss=0.0177, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 604/5616 [07:46<1:04:02,  1.30it/s, gpt_loss=0.0177, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 604/5616 [07:47<1:04:02,  1.30it/s, gpt_loss=0.129, lr=4.93e-6] 
+
Train step of epoch 0:  11%|█         | 605/5616 [07:47<1:02:55,  1.33it/s, gpt_loss=0.129, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 605/5616 [07:48<1:02:55,  1.33it/s, gpt_loss=0.0285, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 606/5616 [07:48<1:01:14,  1.36it/s, gpt_loss=0.0285, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 606/5616 [07:49<1:01:14,  1.36it/s, gpt_loss=0.108, lr=4.93e-6] 
+
Train step of epoch 0:  11%|█         | 607/5616 [07:49<1:02:47,  1.33it/s, gpt_loss=0.108, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 607/5616 [07:49<1:02:47,  1.33it/s, gpt_loss=0.0407, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 608/5616 [07:49<1:03:17,  1.32it/s, gpt_loss=0.0407, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 608/5616 [07:50<1:03:17,  1.32it/s, gpt_loss=0.135, lr=4.93e-6] 
+
Train step of epoch 0:  11%|█         | 609/5616 [07:50<1:00:04,  1.39it/s, gpt_loss=0.135, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 609/5616 [07:51<1:00:04,  1.39it/s, gpt_loss=0.197, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 610/5616 [07:51<1:01:19,  1.36it/s, gpt_loss=0.197, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 610/5616 [07:51<1:01:19,  1.36it/s, gpt_loss=0.155, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 611/5616 [07:51<59:44,  1.40it/s, gpt_loss=0.155, lr=4.93e-6]  
+
Train step of epoch 0:  11%|█         | 611/5616 [07:52<59:44,  1.40it/s, gpt_loss=0.0157, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 612/5616 [07:52<1:00:04,  1.39it/s, gpt_loss=0.0157, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 612/5616 [07:53<1:00:04,  1.39it/s, gpt_loss=0.0609, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 613/5616 [07:53<1:00:06,  1.39it/s, gpt_loss=0.0609, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 613/5616 [07:54<1:00:06,  1.39it/s, gpt_loss=0.0468, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 614/5616 [07:54<57:59,  1.44it/s, gpt_loss=0.0468, lr=4.93e-6]  
+
Train step of epoch 0:  11%|█         | 614/5616 [07:54<57:59,  1.44it/s, gpt_loss=0.0718, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 615/5616 [07:54<57:05,  1.46it/s, gpt_loss=0.0718, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 615/5616 [07:55<57:05,  1.46it/s, gpt_loss=0.038, lr=4.93e-6] 
+
Train step of epoch 0:  11%|█         | 616/5616 [07:55<1:06:43,  1.25it/s, gpt_loss=0.038, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 616/5616 [07:56<1:06:43,  1.25it/s, gpt_loss=0.0817, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 617/5616 [07:56<1:04:27,  1.29it/s, gpt_loss=0.0817, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 617/5616 [07:57<1:04:27,  1.29it/s, gpt_loss=0.0117, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 618/5616 [07:57<1:02:19,  1.34it/s, gpt_loss=0.0117, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 618/5616 [07:57<1:02:19,  1.34it/s, gpt_loss=0.0719, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 619/5616 [07:57<1:00:54,  1.37it/s, gpt_loss=0.0719, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 619/5616 [07:58<1:00:54,  1.37it/s, gpt_loss=0.0197, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 620/5616 [07:58<1:02:12,  1.34it/s, gpt_loss=0.0197, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 620/5616 [07:59<1:02:12,  1.34it/s, gpt_loss=0.0564, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 621/5616 [07:59<59:51,  1.39it/s, gpt_loss=0.0564, lr=4.93e-6]  
+
Train step of epoch 0:  11%|█         | 621/5616 [08:00<59:51,  1.39it/s, gpt_loss=0.0702, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 622/5616 [08:00<1:00:02,  1.39it/s, gpt_loss=0.0702, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 622/5616 [08:00<1:00:02,  1.39it/s, gpt_loss=0.0497, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 623/5616 [08:00<1:00:11,  1.38it/s, gpt_loss=0.0497, lr=4.93e-6]
+
Train step of epoch 0:  11%|█         | 623/5616 [08:01<1:00:11,  1.38it/s, gpt_loss=0.00917, lr=4.92e-6]
+
Train step of epoch 0:  11%|█         | 624/5616 [08:01<1:01:13,  1.36it/s, gpt_loss=0.00917, lr=4.92e-6]
+
Train step of epoch 0:  11%|█         | 624/5616 [08:02<1:01:13,  1.36it/s, gpt_loss=0.0782, lr=4.92e-6] 
+
Train step of epoch 0:  11%|█         | 625/5616 [08:02<58:58,  1.41it/s, gpt_loss=0.0782, lr=4.92e-6]  
+
Train step of epoch 0:  11%|█         | 625/5616 [08:03<58:58,  1.41it/s, gpt_loss=0.0884, lr=4.92e-6]
+
Train step of epoch 0:  11%|█         | 626/5616 [08:03<1:02:00,  1.34it/s, gpt_loss=0.0884, lr=4.92e-6]
+
Train step of epoch 0:  11%|█         | 626/5616 [08:03<1:02:00,  1.34it/s, gpt_loss=0.0579, lr=4.92e-6]
+
Train step of epoch 0:  11%|█         | 627/5616 [08:03<1:00:39,  1.37it/s, gpt_loss=0.0579, lr=4.92e-6]
+
Train step of epoch 0:  11%|█         | 627/5616 [08:04<1:00:39,  1.37it/s, gpt_loss=0.039, lr=4.92e-6] 
+
Train step of epoch 0:  11%|█         | 628/5616 [08:04<1:03:38,  1.31it/s, gpt_loss=0.039, lr=4.92e-6]
+
Train step of epoch 0:  11%|█         | 628/5616 [08:05<1:03:38,  1.31it/s, gpt_loss=0.0146, lr=4.92e-6]
+
Train step of epoch 0:  11%|█         | 629/5616 [08:05<1:01:40,  1.35it/s, gpt_loss=0.0146, lr=4.92e-6]
+
Train step of epoch 0:  11%|█         | 629/5616 [08:06<1:01:40,  1.35it/s, gpt_loss=0.0576, lr=4.92e-6]
+
Train step of epoch 0:  11%|█         | 630/5616 [08:06<1:02:22,  1.33it/s, gpt_loss=0.0576, lr=4.92e-6]
+
Train step of epoch 0:  11%|█         | 630/5616 [08:06<1:02:22,  1.33it/s, gpt_loss=0.0289, lr=4.92e-6]
+
Train step of epoch 0:  11%|█         | 631/5616 [08:06<1:00:24,  1.38it/s, gpt_loss=0.0289, lr=4.92e-6]
+
Train step of epoch 0:  11%|█         | 631/5616 [08:07<1:00:24,  1.38it/s, gpt_loss=0.0473, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 632/5616 [08:07<1:08:24,  1.21it/s, gpt_loss=0.0473, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 632/5616 [08:08<1:08:24,  1.21it/s, gpt_loss=0.0201, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 633/5616 [08:08<1:05:01,  1.28it/s, gpt_loss=0.0201, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 633/5616 [08:09<1:05:01,  1.28it/s, gpt_loss=0.105, lr=4.92e-6] 
+
Train step of epoch 0:  11%|█▏        | 634/5616 [08:09<1:03:42,  1.30it/s, gpt_loss=0.105, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 634/5616 [08:09<1:03:42,  1.30it/s, gpt_loss=0.0272, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 635/5616 [08:09<1:02:50,  1.32it/s, gpt_loss=0.0272, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 635/5616 [08:10<1:02:50,  1.32it/s, gpt_loss=0.00995, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 636/5616 [08:10<1:02:46,  1.32it/s, gpt_loss=0.00995, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 636/5616 [08:11<1:02:46,  1.32it/s, gpt_loss=0.0183, lr=4.92e-6] 
+
Train step of epoch 0:  11%|█▏        | 637/5616 [08:11<1:01:19,  1.35it/s, gpt_loss=0.0183, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 637/5616 [08:11<1:01:19,  1.35it/s, gpt_loss=0.0185, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 638/5616 [08:11<58:14,  1.42it/s, gpt_loss=0.0185, lr=4.92e-6]  
+
Train step of epoch 0:  11%|█▏        | 638/5616 [08:12<58:14,  1.42it/s, gpt_loss=0.0197, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 639/5616 [08:12<57:10,  1.45it/s, gpt_loss=0.0197, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 639/5616 [08:13<57:10,  1.45it/s, gpt_loss=0.0374, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 640/5616 [08:13<1:01:05,  1.36it/s, gpt_loss=0.0374, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 640/5616 [08:14<1:01:05,  1.36it/s, gpt_loss=0.0643, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 641/5616 [08:14<1:00:53,  1.36it/s, gpt_loss=0.0643, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 641/5616 [08:14<1:00:53,  1.36it/s, gpt_loss=0.0825, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 642/5616 [08:14<59:45,  1.39it/s, gpt_loss=0.0825, lr=4.92e-6]  
+
Train step of epoch 0:  11%|█▏        | 642/5616 [08:15<59:45,  1.39it/s, gpt_loss=0.0115, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 643/5616 [08:15<58:12,  1.42it/s, gpt_loss=0.0115, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 643/5616 [08:16<58:12,  1.42it/s, gpt_loss=0.0304, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 644/5616 [08:16<1:00:57,  1.36it/s, gpt_loss=0.0304, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 644/5616 [08:17<1:00:57,  1.36it/s, gpt_loss=0.0396, lr=4.92e-6]
+
Train step of epoch 0:  11%|█▏        | 645/5616 [08:17<58:54,  1.41it/s, gpt_loss=0.0396, lr=4.92e-6]  
+
Train step of epoch 0:  11%|█▏        | 645/5616 [08:17<58:54,  1.41it/s, gpt_loss=0.0393, lr=4.92e-6]
+
Train step of epoch 0:  12%|█▏        | 646/5616 [08:17<56:45,  1.46it/s, gpt_loss=0.0393, lr=4.92e-6]
+
Train step of epoch 0:  12%|█▏        | 646/5616 [08:18<56:45,  1.46it/s, gpt_loss=0.0181, lr=4.92e-6]
+
Train step of epoch 0:  12%|█▏        | 647/5616 [08:18<57:06,  1.45it/s, gpt_loss=0.0181, lr=4.92e-6]
+
Train step of epoch 0:  12%|█▏        | 647/5616 [08:19<57:06,  1.45it/s, gpt_loss=0.0202, lr=4.92e-6]
+
Train step of epoch 0:  12%|█▏        | 648/5616 [08:19<59:03,  1.40it/s, gpt_loss=0.0202, lr=4.92e-6]
+
Train step of epoch 0:  12%|█▏        | 648/5616 [08:19<59:03,  1.40it/s, gpt_loss=0.0108, lr=4.92e-6]
+
Train step of epoch 0:  12%|���▏        | 649/5616 [08:19<1:01:15,  1.35it/s, gpt_loss=0.0108, lr=4.92e-6]
+
Train step of epoch 0:  12%|█▏        | 649/5616 [08:20<1:01:15,  1.35it/s, gpt_loss=0.0149, lr=4.92e-6]
+
Train step of epoch 0:  12%|█▏        | 650/5616 [08:20<59:48,  1.38it/s, gpt_loss=0.0149, lr=4.92e-6]  
+
Train step of epoch 0:  12%|█▏        | 650/5616 [08:21<59:48,  1.38it/s, gpt_loss=0.135, lr=4.92e-6] 
+
Train step of epoch 0:  12%|█▏        | 651/5616 [08:21<59:10,  1.40it/s, gpt_loss=0.135, lr=4.92e-6]
+
Train step of epoch 0:  12%|█▏        | 651/5616 [08:22<59:10,  1.40it/s, gpt_loss=0.0188, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 652/5616 [08:22<1:09:18,  1.19it/s, gpt_loss=0.0188, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 652/5616 [08:23<1:09:18,  1.19it/s, gpt_loss=0.0139, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 653/5616 [08:23<1:04:24,  1.28it/s, gpt_loss=0.0139, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 653/5616 [08:24<1:04:24,  1.28it/s, gpt_loss=0.0919, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 654/5616 [08:24<1:10:15,  1.18it/s, gpt_loss=0.0919, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 654/5616 [08:24<1:10:15,  1.18it/s, gpt_loss=0.0574, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 655/5616 [08:24<1:07:02,  1.23it/s, gpt_loss=0.0574, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 655/5616 [08:25<1:07:02,  1.23it/s, gpt_loss=0.132, lr=4.91e-6] 
+
Train step of epoch 0:  12%|█▏        | 656/5616 [08:25<1:06:35,  1.24it/s, gpt_loss=0.132, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 656/5616 [08:26<1:06:35,  1.24it/s, gpt_loss=0.045, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 657/5616 [08:26<1:06:59,  1.23it/s, gpt_loss=0.045, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 657/5616 [08:27<1:06:59,  1.23it/s, gpt_loss=0.0654, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 658/5616 [08:27<1:07:08,  1.23it/s, gpt_loss=0.0654, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 658/5616 [08:28<1:07:08,  1.23it/s, gpt_loss=0.0504, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 659/5616 [08:28<1:12:49,  1.13it/s, gpt_loss=0.0504, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 659/5616 [08:29<1:12:49,  1.13it/s, gpt_loss=0.0451, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 660/5616 [08:29<1:20:59,  1.02it/s, gpt_loss=0.0451, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 660/5616 [08:30<1:20:59,  1.02it/s, gpt_loss=0.00481, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 661/5616 [08:30<1:14:26,  1.11it/s, gpt_loss=0.00481, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 661/5616 [08:30<1:14:26,  1.11it/s, gpt_loss=0.0255, lr=4.91e-6] 
+
Train step of epoch 0:  12%|█▏        | 662/5616 [08:30<1:10:42,  1.17it/s, gpt_loss=0.0255, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 662/5616 [08:31<1:10:42,  1.17it/s, gpt_loss=0.00748, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 663/5616 [08:31<1:14:57,  1.10it/s, gpt_loss=0.00748, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 663/5616 [08:32<1:14:57,  1.10it/s, gpt_loss=0.0154, lr=4.91e-6] 
+
Train step of epoch 0:  12%|█▏        | 664/5616 [08:32<1:11:43,  1.15it/s, gpt_loss=0.0154, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 664/5616 [08:33<1:11:43,  1.15it/s, gpt_loss=0.112, lr=4.91e-6] 
+
Train step of epoch 0:  12%|█▏        | 665/5616 [08:33<1:06:34,  1.24it/s, gpt_loss=0.112, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 665/5616 [08:34<1:06:34,  1.24it/s, gpt_loss=0.0454, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 666/5616 [08:34<1:03:37,  1.30it/s, gpt_loss=0.0454, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 666/5616 [08:34<1:03:37,  1.30it/s, gpt_loss=0.0354, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 667/5616 [08:34<1:00:58,  1.35it/s, gpt_loss=0.0354, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 667/5616 [08:35<1:00:58,  1.35it/s, gpt_loss=0.00616, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 668/5616 [08:35<1:01:19,  1.34it/s, gpt_loss=0.00616, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 668/5616 [08:36<1:01:19,  1.34it/s, gpt_loss=0.0347, lr=4.91e-6] 
+
Train step of epoch 0:  12%|█▏        | 669/5616 [08:36<1:03:13,  1.30it/s, gpt_loss=0.0347, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 669/5616 [08:37<1:03:13,  1.30it/s, gpt_loss=0.076, lr=4.91e-6] 
+
Train step of epoch 0:  12%|█▏        | 670/5616 [08:37<1:02:59,  1.31it/s, gpt_loss=0.076, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 670/5616 [08:37<1:02:59,  1.31it/s, gpt_loss=0.0145, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 671/5616 [08:37<1:02:34,  1.32it/s, gpt_loss=0.0145, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 671/5616 [08:38<1:02:34,  1.32it/s, gpt_loss=0.131, lr=4.91e-6] 
+
Train step of epoch 0:  12%|█▏        | 672/5616 [08:38<1:04:59,  1.27it/s, gpt_loss=0.131, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 672/5616 [08:39<1:04:59,  1.27it/s, gpt_loss=0.203, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 673/5616 [08:39<1:07:56,  1.21it/s, gpt_loss=0.203, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 673/5616 [08:40<1:07:56,  1.21it/s, gpt_loss=0.198, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 674/5616 [08:40<1:07:12,  1.23it/s, gpt_loss=0.198, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 674/5616 [08:41<1:07:12,  1.23it/s, gpt_loss=0.104, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 675/5616 [08:41<1:05:23,  1.26it/s, gpt_loss=0.104, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 675/5616 [08:41<1:05:23,  1.26it/s, gpt_loss=0.0273, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 676/5616 [08:41<1:04:36,  1.27it/s, gpt_loss=0.0273, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 676/5616 [08:42<1:04:36,  1.27it/s, gpt_loss=0.0496, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 677/5616 [08:42<1:05:12,  1.26it/s, gpt_loss=0.0496, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 677/5616 [08:43<1:05:12,  1.26it/s, gpt_loss=0.0524, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 678/5616 [08:43<1:05:16,  1.26it/s, gpt_loss=0.0524, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 678/5616 [08:44<1:05:16,  1.26it/s, gpt_loss=0.059, lr=4.91e-6] 
+
Train step of epoch 0:  12%|█▏        | 679/5616 [08:44<1:04:32,  1.27it/s, gpt_loss=0.059, lr=4.91e-6]
+
Train step of epoch 0:  12%|█▏        | 679/5616 [08:44<1:04:32,  1.27it/s, gpt_loss=0.0263, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 680/5616 [08:44<1:02:34,  1.31it/s, gpt_loss=0.0263, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 680/5616 [08:45<1:02:34,  1.31it/s, gpt_loss=0.0358, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 681/5616 [08:45<1:01:04,  1.35it/s, gpt_loss=0.0358, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 681/5616 [08:46<1:01:04,  1.35it/s, gpt_loss=0.114, lr=4.9e-6] 
+
Train step of epoch 0:  12%|█▏        | 682/5616 [08:46<59:55,  1.37it/s, gpt_loss=0.114, lr=4.9e-6]  
+
Train step of epoch 0:  12%|█▏        | 682/5616 [08:47<59:55,  1.37it/s, gpt_loss=0.0295, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 683/5616 [08:47<1:00:36,  1.36it/s, gpt_loss=0.0295, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 683/5616 [08:47<1:00:36,  1.36it/s, gpt_loss=0.0568, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 684/5616 [08:47<1:02:38,  1.31it/s, gpt_loss=0.0568, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 684/5616 [08:48<1:02:38,  1.31it/s, gpt_loss=0.041, lr=4.9e-6] 
+
Train step of epoch 0:  12%|█▏        | 685/5616 [08:48<59:45,  1.38it/s, gpt_loss=0.041, lr=4.9e-6]  
+
Train step of epoch 0:  12%|█▏        | 685/5616 [08:49<59:45,  1.38it/s, gpt_loss=0.0495, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 686/5616 [08:49<1:00:25,  1.36it/s, gpt_loss=0.0495, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 686/5616 [08:50<1:00:25,  1.36it/s, gpt_loss=0.0585, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 687/5616 [08:50<59:53,  1.37it/s, gpt_loss=0.0585, lr=4.9e-6]  
+
Train step of epoch 0:  12%|█▏        | 687/5616 [08:50<59:53,  1.37it/s, gpt_loss=0.0135, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 688/5616 [08:50<1:02:26,  1.32it/s, gpt_loss=0.0135, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 688/5616 [08:51<1:02:26,  1.32it/s, gpt_loss=0.0827, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 689/5616 [08:51<1:01:48,  1.33it/s, gpt_loss=0.0827, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 689/5616 [08:52<1:01:48,  1.33it/s, gpt_loss=0.125, lr=4.9e-6] 
+
Train step of epoch 0:  12%|█▏        | 690/5616 [08:52<1:00:11,  1.36it/s, gpt_loss=0.125, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 690/5616 [08:53<1:00:11,  1.36it/s, gpt_loss=0.0762, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 691/5616 [08:53<1:01:35,  1.33it/s, gpt_loss=0.0762, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 691/5616 [08:53<1:01:35,  1.33it/s, gpt_loss=0.0527, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 692/5616 [08:53<1:03:14,  1.30it/s, gpt_loss=0.0527, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 692/5616 [08:54<1:03:14,  1.30it/s, gpt_loss=0.0869, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 693/5616 [08:54<1:01:46,  1.33it/s, gpt_loss=0.0869, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 693/5616 [08:55<1:01:46,  1.33it/s, gpt_loss=0.0103, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 694/5616 [08:55<1:00:21,  1.36it/s, gpt_loss=0.0103, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 694/5616 [08:55<1:00:21,  1.36it/s, gpt_loss=0.163, lr=4.9e-6] 
+
Train step of epoch 0:  12%|█▏        | 695/5616 [08:55<58:03,  1.41it/s, gpt_loss=0.163, lr=4.9e-6]  
+
Train step of epoch 0:  12%|█▏        | 695/5616 [08:56<58:03,  1.41it/s, gpt_loss=0.104, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 696/5616 [08:56<1:02:06,  1.32it/s, gpt_loss=0.104, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 696/5616 [08:57<1:02:06,  1.32it/s, gpt_loss=0.0521, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 697/5616 [08:57<1:09:30,  1.18it/s, gpt_loss=0.0521, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 697/5616 [08:58<1:09:30,  1.18it/s, gpt_loss=0.0898, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 698/5616 [08:58<1:07:49,  1.21it/s, gpt_loss=0.0898, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 698/5616 [08:59<1:07:49,  1.21it/s, gpt_loss=0.0709, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 699/5616 [08:59<1:07:56,  1.21it/s, gpt_loss=0.0709, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 699/5616 [09:00<1:07:56,  1.21it/s, gpt_loss=0.0456, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 700/5616 [09:00<1:11:33,  1.14it/s, gpt_loss=0.0456, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 700/5616 [09:01<1:11:33,  1.14it/s, gpt_loss=0.0148, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 701/5616 [09:01<1:08:48,  1.19it/s, gpt_loss=0.0148, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▏        | 701/5616 [09:01<1:08:48,  1.19it/s, gpt_loss=0.0225, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▎        | 702/5616 [09:01<1:03:46,  1.28it/s, gpt_loss=0.0225, lr=4.9e-6]
+
Train step of epoch 0:  12%|█▎        | 702/5616 [09:02<1:03:46,  1.28it/s, gpt_loss=0.0609, lr=4.9e-6]
+
Train step of epoch 0:  13%|█▎        | 703/5616 [09:02<1:01:16,  1.34it/s, gpt_loss=0.0609, lr=4.9e-6]
+
Train step of epoch 0:  13%|█▎        | 703/5616 [09:03<1:01:16,  1.34it/s, gpt_loss=0.0401, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 704/5616 [09:03<1:01:31,  1.33it/s, gpt_loss=0.0401, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 704/5616 [09:04<1:01:31,  1.33it/s, gpt_loss=0.0475, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 705/5616 [09:04<59:45,  1.37it/s, gpt_loss=0.0475, lr=4.89e-6]  
+
Train step of epoch 0:  13%|█▎        | 705/5616 [09:04<59:45,  1.37it/s, gpt_loss=0.0668, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 706/5616 [09:04<59:36,  1.37it/s, gpt_loss=0.0668, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 706/5616 [09:05<59:36,  1.37it/s, gpt_loss=0.0634, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 707/5616 [09:05<57:59,  1.41it/s, gpt_loss=0.0634, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 707/5616 [09:06<57:59,  1.41it/s, gpt_loss=0.0165, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 708/5616 [09:06<1:01:40,  1.33it/s, gpt_loss=0.0165, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 708/5616 [09:06<1:01:40,  1.33it/s, gpt_loss=0.0089, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 709/5616 [09:06<1:00:18,  1.36it/s, gpt_loss=0.0089, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 709/5616 [09:07<1:00:18,  1.36it/s, gpt_loss=0.077, lr=4.89e-6] 
+
Train step of epoch 0:  13%|█▎        | 710/5616 [09:07<58:04,  1.41it/s, gpt_loss=0.077, lr=4.89e-6]  
+
Train step of epoch 0:  13%|█▎        | 710/5616 [09:08<58:04,  1.41it/s, gpt_loss=0.0754, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 711/5616 [09:08<59:33,  1.37it/s, gpt_loss=0.0754, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 711/5616 [09:09<59:33,  1.37it/s, gpt_loss=0.0939, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 712/5616 [09:09<1:02:03,  1.32it/s, gpt_loss=0.0939, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 712/5616 [09:09<1:02:03,  1.32it/s, gpt_loss=0.0322, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 713/5616 [09:09<1:02:00,  1.32it/s, gpt_loss=0.0322, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 713/5616 [09:10<1:02:00,  1.32it/s, gpt_loss=0.0238, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 714/5616 [09:10<1:00:34,  1.35it/s, gpt_loss=0.0238, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 714/5616 [09:11<1:00:34,  1.35it/s, gpt_loss=0.0772, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 715/5616 [09:11<59:02,  1.38it/s, gpt_loss=0.0772, lr=4.89e-6]  
+
Train step of epoch 0:  13%|█▎        | 715/5616 [09:12<59:02,  1.38it/s, gpt_loss=0.0261, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 716/5616 [09:12<59:52,  1.36it/s, gpt_loss=0.0261, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 716/5616 [09:12<59:52,  1.36it/s, gpt_loss=0.0278, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 717/5616 [09:12<58:51,  1.39it/s, gpt_loss=0.0278, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 717/5616 [09:13<58:51,  1.39it/s, gpt_loss=0.0257, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 718/5616 [09:13<57:08,  1.43it/s, gpt_loss=0.0257, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 718/5616 [09:14<57:08,  1.43it/s, gpt_loss=0.0862, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 719/5616 [09:14<56:46,  1.44it/s, gpt_loss=0.0862, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 719/5616 [09:14<56:46,  1.44it/s, gpt_loss=0.0618, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 720/5616 [09:14<56:41,  1.44it/s, gpt_loss=0.0618, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 720/5616 [09:15<56:41,  1.44it/s, gpt_loss=0.0899, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 721/5616 [09:15<58:33,  1.39it/s, gpt_loss=0.0899, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 721/5616 [09:16<58:33,  1.39it/s, gpt_loss=0.0267, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 722/5616 [09:16<59:16,  1.38it/s, gpt_loss=0.0267, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 722/5616 [09:17<59:16,  1.38it/s, gpt_loss=0.0343, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 723/5616 [09:17<58:07,  1.40it/s, gpt_loss=0.0343, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 723/5616 [09:17<58:07,  1.40it/s, gpt_loss=0.0233, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 724/5616 [09:17<1:01:37,  1.32it/s, gpt_loss=0.0233, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 724/5616 [09:18<1:01:37,  1.32it/s, gpt_loss=0.00706, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 725/5616 [09:18<59:37,  1.37it/s, gpt_loss=0.00706, lr=4.89e-6]  
+
Train step of epoch 0:  13%|█▎        | 725/5616 [09:19<59:37,  1.37it/s, gpt_loss=0.0683, lr=4.89e-6] 
+
Train step of epoch 0:  13%|█▎        | 726/5616 [09:19<1:00:24,  1.35it/s, gpt_loss=0.0683, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 726/5616 [09:19<1:00:24,  1.35it/s, gpt_loss=0.0116, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 727/5616 [09:19<58:01,  1.40it/s, gpt_loss=0.0116, lr=4.89e-6]  
+
Train step of epoch 0:  13%|█▎        | 727/5616 [09:20<58:01,  1.40it/s, gpt_loss=0.0716, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 728/5616 [09:20<59:15,  1.37it/s, gpt_loss=0.0716, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 728/5616 [09:21<59:15,  1.37it/s, gpt_loss=0.0134, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 729/5616 [09:21<58:40,  1.39it/s, gpt_loss=0.0134, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 729/5616 [09:22<58:40,  1.39it/s, gpt_loss=0.0563, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 730/5616 [09:22<58:12,  1.40it/s, gpt_loss=0.0563, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 730/5616 [09:22<58:12,  1.40it/s, gpt_loss=0.1, lr=4.89e-6]   
+
Train step of epoch 0:  13%|█▎        | 731/5616 [09:22<56:34,  1.44it/s, gpt_loss=0.1, lr=4.89e-6]
+
Train step of epoch 0:  13%|█▎        | 731/5616 [09:23<56:34,  1.44it/s, gpt_loss=0.0124, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 732/5616 [09:23<56:32,  1.44it/s, gpt_loss=0.0124, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 732/5616 [09:24<56:32,  1.44it/s, gpt_loss=0.127, lr=4.88e-6] 
+
Train step of epoch 0:  13%|█▎        | 733/5616 [09:24<57:16,  1.42it/s, gpt_loss=0.127, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 733/5616 [09:24<57:16,  1.42it/s, gpt_loss=0.00998, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 734/5616 [09:24<56:31,  1.44it/s, gpt_loss=0.00998, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 734/5616 [09:25<56:31,  1.44it/s, gpt_loss=0.0872, lr=4.88e-6] 
+
Train step of epoch 0:  13%|█▎        | 735/5616 [09:25<54:57,  1.48it/s, gpt_loss=0.0872, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 735/5616 [09:26<54:57,  1.48it/s, gpt_loss=0.0681, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 736/5616 [09:26<57:12,  1.42it/s, gpt_loss=0.0681, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 736/5616 [09:26<57:12,  1.42it/s, gpt_loss=0.0951, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 737/5616 [09:26<56:48,  1.43it/s, gpt_loss=0.0951, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 737/5616 [09:27<56:48,  1.43it/s, gpt_loss=0.0305, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 738/5616 [09:27<56:54,  1.43it/s, gpt_loss=0.0305, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 738/5616 [09:28<56:54,  1.43it/s, gpt_loss=0.0384, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 739/5616 [09:28<56:50,  1.43it/s, gpt_loss=0.0384, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 739/5616 [09:29<56:50,  1.43it/s, gpt_loss=0.0181, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 740/5616 [09:29<59:17,  1.37it/s, gpt_loss=0.0181, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 740/5616 [09:30<59:17,  1.37it/s, gpt_loss=0.0121, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 741/5616 [09:30<1:01:40,  1.32it/s, gpt_loss=0.0121, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 741/5616 [09:30<1:01:40,  1.32it/s, gpt_loss=0.0321, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 742/5616 [09:30<1:00:16,  1.35it/s, gpt_loss=0.0321, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 742/5616 [09:31<1:00:16,  1.35it/s, gpt_loss=0.0258, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 743/5616 [09:31<59:35,  1.36it/s, gpt_loss=0.0258, lr=4.88e-6]  
+
Train step of epoch 0:  13%|█▎        | 743/5616 [09:32<59:35,  1.36it/s, gpt_loss=0.0632, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 744/5616 [09:32<1:02:21,  1.30it/s, gpt_loss=0.0632, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 744/5616 [09:33<1:02:21,  1.30it/s, gpt_loss=0.0782, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 745/5616 [09:33<1:01:31,  1.32it/s, gpt_loss=0.0782, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 745/5616 [09:33<1:01:31,  1.32it/s, gpt_loss=0.068, lr=4.88e-6] 
+
Train step of epoch 0:  13%|█▎        | 746/5616 [09:33<1:00:03,  1.35it/s, gpt_loss=0.068, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 746/5616 [09:34<1:00:03,  1.35it/s, gpt_loss=0.029, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 747/5616 [09:34<58:16,  1.39it/s, gpt_loss=0.029, lr=4.88e-6]  
+
Train step of epoch 0:  13%|█▎        | 747/5616 [09:35<58:16,  1.39it/s, gpt_loss=0.0659, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 748/5616 [09:35<1:03:17,  1.28it/s, gpt_loss=0.0659, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 748/5616 [09:36<1:03:17,  1.28it/s, gpt_loss=0.0392, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 749/5616 [09:36<1:05:17,  1.24it/s, gpt_loss=0.0392, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 749/5616 [09:36<1:05:17,  1.24it/s, gpt_loss=0.0104, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 750/5616 [09:36<1:01:24,  1.32it/s, gpt_loss=0.0104, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 750/5616 [09:37<1:01:24,  1.32it/s, gpt_loss=0.0374, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 751/5616 [09:37<1:00:29,  1.34it/s, gpt_loss=0.0374, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 751/5616 [09:38<1:00:29,  1.34it/s, gpt_loss=0.0129, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 752/5616 [09:38<1:01:05,  1.33it/s, gpt_loss=0.0129, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 752/5616 [09:39<1:01:05,  1.33it/s, gpt_loss=0.101, lr=4.88e-6] 
+
Train step of epoch 0:  13%|█▎        | 753/5616 [09:39<1:02:40,  1.29it/s, gpt_loss=0.101, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 753/5616 [09:39<1:02:40,  1.29it/s, gpt_loss=0.118, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 754/5616 [09:39<59:42,  1.36it/s, gpt_loss=0.118, lr=4.88e-6]  
+
Train step of epoch 0:  13%|█▎        | 754/5616 [09:40<59:42,  1.36it/s, gpt_loss=0.0127, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 755/5616 [09:40<58:40,  1.38it/s, gpt_loss=0.0127, lr=4.88e-6]
+
Train step of epoch 0:  13%|█▎        | 755/5616 [09:41<58:40,  1.38it/s, gpt_loss=0.0161, lr=4.87e-6]
+
Train step of epoch 0:  13%|█▎        | 756/5616 [09:41<59:39,  1.36it/s, gpt_loss=0.0161, lr=4.87e-6]
+
Train step of epoch 0:  13%|█▎        | 756/5616 [09:41<59:39,  1.36it/s, gpt_loss=0.0255, lr=4.87e-6]
+
Train step of epoch 0:  13%|█▎        | 757/5616 [09:41<57:43,  1.40it/s, gpt_loss=0.0255, lr=4.87e-6]
+
Train step of epoch 0:  13%|█▎        | 757/5616 [09:42<57:43,  1.40it/s, gpt_loss=0.037, lr=4.87e-6] 
+
Train step of epoch 0:  13%|█▎        | 758/5616 [09:42<58:33,  1.38it/s, gpt_loss=0.037, lr=4.87e-6]
+
Train step of epoch 0:  13%|█▎        | 758/5616 [09:43<58:33,  1.38it/s, gpt_loss=0.063, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 759/5616 [09:43<57:44,  1.40it/s, gpt_loss=0.063, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 759/5616 [09:44<57:44,  1.40it/s, gpt_loss=0.034, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 760/5616 [09:44<59:05,  1.37it/s, gpt_loss=0.034, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 760/5616 [09:44<59:05,  1.37it/s, gpt_loss=0.034, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 761/5616 [09:44<58:36,  1.38it/s, gpt_loss=0.034, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 761/5616 [09:45<58:36,  1.38it/s, gpt_loss=0.0763, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 762/5616 [09:45<58:38,  1.38it/s, gpt_loss=0.0763, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 762/5616 [09:46<58:38,  1.38it/s, gpt_loss=0.0124, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 763/5616 [09:46<59:20,  1.36it/s, gpt_loss=0.0124, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 763/5616 [09:47<59:20,  1.36it/s, gpt_loss=0.0059, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 764/5616 [09:47<1:01:27,  1.32it/s, gpt_loss=0.0059, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 764/5616 [09:47<1:01:27,  1.32it/s, gpt_loss=0.0824, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 765/5616 [09:47<1:01:36,  1.31it/s, gpt_loss=0.0824, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 765/5616 [09:48<1:01:36,  1.31it/s, gpt_loss=0.068, lr=4.87e-6] 
+
Train step of epoch 0:  14%|█▎        | 766/5616 [09:48<59:41,  1.35it/s, gpt_loss=0.068, lr=4.87e-6]  
+
Train step of epoch 0:  14%|█▎        | 766/5616 [09:49<59:41,  1.35it/s, gpt_loss=0.0607, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 767/5616 [09:49<1:09:11,  1.17it/s, gpt_loss=0.0607, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 767/5616 [09:50<1:09:11,  1.17it/s, gpt_loss=0.103, lr=4.87e-6] 
+
Train step of epoch 0:  14%|█▎        | 768/5616 [09:50<1:10:37,  1.14it/s, gpt_loss=0.103, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 768/5616 [09:51<1:10:37,  1.14it/s, gpt_loss=0.0153, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 769/5616 [09:51<1:04:52,  1.25it/s, gpt_loss=0.0153, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 769/5616 [09:51<1:04:52,  1.25it/s, gpt_loss=0.143, lr=4.87e-6] 
+
Train step of epoch 0:  14%|█▎        | 770/5616 [09:51<1:03:02,  1.28it/s, gpt_loss=0.143, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 770/5616 [09:52<1:03:02,  1.28it/s, gpt_loss=0.0653, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 771/5616 [09:52<1:00:11,  1.34it/s, gpt_loss=0.0653, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 771/5616 [09:53<1:00:11,  1.34it/s, gpt_loss=0.0308, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 772/5616 [09:53<1:07:43,  1.19it/s, gpt_loss=0.0308, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▎        | 772/5616 [09:54<1:07:43,  1.19it/s, gpt_loss=0.0494, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▍        | 773/5616 [09:54<1:06:53,  1.21it/s, gpt_loss=0.0494, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▍        | 773/5616 [09:55<1:06:53,  1.21it/s, gpt_loss=0.0236, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▍        | 774/5616 [09:55<1:06:18,  1.22it/s, gpt_loss=0.0236, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▍        | 774/5616 [09:55<1:06:18,  1.22it/s, gpt_loss=0.00231, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▍        | 775/5616 [09:55<1:02:54,  1.28it/s, gpt_loss=0.00231, lr=4.87e-6]
+
Train step of epoch 0:  14%|█▍        | 775/5616 [09:56<1:02:54,  1.28it/s, gpt_loss=0.0202, lr=4.86e-6] 
+
Train step of epoch 0:  14%|█▍        | 776/5616 [09:56<1:02:29,  1.29it/s, gpt_loss=0.0202, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 776/5616 [09:57<1:02:29,  1.29it/s, gpt_loss=0.031, lr=4.86e-6] 
+
Train step of epoch 0:  14%|█▍        | 777/5616 [09:57<1:03:00,  1.28it/s, gpt_loss=0.031, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 777/5616 [09:58<1:03:00,  1.28it/s, gpt_loss=0.131, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 778/5616 [09:58<1:01:07,  1.32it/s, gpt_loss=0.131, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 778/5616 [09:58<1:01:07,  1.32it/s, gpt_loss=0.0124, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 779/5616 [09:58<58:20,  1.38it/s, gpt_loss=0.0124, lr=4.86e-6]  
+
Train step of epoch 0:  14%|█▍        | 779/5616 [09:59<58:20,  1.38it/s, gpt_loss=0.0443, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 780/5616 [09:59<1:01:58,  1.30it/s, gpt_loss=0.0443, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 780/5616 [10:00<1:01:58,  1.30it/s, gpt_loss=0.0342, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 781/5616 [10:00<1:00:02,  1.34it/s, gpt_loss=0.0342, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 781/5616 [10:01<1:00:02,  1.34it/s, gpt_loss=0.0515, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 782/5616 [10:01<1:07:56,  1.19it/s, gpt_loss=0.0515, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 782/5616 [10:02<1:07:56,  1.19it/s, gpt_loss=0.0263, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 783/5616 [10:02<1:08:36,  1.17it/s, gpt_loss=0.0263, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 783/5616 [10:03<1:08:36,  1.17it/s, gpt_loss=0.011, lr=4.86e-6] 
+
Train step of epoch 0:  14%|█▍        | 784/5616 [10:03<1:06:07,  1.22it/s, gpt_loss=0.011, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 784/5616 [10:03<1:06:07,  1.22it/s, gpt_loss=0.0303, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 785/5616 [10:03<1:02:48,  1.28it/s, gpt_loss=0.0303, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 785/5616 [10:04<1:02:48,  1.28it/s, gpt_loss=0.0472, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 786/5616 [10:04<1:02:31,  1.29it/s, gpt_loss=0.0472, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 786/5616 [10:05<1:02:31,  1.29it/s, gpt_loss=0.022, lr=4.86e-6] 
+
Train step of epoch 0:  14%|█▍        | 787/5616 [10:05<1:00:37,  1.33it/s, gpt_loss=0.022, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 787/5616 [10:06<1:00:37,  1.33it/s, gpt_loss=0.0868, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 788/5616 [10:06<1:02:24,  1.29it/s, gpt_loss=0.0868, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 788/5616 [10:06<1:02:24,  1.29it/s, gpt_loss=0.0485, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 789/5616 [10:06<58:29,  1.38it/s, gpt_loss=0.0485, lr=4.86e-6]  
+
Train step of epoch 0:  14%|█▍        | 789/5616 [10:07<58:29,  1.38it/s, gpt_loss=0.0264, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 790/5616 [10:07<59:02,  1.36it/s, gpt_loss=0.0264, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 790/5616 [10:08<59:02,  1.36it/s, gpt_loss=0.0468, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 791/5616 [10:08<58:45,  1.37it/s, gpt_loss=0.0468, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 791/5616 [10:09<58:45,  1.37it/s, gpt_loss=0.118, lr=4.86e-6] 
+
Train step of epoch 0:  14%|█▍        | 792/5616 [10:09<1:02:49,  1.28it/s, gpt_loss=0.118, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 792/5616 [10:09<1:02:49,  1.28it/s, gpt_loss=0.0411, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 793/5616 [10:09<1:00:30,  1.33it/s, gpt_loss=0.0411, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 793/5616 [10:10<1:00:30,  1.33it/s, gpt_loss=0.144, lr=4.86e-6] 
+
Train step of epoch 0:  14%|█▍        | 794/5616 [10:10<1:02:18,  1.29it/s, gpt_loss=0.144, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 794/5616 [10:11<1:02:18,  1.29it/s, gpt_loss=0.0151, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 795/5616 [10:11<1:01:01,  1.32it/s, gpt_loss=0.0151, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 795/5616 [10:12<1:01:01,  1.32it/s, gpt_loss=0.0146, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 796/5616 [10:12<59:59,  1.34it/s, gpt_loss=0.0146, lr=4.86e-6]  
+
Train step of epoch 0:  14%|█▍        | 796/5616 [10:12<59:59,  1.34it/s, gpt_loss=0.0305, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 797/5616 [10:12<57:15,  1.40it/s, gpt_loss=0.0305, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 797/5616 [10:13<57:15,  1.40it/s, gpt_loss=0.0789, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 798/5616 [10:13<1:02:50,  1.28it/s, gpt_loss=0.0789, lr=4.86e-6]
+
Train step of epoch 0:  14%|█▍        | 798/5616 [10:14<1:02:50,  1.28it/s, gpt_loss=0.197, lr=4.86e-6] 
+
Train step of epoch 0:  14%|█▍        | 799/5616 [10:14<1:02:05,  1.29it/s, gpt_loss=0.197, lr=4.86e-6][2025-06-30 06:48:23,801] [INFO] [logging.py:128:log_dist] [Rank 0] step=200, skipped=0, lr=[4.853857372381794e-06, 4.853857372381794e-06], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 06:48:23,802] [INFO] [timer.py:264:stop] epoch=0/micro_step=800/global_step=200, RunningAvgSamplesPerSec=42.30022955415716, CurrSamplesPerSec=40.42100537631044, MemAllocated=16.03GB, MaxMemAllocated=51.79GB
+
+
Train step of epoch 0:  14%|█▍        | 799/5616 [10:15<1:02:05,  1.29it/s, gpt_loss=0.0327, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 800/5616 [10:15<1:04:21,  1.25it/s, gpt_loss=0.0327, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 800/5616 [10:15<1:04:21,  1.25it/s, gpt_loss=0.0528, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 801/5616 [10:15<1:00:41,  1.32it/s, gpt_loss=0.0528, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 801/5616 [10:16<1:00:41,  1.32it/s, gpt_loss=0.0788, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 802/5616 [10:16<58:29,  1.37it/s, gpt_loss=0.0788, lr=4.85e-6]  
+
Train step of epoch 0:  14%|█▍        | 802/5616 [10:17<58:29,  1.37it/s, gpt_loss=0.0351, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 803/5616 [10:17<58:58,  1.36it/s, gpt_loss=0.0351, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 803/5616 [10:18<58:58,  1.36it/s, gpt_loss=0.04, lr=4.85e-6]  
+
Train step of epoch 0:  14%|█▍        | 804/5616 [10:18<59:52,  1.34it/s, gpt_loss=0.04, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 804/5616 [10:18<59:52,  1.34it/s, gpt_loss=0.0503, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 805/5616 [10:18<59:30,  1.35it/s, gpt_loss=0.0503, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 805/5616 [10:19<59:30,  1.35it/s, gpt_loss=0.0149, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 806/5616 [10:19<56:50,  1.41it/s, gpt_loss=0.0149, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 806/5616 [10:20<56:50,  1.41it/s, gpt_loss=0.0366, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 807/5616 [10:20<57:44,  1.39it/s, gpt_loss=0.0366, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 807/5616 [10:20<57:44,  1.39it/s, gpt_loss=0.0953, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 808/5616 [10:20<58:10,  1.38it/s, gpt_loss=0.0953, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 808/5616 [10:21<58:10,  1.38it/s, gpt_loss=0.062, lr=4.85e-6] 
+
Train step of epoch 0:  14%|█▍        | 809/5616 [10:21<57:43,  1.39it/s, gpt_loss=0.062, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 809/5616 [10:22<57:43,  1.39it/s, gpt_loss=0.0577, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 810/5616 [10:22<56:45,  1.41it/s, gpt_loss=0.0577, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 810/5616 [10:23<56:45,  1.41it/s, gpt_loss=0.0224, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 811/5616 [10:23<57:32,  1.39it/s, gpt_loss=0.0224, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 811/5616 [10:23<57:32,  1.39it/s, gpt_loss=0.0191, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 812/5616 [10:23<1:01:50,  1.29it/s, gpt_loss=0.0191, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 812/5616 [10:24<1:01:50,  1.29it/s, gpt_loss=0.0271, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 813/5616 [10:24<1:00:41,  1.32it/s, gpt_loss=0.0271, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 813/5616 [10:25<1:00:41,  1.32it/s, gpt_loss=0.0488, lr=4.85e-6]
+
Train step of epoch 0:  14%|█▍        | 814/5616 [10:25<58:37,  1.37it/s, gpt_loss=0.0488, lr=4.85e-6]  
+
Train step of epoch 0:  14%|█▍        | 814/5616 [10:26<58:37,  1.37it/s, gpt_loss=0.0343, lr=4.85e-6]
+
Train step of epoch 0:  15%|█▍        | 815/5616 [10:26<57:38,  1.39it/s, gpt_loss=0.0343, lr=4.85e-6]
+
Train step of epoch 0:  15%|█▍        | 815/5616 [10:26<57:38,  1.39it/s, gpt_loss=0.225, lr=4.85e-6] 
+
Train step of epoch 0:  15%|█▍        | 816/5616 [10:26<1:00:08,  1.33it/s, gpt_loss=0.225, lr=4.85e-6]
+
Train step of epoch 0:  15%|█▍        | 816/5616 [10:27<1:00:08,  1.33it/s, gpt_loss=0.024, lr=4.85e-6]
+
Train step of epoch 0:  15%|█▍        | 817/5616 [10:27<1:01:10,  1.31it/s, gpt_loss=0.024, lr=4.85e-6]
+
Train step of epoch 0:  15%|█▍        | 817/5616 [10:28<1:01:10,  1.31it/s, gpt_loss=0.0534, lr=4.85e-6]
+
Train step of epoch 0:  15%|█▍        | 818/5616 [10:28<58:22,  1.37it/s, gpt_loss=0.0534, lr=4.85e-6]  
+
Train step of epoch 0:  15%|█▍        | 818/5616 [10:29<58:22,  1.37it/s, gpt_loss=0.0682, lr=4.85e-6]
+
Train step of epoch 0:  15%|█▍        | 819/5616 [10:29<59:06,  1.35it/s, gpt_loss=0.0682, lr=4.85e-6]
+
Train step of epoch 0:  15%|█▍        | 819/5616 [10:29<59:06,  1.35it/s, gpt_loss=0.0243, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 820/5616 [10:29<59:10,  1.35it/s, gpt_loss=0.0243, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 820/5616 [10:30<59:10,  1.35it/s, gpt_loss=0.0119, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 821/5616 [10:30<57:19,  1.39it/s, gpt_loss=0.0119, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 821/5616 [10:31<57:19,  1.39it/s, gpt_loss=0.0509, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 822/5616 [10:31<1:01:14,  1.30it/s, gpt_loss=0.0509, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 822/5616 [10:32<1:01:14,  1.30it/s, gpt_loss=0.0746, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 823/5616 [10:32<58:20,  1.37it/s, gpt_loss=0.0746, lr=4.84e-6]  
+
Train step of epoch 0:  15%|█▍        | 823/5616 [10:32<58:20,  1.37it/s, gpt_loss=0.0463, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 824/5616 [10:32<58:48,  1.36it/s, gpt_loss=0.0463, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 824/5616 [10:33<58:48,  1.36it/s, gpt_loss=0.0907, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 825/5616 [10:33<1:01:12,  1.30it/s, gpt_loss=0.0907, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 825/5616 [10:34<1:01:12,  1.30it/s, gpt_loss=0.0317, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 826/5616 [10:34<1:01:09,  1.31it/s, gpt_loss=0.0317, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 826/5616 [10:35<1:01:09,  1.31it/s, gpt_loss=0.12, lr=4.84e-6]  
+
Train step of epoch 0:  15%|█▍        | 827/5616 [10:35<58:26,  1.37it/s, gpt_loss=0.12, lr=4.84e-6]  
+
Train step of epoch 0:  15%|█▍        | 827/5616 [10:35<58:26,  1.37it/s, gpt_loss=0.0396, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 828/5616 [10:35<1:00:59,  1.31it/s, gpt_loss=0.0396, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 828/5616 [10:36<1:00:59,  1.31it/s, gpt_loss=0.0254, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 829/5616 [10:36<59:26,  1.34it/s, gpt_loss=0.0254, lr=4.84e-6]  
+
Train step of epoch 0:  15%|█▍        | 829/5616 [10:37<59:26,  1.34it/s, gpt_loss=0.0564, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 830/5616 [10:37<59:31,  1.34it/s, gpt_loss=0.0564, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 830/5616 [10:38<59:31,  1.34it/s, gpt_loss=0.0301, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 831/5616 [10:38<58:48,  1.36it/s, gpt_loss=0.0301, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 831/5616 [10:38<58:48,  1.36it/s, gpt_loss=0.0563, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 832/5616 [10:38<1:01:58,  1.29it/s, gpt_loss=0.0563, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 832/5616 [10:39<1:01:58,  1.29it/s, gpt_loss=0.0635, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 833/5616 [10:39<1:00:52,  1.31it/s, gpt_loss=0.0635, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 833/5616 [10:40<1:00:52,  1.31it/s, gpt_loss=0.11, lr=4.84e-6]  
+
Train step of epoch 0:  15%|█▍        | 834/5616 [10:40<1:02:25,  1.28it/s, gpt_loss=0.11, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 834/5616 [10:41<1:02:25,  1.28it/s, gpt_loss=0.0168, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 835/5616 [10:41<1:00:37,  1.31it/s, gpt_loss=0.0168, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 835/5616 [10:42<1:00:37,  1.31it/s, gpt_loss=0.0321, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 836/5616 [10:42<1:02:06,  1.28it/s, gpt_loss=0.0321, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 836/5616 [10:42<1:02:06,  1.28it/s, gpt_loss=0.109, lr=4.84e-6] 
+
Train step of epoch 0:  15%|█▍        | 837/5616 [10:42<1:03:14,  1.26it/s, gpt_loss=0.109, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 837/5616 [10:43<1:03:14,  1.26it/s, gpt_loss=0.0321, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 838/5616 [10:43<1:00:45,  1.31it/s, gpt_loss=0.0321, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 838/5616 [10:44<1:00:45,  1.31it/s, gpt_loss=0.0987, lr=4.84e-6]
+
Train step of epoch 0:  15%|█▍        | 839/5616 [10:44<58:47,  1.35it/s, gpt_loss=0.0987, lr=4.84e-6]  
+
Train step of epoch 0:  15%|█▍        | 839/5616 [10:44<58:47,  1.35it/s, gpt_loss=0.0163, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▍        | 840/5616 [10:44<59:04,  1.35it/s, gpt_loss=0.0163, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▍        | 840/5616 [10:45<59:04,  1.35it/s, gpt_loss=0.0665, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▍        | 841/5616 [10:45<59:53,  1.33it/s, gpt_loss=0.0665, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▍        | 841/5616 [10:46<59:53,  1.33it/s, gpt_loss=0.00297, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▍        | 842/5616 [10:46<58:07,  1.37it/s, gpt_loss=0.00297, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▍        | 842/5616 [10:47<58:07,  1.37it/s, gpt_loss=0.00162, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 843/5616 [10:47<56:51,  1.40it/s, gpt_loss=0.00162, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 843/5616 [10:47<56:51,  1.40it/s, gpt_loss=0.0167, lr=4.83e-6] 
+
Train step of epoch 0:  15%|█▌        | 844/5616 [10:47<57:09,  1.39it/s, gpt_loss=0.0167, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 844/5616 [10:48<57:09,  1.39it/s, gpt_loss=0.0993, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 845/5616 [10:48<59:28,  1.34it/s, gpt_loss=0.0993, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 845/5616 [10:49<59:28,  1.34it/s, gpt_loss=0.00772, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 846/5616 [10:49<57:47,  1.38it/s, gpt_loss=0.00772, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 846/5616 [10:50<57:47,  1.38it/s, gpt_loss=0.0365, lr=4.83e-6] 
+
Train step of epoch 0:  15%|█▌        | 847/5616 [10:50<57:50,  1.37it/s, gpt_loss=0.0365, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 847/5616 [10:50<57:50,  1.37it/s, gpt_loss=0.0955, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 848/5616 [10:50<1:00:38,  1.31it/s, gpt_loss=0.0955, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 848/5616 [10:51<1:00:38,  1.31it/s, gpt_loss=0.0168, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 849/5616 [10:51<58:24,  1.36it/s, gpt_loss=0.0168, lr=4.83e-6]  
+
Train step of epoch 0:  15%|█▌        | 849/5616 [10:52<58:24,  1.36it/s, gpt_loss=0.0101, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 850/5616 [10:52<58:27,  1.36it/s, gpt_loss=0.0101, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 850/5616 [10:52<58:27,  1.36it/s, gpt_loss=0.0389, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 851/5616 [10:52<56:50,  1.40it/s, gpt_loss=0.0389, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 851/5616 [10:53<56:50,  1.40it/s, gpt_loss=0.0495, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 852/5616 [10:53<58:19,  1.36it/s, gpt_loss=0.0495, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 852/5616 [10:54<58:19,  1.36it/s, gpt_loss=0.0658, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 853/5616 [10:54<56:14,  1.41it/s, gpt_loss=0.0658, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 853/5616 [10:55<56:14,  1.41it/s, gpt_loss=0.00606, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 854/5616 [10:55<55:53,  1.42it/s, gpt_loss=0.00606, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 854/5616 [10:55<55:53,  1.42it/s, gpt_loss=0.107, lr=4.83e-6]  
+
Train step of epoch 0:  15%|█▌        | 855/5616 [10:55<55:10,  1.44it/s, gpt_loss=0.107, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 855/5616 [10:56<55:10,  1.44it/s, gpt_loss=0.0058, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 856/5616 [10:56<56:47,  1.40it/s, gpt_loss=0.0058, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 856/5616 [10:57<56:47,  1.40it/s, gpt_loss=0.0434, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 857/5616 [10:57<58:57,  1.35it/s, gpt_loss=0.0434, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 857/5616 [10:58<58:57,  1.35it/s, gpt_loss=0.0824, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 858/5616 [10:58<58:43,  1.35it/s, gpt_loss=0.0824, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 858/5616 [10:58<58:43,  1.35it/s, gpt_loss=0.0444, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 859/5616 [10:58<59:04,  1.34it/s, gpt_loss=0.0444, lr=4.83e-6]
+
Train step of epoch 0:  15%|█▌        | 859/5616 [10:59<59:04,  1.34it/s, gpt_loss=0.147, lr=4.82e-6] 
+
Train step of epoch 0:  15%|█▌        | 860/5616 [10:59<59:49,  1.33it/s, gpt_loss=0.147, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 860/5616 [11:00<59:49,  1.33it/s, gpt_loss=0.0612, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 861/5616 [11:00<57:13,  1.38it/s, gpt_loss=0.0612, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 861/5616 [11:01<57:13,  1.38it/s, gpt_loss=0.0173, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 862/5616 [11:01<1:02:11,  1.27it/s, gpt_loss=0.0173, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 862/5616 [11:01<1:02:11,  1.27it/s, gpt_loss=0.0988, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 863/5616 [11:01<1:00:02,  1.32it/s, gpt_loss=0.0988, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 863/5616 [11:02<1:00:02,  1.32it/s, gpt_loss=0.0283, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 864/5616 [11:02<59:31,  1.33it/s, gpt_loss=0.0283, lr=4.82e-6]  
+
Train step of epoch 0:  15%|█▌        | 864/5616 [11:03<59:31,  1.33it/s, gpt_loss=0.0518, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 865/5616 [11:03<58:27,  1.35it/s, gpt_loss=0.0518, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 865/5616 [11:03<58:27,  1.35it/s, gpt_loss=0.0306, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 866/5616 [11:03<57:09,  1.38it/s, gpt_loss=0.0306, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 866/5616 [11:04<57:09,  1.38it/s, gpt_loss=0.118, lr=4.82e-6] 
+
Train step of epoch 0:  15%|█▌        | 867/5616 [11:04<57:38,  1.37it/s, gpt_loss=0.118, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 867/5616 [11:05<57:38,  1.37it/s, gpt_loss=0.0852, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 868/5616 [11:05<1:01:44,  1.28it/s, gpt_loss=0.0852, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 868/5616 [11:06<1:01:44,  1.28it/s, gpt_loss=0.0208, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 869/5616 [11:06<59:43,  1.32it/s, gpt_loss=0.0208, lr=4.82e-6]  
+
Train step of epoch 0:  15%|█▌        | 869/5616 [11:07<59:43,  1.32it/s, gpt_loss=0.0101, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 870/5616 [11:07<1:01:58,  1.28it/s, gpt_loss=0.0101, lr=4.82e-6]
+
Train step of epoch 0:  15%|█▌        | 870/5616 [11:07<1:01:58,  1.28it/s, gpt_loss=0.0353, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 871/5616 [11:07<59:40,  1.33it/s, gpt_loss=0.0353, lr=4.82e-6]  
+
Train step of epoch 0:  16%|█▌        | 871/5616 [11:08<59:40,  1.33it/s, gpt_loss=0.0554, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 872/5616 [11:08<1:01:49,  1.28it/s, gpt_loss=0.0554, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 872/5616 [11:09<1:01:49,  1.28it/s, gpt_loss=0.0297, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 873/5616 [11:09<1:01:46,  1.28it/s, gpt_loss=0.0297, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 873/5616 [11:10<1:01:46,  1.28it/s, gpt_loss=0.0333, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 874/5616 [11:10<1:01:27,  1.29it/s, gpt_loss=0.0333, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 874/5616 [11:11<1:01:27,  1.29it/s, gpt_loss=0.05, lr=4.82e-6]  
+
Train step of epoch 0:  16%|█▌        | 875/5616 [11:11<1:01:31,  1.28it/s, gpt_loss=0.05, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 875/5616 [11:11<1:01:31,  1.28it/s, gpt_loss=0.0446, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 876/5616 [11:11<1:01:49,  1.28it/s, gpt_loss=0.0446, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 876/5616 [11:12<1:01:49,  1.28it/s, gpt_loss=0.0132, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 877/5616 [11:12<59:26,  1.33it/s, gpt_loss=0.0132, lr=4.82e-6]  
+
Train step of epoch 0:  16%|█▌        | 877/5616 [11:13<59:26,  1.33it/s, gpt_loss=0.0287, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 878/5616 [11:13<59:53,  1.32it/s, gpt_loss=0.0287, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 878/5616 [11:14<59:53,  1.32it/s, gpt_loss=0.157, lr=4.82e-6] 
+
Train step of epoch 0:  16%|█▌        | 879/5616 [11:14<58:45,  1.34it/s, gpt_loss=0.157, lr=4.82e-6]
+
Train step of epoch 0:  16%|█▌        | 879/5616 [11:15<58:45,  1.34it/s, gpt_loss=0.0215, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 880/5616 [11:15<1:09:36,  1.13it/s, gpt_loss=0.0215, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 880/5616 [11:15<1:09:36,  1.13it/s, gpt_loss=0.0348, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 881/5616 [11:15<1:05:13,  1.21it/s, gpt_loss=0.0348, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 881/5616 [11:16<1:05:13,  1.21it/s, gpt_loss=0.0411, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 882/5616 [11:16<1:02:06,  1.27it/s, gpt_loss=0.0411, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 882/5616 [11:17<1:02:06,  1.27it/s, gpt_loss=0.0309, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 883/5616 [11:17<59:24,  1.33it/s, gpt_loss=0.0309, lr=4.81e-6]  
+
Train step of epoch 0:  16%|█▌        | 883/5616 [11:18<59:24,  1.33it/s, gpt_loss=0.197, lr=4.81e-6] 
+
Train step of epoch 0:  16%|█▌        | 884/5616 [11:18<59:54,  1.32it/s, gpt_loss=0.197, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 884/5616 [11:19<59:54,  1.32it/s, gpt_loss=0.0475, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 885/5616 [11:19<1:05:26,  1.20it/s, gpt_loss=0.0475, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 885/5616 [11:19<1:05:26,  1.20it/s, gpt_loss=0.0653, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 886/5616 [11:19<1:02:43,  1.26it/s, gpt_loss=0.0653, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 886/5616 [11:20<1:02:43,  1.26it/s, gpt_loss=0.0199, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 887/5616 [11:20<1:00:09,  1.31it/s, gpt_loss=0.0199, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 887/5616 [11:21<1:00:09,  1.31it/s, gpt_loss=0.0266, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 888/5616 [11:21<1:01:20,  1.28it/s, gpt_loss=0.0266, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 888/5616 [11:22<1:01:20,  1.28it/s, gpt_loss=0.144, lr=4.81e-6] 
+
Train step of epoch 0:  16%|█▌        | 889/5616 [11:22<1:01:04,  1.29it/s, gpt_loss=0.144, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 889/5616 [11:22<1:01:04,  1.29it/s, gpt_loss=0.153, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 890/5616 [11:22<1:02:13,  1.27it/s, gpt_loss=0.153, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 890/5616 [11:23<1:02:13,  1.27it/s, gpt_loss=0.0217, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 891/5616 [11:23<1:01:06,  1.29it/s, gpt_loss=0.0217, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 891/5616 [11:24<1:01:06,  1.29it/s, gpt_loss=0.0155, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 892/5616 [11:24<1:01:47,  1.27it/s, gpt_loss=0.0155, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 892/5616 [11:25<1:01:47,  1.27it/s, gpt_loss=0.0729, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 893/5616 [11:25<58:56,  1.34it/s, gpt_loss=0.0729, lr=4.81e-6]  
+
Train step of epoch 0:  16%|█▌        | 893/5616 [11:25<58:56,  1.34it/s, gpt_loss=0.0145, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 894/5616 [11:25<1:00:30,  1.30it/s, gpt_loss=0.0145, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 894/5616 [11:26<1:00:30,  1.30it/s, gpt_loss=0.00183, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 895/5616 [11:26<58:53,  1.34it/s, gpt_loss=0.00183, lr=4.81e-6]  
+
Train step of epoch 0:  16%|█▌        | 895/5616 [11:27<58:53,  1.34it/s, gpt_loss=0.103, lr=4.81e-6]  
+
Train step of epoch 0:  16%|█▌        | 896/5616 [11:27<59:22,  1.32it/s, gpt_loss=0.103, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 896/5616 [11:28<59:22,  1.32it/s, gpt_loss=0.0125, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 897/5616 [11:28<56:53,  1.38it/s, gpt_loss=0.0125, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 897/5616 [11:28<56:53,  1.38it/s, gpt_loss=0.0358, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 898/5616 [11:28<58:08,  1.35it/s, gpt_loss=0.0358, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 898/5616 [11:29<58:08,  1.35it/s, gpt_loss=0.00648, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 899/5616 [11:29<1:02:35,  1.26it/s, gpt_loss=0.00648, lr=4.81e-6]
+
Train step of epoch 0:  16%|█▌        | 899/5616 [11:30<1:02:35,  1.26it/s, gpt_loss=0.00579, lr=4.8e-6] 
+
Train step of epoch 0:  16%|█▌        | 900/5616 [11:30<1:01:09,  1.29it/s, gpt_loss=0.00579, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 900/5616 [11:31<1:01:09,  1.29it/s, gpt_loss=0.0238, lr=4.8e-6] 
+
Train step of epoch 0:  16%|█▌        | 901/5616 [11:31<58:57,  1.33it/s, gpt_loss=0.0238, lr=4.8e-6]  
+
Train step of epoch 0:  16%|█▌        | 901/5616 [11:31<58:57,  1.33it/s, gpt_loss=0.0239, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 902/5616 [11:31<57:10,  1.37it/s, gpt_loss=0.0239, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 902/5616 [11:32<57:10,  1.37it/s, gpt_loss=0.031, lr=4.8e-6] 
+
Train step of epoch 0:  16%|█▌        | 903/5616 [11:32<58:39,  1.34it/s, gpt_loss=0.031, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 903/5616 [11:33<58:39,  1.34it/s, gpt_loss=0.0123, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 904/5616 [11:33<59:50,  1.31it/s, gpt_loss=0.0123, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 904/5616 [11:34<59:50,  1.31it/s, gpt_loss=0.0162, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 905/5616 [11:34<1:07:56,  1.16it/s, gpt_loss=0.0162, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 905/5616 [11:35<1:07:56,  1.16it/s, gpt_loss=0.0567, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 906/5616 [11:35<1:02:48,  1.25it/s, gpt_loss=0.0567, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 906/5616 [11:35<1:02:48,  1.25it/s, gpt_loss=0.0152, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 907/5616 [11:35<1:00:01,  1.31it/s, gpt_loss=0.0152, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 907/5616 [11:36<1:00:01,  1.31it/s, gpt_loss=0.0299, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 908/5616 [11:36<1:00:22,  1.30it/s, gpt_loss=0.0299, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 908/5616 [11:37<1:00:22,  1.30it/s, gpt_loss=0.124, lr=4.8e-6] 
+
Train step of epoch 0:  16%|█▌        | 909/5616 [11:37<1:01:44,  1.27it/s, gpt_loss=0.124, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 909/5616 [11:38<1:01:44,  1.27it/s, gpt_loss=0.0883, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 910/5616 [11:38<1:00:19,  1.30it/s, gpt_loss=0.0883, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 910/5616 [11:38<1:00:19,  1.30it/s, gpt_loss=0.0182, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 911/5616 [11:38<58:56,  1.33it/s, gpt_loss=0.0182, lr=4.8e-6]  
+
Train step of epoch 0:  16%|█▌        | 911/5616 [11:39<58:56,  1.33it/s, gpt_loss=0.00377, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 912/5616 [11:39<59:22,  1.32it/s, gpt_loss=0.00377, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▌        | 912/5616 [11:40<59:22,  1.32it/s, gpt_loss=0.039, lr=4.8e-6]  
+
Train step of epoch 0:  16%|█▋        | 913/5616 [11:40<1:00:11,  1.30it/s, gpt_loss=0.039, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▋        | 913/5616 [11:41<1:00:11,  1.30it/s, gpt_loss=0.0671, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▋        | 914/5616 [11:41<58:19,  1.34it/s, gpt_loss=0.0671, lr=4.8e-6]  
+
Train step of epoch 0:  16%|█▋        | 914/5616 [11:41<58:19,  1.34it/s, gpt_loss=0.102, lr=4.8e-6] 
+
Train step of epoch 0:  16%|█▋        | 915/5616 [11:41<56:48,  1.38it/s, gpt_loss=0.102, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▋        | 915/5616 [11:42<56:48,  1.38it/s, gpt_loss=0.0543, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▋        | 916/5616 [11:42<1:00:46,  1.29it/s, gpt_loss=0.0543, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▋        | 916/5616 [11:43<1:00:46,  1.29it/s, gpt_loss=0.0378, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▋        | 917/5616 [11:43<58:11,  1.35it/s, gpt_loss=0.0378, lr=4.8e-6]  
+
Train step of epoch 0:  16%|█▋        | 917/5616 [11:44<58:11,  1.35it/s, gpt_loss=0.0282, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▋        | 918/5616 [11:44<56:39,  1.38it/s, gpt_loss=0.0282, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▋        | 918/5616 [11:44<56:39,  1.38it/s, gpt_loss=0.0308, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▋        | 919/5616 [11:44<59:06,  1.32it/s, gpt_loss=0.0308, lr=4.8e-6]
+
Train step of epoch 0:  16%|█▋        | 919/5616 [11:45<59:06,  1.32it/s, gpt_loss=0.00342, lr=4.79e-6]
+
Train step of epoch 0:  16%|█▋        | 920/5616 [11:45<1:01:25,  1.27it/s, gpt_loss=0.00342, lr=4.79e-6]
+
Train step of epoch 0:  16%|█▋        | 920/5616 [11:46<1:01:25,  1.27it/s, gpt_loss=0.0238, lr=4.79e-6] 
+
Train step of epoch 0:  16%|█▋        | 921/5616 [11:46<59:12,  1.32it/s, gpt_loss=0.0238, lr=4.79e-6]  
+
Train step of epoch 0:  16%|█▋        | 921/5616 [11:47<59:12,  1.32it/s, gpt_loss=0.0301, lr=4.79e-6]
+
Train step of epoch 0:  16%|█▋        | 922/5616 [11:47<58:38,  1.33it/s, gpt_loss=0.0301, lr=4.79e-6]
+
Train step of epoch 0:  16%|█▋        | 922/5616 [11:47<58:38,  1.33it/s, gpt_loss=0.0266, lr=4.79e-6]
+
Train step of epoch 0:  16%|█▋        | 923/5616 [11:47<58:26,  1.34it/s, gpt_loss=0.0266, lr=4.79e-6]
+
Train step of epoch 0:  16%|█▋        | 923/5616 [11:48<58:26,  1.34it/s, gpt_loss=0.0836, lr=4.79e-6]
+
Train step of epoch 0:  16%|█▋        | 924/5616 [11:48<58:30,  1.34it/s, gpt_loss=0.0836, lr=4.79e-6]
+
Train step of epoch 0:  16%|█▋        | 924/5616 [11:49<58:30,  1.34it/s, gpt_loss=0.0438, lr=4.79e-6]
+
Train step of epoch 0:  16%|█▋        | 925/5616 [11:49<59:07,  1.32it/s, gpt_loss=0.0438, lr=4.79e-6]
+
Train step of epoch 0:  16%|█▋        | 925/5616 [11:50<59:07,  1.32it/s, gpt_loss=0.00694, lr=4.79e-6]
+
Train step of epoch 0:  16%|█▋        | 926/5616 [11:50<58:04,  1.35it/s, gpt_loss=0.00694, lr=4.79e-6]
+
Train step of epoch 0:  16%|█▋        | 926/5616 [11:51<58:04,  1.35it/s, gpt_loss=0.0232, lr=4.79e-6] 
+
Train step of epoch 0:  17%|█▋        | 927/5616 [11:51<1:01:33,  1.27it/s, gpt_loss=0.0232, lr=4.79e-6]
+
Train step of epoch 0:  17%|█▋        | 927/5616 [11:51<1:01:33,  1.27it/s, gpt_loss=0.215, lr=4.79e-6] 
+
Train step of epoch 0:  17%|█▋        | 928/5616 [11:51<1:01:40,  1.27it/s, gpt_loss=0.215, lr=4.79e-6]
+
Train step of epoch 0:  17%|█▋        | 928/5616 [11:52<1:01:40,  1.27it/s, gpt_loss=0.0809, lr=4.79e-6]
+
Train step of epoch 0:  17%|█▋        | 929/5616 [11:52<1:01:03,  1.28it/s, gpt_loss=0.0809, lr=4.79e-6]
+
Train step of epoch 0:  17%|█▋        | 929/5616 [11:53<1:01:03,  1.28it/s, gpt_loss=0.0449, lr=4.79e-6]
+
Train step of epoch 0:  17%|█▋        | 930/5616 [11:53<58:21,  1.34it/s, gpt_loss=0.0449, lr=4.79e-6]  
+
Train step of epoch 0:  17%|█▋        | 930/5616 [11:54<58:21,  1.34it/s, gpt_loss=0.0294, lr=4.79e-6]
+
Train step of epoch 0:  17%|█▋        | 931/5616 [11:54<1:00:56,  1.28it/s, gpt_loss=0.0294, lr=4.79e-6]
+
Train step of epoch 0:  17%|█▋        | 931/5616 [11:54<1:00:56,  1.28it/s, gpt_loss=0.0633, lr=4.79e-6]
+
Train step of epoch 0:  17%|█▋        | 932/5616 [11:54<59:50,  1.30it/s, gpt_loss=0.0633, lr=4.79e-6]  
+
Train step of epoch 0:  17%|█▋        | 932/5616 [11:55<59:50,  1.30it/s, gpt_loss=0.0168, lr=4.79e-6]
+
Train step of epoch 0:  17%|█▋        | 933/5616 [11:55<58:48,  1.33it/s, gpt_loss=0.0168, lr=4.79e-6]
+
Train step of epoch 0:  17%|█▋        | 933/5616 [11:56<58:48,  1.33it/s, gpt_loss=0.21, lr=4.79e-6]  
+
Train step of epoch 0:  17%|█▋        | 934/5616 [11:56<1:01:18,  1.27it/s, gpt_loss=0.21, lr=4.79e-6]
+
Train step of epoch 0:  17%|█▋        | 934/5616 [11:57<1:01:18,  1.27it/s, gpt_loss=0.0446, lr=4.79e-6]
+
Train step of epoch 0:  17%|█▋        | 935/5616 [11:57<58:37,  1.33it/s, gpt_loss=0.0446, lr=4.79e-6]  
+
Train step of epoch 0:  17%|█▋        | 935/5616 [11:57<58:37,  1.33it/s, gpt_loss=0.0427, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 936/5616 [11:57<59:49,  1.30it/s, gpt_loss=0.0427, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 936/5616 [11:58<59:49,  1.30it/s, gpt_loss=0.0668, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 937/5616 [11:58<1:04:18,  1.21it/s, gpt_loss=0.0668, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 937/5616 [11:59<1:04:18,  1.21it/s, gpt_loss=0.109, lr=4.78e-6] 
+
Train step of epoch 0:  17%|█▋        | 938/5616 [11:59<1:02:18,  1.25it/s, gpt_loss=0.109, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 938/5616 [12:00<1:02:18,  1.25it/s, gpt_loss=0.00906, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 939/5616 [12:00<1:00:33,  1.29it/s, gpt_loss=0.00906, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 939/5616 [12:01<1:00:33,  1.29it/s, gpt_loss=0.0434, lr=4.78e-6] 
+
Train step of epoch 0:  17%|█▋        | 940/5616 [12:01<1:05:20,  1.19it/s, gpt_loss=0.0434, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 940/5616 [12:02<1:05:20,  1.19it/s, gpt_loss=0.106, lr=4.78e-6] 
+
Train step of epoch 0:  17%|█▋        | 941/5616 [12:02<1:02:46,  1.24it/s, gpt_loss=0.106, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 941/5616 [12:02<1:02:46,  1.24it/s, gpt_loss=0.103, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 942/5616 [12:02<1:03:15,  1.23it/s, gpt_loss=0.103, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 942/5616 [12:03<1:03:15,  1.23it/s, gpt_loss=0.0159, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 943/5616 [12:03<1:00:43,  1.28it/s, gpt_loss=0.0159, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 943/5616 [12:04<1:00:43,  1.28it/s, gpt_loss=0.0352, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 944/5616 [12:04<1:01:14,  1.27it/s, gpt_loss=0.0352, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 944/5616 [12:05<1:01:14,  1.27it/s, gpt_loss=0.101, lr=4.78e-6] 
+
Train step of epoch 0:  17%|█▋        | 945/5616 [12:05<1:00:35,  1.28it/s, gpt_loss=0.101, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 945/5616 [12:05<1:00:35,  1.28it/s, gpt_loss=0.0167, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 946/5616 [12:05<1:01:46,  1.26it/s, gpt_loss=0.0167, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 946/5616 [12:06<1:01:46,  1.26it/s, gpt_loss=0.0194, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 947/5616 [12:06<59:46,  1.30it/s, gpt_loss=0.0194, lr=4.78e-6]  
+
Train step of epoch 0:  17%|█▋        | 947/5616 [12:07<59:46,  1.30it/s, gpt_loss=0.0197, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 948/5616 [12:07<59:31,  1.31it/s, gpt_loss=0.0197, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 948/5616 [12:08<59:31,  1.31it/s, gpt_loss=0.0428, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 949/5616 [12:08<1:03:45,  1.22it/s, gpt_loss=0.0428, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 949/5616 [12:09<1:03:45,  1.22it/s, gpt_loss=0.103, lr=4.78e-6] 
+
Train step of epoch 0:  17%|█▋        | 950/5616 [12:09<1:01:24,  1.27it/s, gpt_loss=0.103, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 950/5616 [12:09<1:01:24,  1.27it/s, gpt_loss=0.047, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 951/5616 [12:09<58:31,  1.33it/s, gpt_loss=0.047, lr=4.78e-6]  
+
Train step of epoch 0:  17%|█▋        | 951/5616 [12:10<58:31,  1.33it/s, gpt_loss=0.062, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 952/5616 [12:10<58:36,  1.33it/s, gpt_loss=0.062, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 952/5616 [12:11<58:36,  1.33it/s, gpt_loss=0.071, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 953/5616 [12:11<57:02,  1.36it/s, gpt_loss=0.071, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 953/5616 [12:11<57:02,  1.36it/s, gpt_loss=0.064, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 954/5616 [12:11<55:31,  1.40it/s, gpt_loss=0.064, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 954/5616 [12:12<55:31,  1.40it/s, gpt_loss=0.0314, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 955/5616 [12:12<56:09,  1.38it/s, gpt_loss=0.0314, lr=4.78e-6]
+
Train step of epoch 0:  17%|█▋        | 955/5616 [12:13<56:09,  1.38it/s, gpt_loss=0.0206, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 956/5616 [12:13<57:53,  1.34it/s, gpt_loss=0.0206, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 956/5616 [12:14<57:53,  1.34it/s, gpt_loss=0.0524, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 957/5616 [12:14<59:41,  1.30it/s, gpt_loss=0.0524, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 957/5616 [12:15<59:41,  1.30it/s, gpt_loss=0.0455, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 958/5616 [12:15<1:00:40,  1.28it/s, gpt_loss=0.0455, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 958/5616 [12:15<1:00:40,  1.28it/s, gpt_loss=0.0347, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 959/5616 [12:15<59:04,  1.31it/s, gpt_loss=0.0347, lr=4.77e-6]  
+
Train step of epoch 0:  17%|█▋        | 959/5616 [12:16<59:04,  1.31it/s, gpt_loss=0.0426, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 960/5616 [12:16<1:06:36,  1.16it/s, gpt_loss=0.0426, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 960/5616 [12:17<1:06:36,  1.16it/s, gpt_loss=0.0479, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 961/5616 [12:17<1:04:50,  1.20it/s, gpt_loss=0.0479, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 961/5616 [12:18<1:04:50,  1.20it/s, gpt_loss=0.142, lr=4.77e-6] 
+
Train step of epoch 0:  17%|█▋        | 962/5616 [12:18<1:05:49,  1.18it/s, gpt_loss=0.142, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 962/5616 [12:19<1:05:49,  1.18it/s, gpt_loss=0.018, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 963/5616 [12:19<1:03:30,  1.22it/s, gpt_loss=0.018, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 963/5616 [12:20<1:03:30,  1.22it/s, gpt_loss=0.0465, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 964/5616 [12:20<1:04:01,  1.21it/s, gpt_loss=0.0465, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 964/5616 [12:20<1:04:01,  1.21it/s, gpt_loss=0.0393, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 965/5616 [12:20<1:00:22,  1.28it/s, gpt_loss=0.0393, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 965/5616 [12:21<1:00:22,  1.28it/s, gpt_loss=0.0592, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 966/5616 [12:21<58:23,  1.33it/s, gpt_loss=0.0592, lr=4.77e-6]  
+
Train step of epoch 0:  17%|█▋        | 966/5616 [12:22<58:23,  1.33it/s, gpt_loss=0.0475, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 967/5616 [12:22<56:44,  1.37it/s, gpt_loss=0.0475, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 967/5616 [12:23<56:44,  1.37it/s, gpt_loss=0.0327, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 968/5616 [12:23<1:00:36,  1.28it/s, gpt_loss=0.0327, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 968/5616 [12:23<1:00:36,  1.28it/s, gpt_loss=0.034, lr=4.77e-6] 
+
Train step of epoch 0:  17%|█▋        | 969/5616 [12:23<59:11,  1.31it/s, gpt_loss=0.034, lr=4.77e-6]  
+
Train step of epoch 0:  17%|█▋        | 969/5616 [12:24<59:11,  1.31it/s, gpt_loss=0.00566, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 970/5616 [12:24<57:21,  1.35it/s, gpt_loss=0.00566, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 970/5616 [12:25<57:21,  1.35it/s, gpt_loss=0.0182, lr=4.77e-6] 
+
Train step of epoch 0:  17%|█▋        | 971/5616 [12:25<56:17,  1.38it/s, gpt_loss=0.0182, lr=4.77e-6]
+
Train step of epoch 0:  17%|█▋        | 971/5616 [12:25<56:17,  1.38it/s, gpt_loss=0.0754, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 972/5616 [12:25<56:41,  1.37it/s, gpt_loss=0.0754, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 972/5616 [12:26<56:41,  1.37it/s, gpt_loss=0.0651, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 973/5616 [12:26<55:17,  1.40it/s, gpt_loss=0.0651, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 973/5616 [12:27<55:17,  1.40it/s, gpt_loss=0.05, lr=4.76e-6]  
+
Train step of epoch 0:  17%|█▋        | 974/5616 [12:27<56:13,  1.38it/s, gpt_loss=0.05, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 974/5616 [12:28<56:13,  1.38it/s, gpt_loss=0.0586, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 975/5616 [12:28<1:00:15,  1.28it/s, gpt_loss=0.0586, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 975/5616 [12:28<1:00:15,  1.28it/s, gpt_loss=0.0746, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 976/5616 [12:28<58:58,  1.31it/s, gpt_loss=0.0746, lr=4.76e-6]  
+
Train step of epoch 0:  17%|█▋        | 976/5616 [12:29<58:58,  1.31it/s, gpt_loss=0.0336, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 977/5616 [12:29<59:07,  1.31it/s, gpt_loss=0.0336, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 977/5616 [12:30<59:07,  1.31it/s, gpt_loss=0.11, lr=4.76e-6]  
+
Train step of epoch 0:  17%|█▋        | 978/5616 [12:30<56:28,  1.37it/s, gpt_loss=0.11, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 978/5616 [12:31<56:28,  1.37it/s, gpt_loss=0.0161, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 979/5616 [12:31<56:02,  1.38it/s, gpt_loss=0.0161, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 979/5616 [12:31<56:02,  1.38it/s, gpt_loss=0.0385, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 980/5616 [12:31<56:52,  1.36it/s, gpt_loss=0.0385, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 980/5616 [12:32<56:52,  1.36it/s, gpt_loss=0.0205, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 981/5616 [12:32<54:21,  1.42it/s, gpt_loss=0.0205, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 981/5616 [12:33<54:21,  1.42it/s, gpt_loss=0.0752, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 982/5616 [12:33<56:33,  1.37it/s, gpt_loss=0.0752, lr=4.76e-6]
+
Train step of epoch 0:  17%|█▋        | 982/5616 [12:34<56:33,  1.37it/s, gpt_loss=0.0372, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 983/5616 [12:34<59:51,  1.29it/s, gpt_loss=0.0372, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 983/5616 [12:34<59:51,  1.29it/s, gpt_loss=0.041, lr=4.76e-6] 
+
Train step of epoch 0:  18%|█▊        | 984/5616 [12:34<1:01:02,  1.26it/s, gpt_loss=0.041, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 984/5616 [12:35<1:01:02,  1.26it/s, gpt_loss=0.0592, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 985/5616 [12:35<59:57,  1.29it/s, gpt_loss=0.0592, lr=4.76e-6]  
+
Train step of epoch 0:  18%|█▊        | 985/5616 [12:36<59:57,  1.29it/s, gpt_loss=0.0145, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 986/5616 [12:36<1:02:51,  1.23it/s, gpt_loss=0.0145, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 986/5616 [12:37<1:02:51,  1.23it/s, gpt_loss=0.112, lr=4.76e-6] 
+
Train step of epoch 0:  18%|█▊        | 987/5616 [12:37<1:00:41,  1.27it/s, gpt_loss=0.112, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 987/5616 [12:38<1:00:41,  1.27it/s, gpt_loss=0.0774, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 988/5616 [12:38<1:00:24,  1.28it/s, gpt_loss=0.0774, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 988/5616 [12:38<1:00:24,  1.28it/s, gpt_loss=0.0281, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 989/5616 [12:38<58:15,  1.32it/s, gpt_loss=0.0281, lr=4.76e-6]  
+
Train step of epoch 0:  18%|█▊        | 989/5616 [12:39<58:15,  1.32it/s, gpt_loss=0.0115, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 990/5616 [12:39<59:57,  1.29it/s, gpt_loss=0.0115, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 990/5616 [12:40<59:57,  1.29it/s, gpt_loss=0.0163, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 991/5616 [12:40<57:47,  1.33it/s, gpt_loss=0.0163, lr=4.76e-6]
+
Train step of epoch 0:  18%|█▊        | 991/5616 [12:41<57:47,  1.33it/s, gpt_loss=0.0609, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 992/5616 [12:41<1:00:01,  1.28it/s, gpt_loss=0.0609, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 992/5616 [12:42<1:00:01,  1.28it/s, gpt_loss=0.0641, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 993/5616 [12:42<1:07:18,  1.14it/s, gpt_loss=0.0641, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 993/5616 [12:43<1:07:18,  1.14it/s, gpt_loss=0.146, lr=4.75e-6] 
+
Train step of epoch 0:  18%|█▊        | 994/5616 [12:43<1:04:58,  1.19it/s, gpt_loss=0.146, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 994/5616 [12:43<1:04:58,  1.19it/s, gpt_loss=0.00587, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 995/5616 [12:43<1:00:27,  1.27it/s, gpt_loss=0.00587, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 995/5616 [12:44<1:00:27,  1.27it/s, gpt_loss=0.108, lr=4.75e-6]  
+
Train step of epoch 0:  18%|█▊        | 996/5616 [12:44<59:18,  1.30it/s, gpt_loss=0.108, lr=4.75e-6]  
+
Train step of epoch 0:  18%|█▊        | 996/5616 [12:45<59:18,  1.30it/s, gpt_loss=0.0187, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 997/5616 [12:45<1:02:24,  1.23it/s, gpt_loss=0.0187, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 997/5616 [12:46<1:02:24,  1.23it/s, gpt_loss=0.0566, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 998/5616 [12:46<1:07:24,  1.14it/s, gpt_loss=0.0566, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 998/5616 [12:47<1:07:24,  1.14it/s, gpt_loss=0.0661, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 999/5616 [12:47<1:03:57,  1.20it/s, gpt_loss=0.0661, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 999/5616 [12:47<1:03:57,  1.20it/s, gpt_loss=0.0827, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 1000/5616 [12:47<1:02:27,  1.23it/s, gpt_loss=0.0827, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 1000/5616 [12:48<1:02:27,  1.23it/s, gpt_loss=0.168, lr=4.75e-6] 
+
Train step of epoch 0:  18%|█▊        | 1001/5616 [12:48<59:04,  1.30it/s, gpt_loss=0.168, lr=4.75e-6]  
+
Train step of epoch 0:  18%|█▊        | 1001/5616 [12:49<59:04,  1.30it/s, gpt_loss=0.0114, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 1002/5616 [12:49<59:31,  1.29it/s, gpt_loss=0.0114, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 1002/5616 [12:50<59:31,  1.29it/s, gpt_loss=0.0477, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 1003/5616 [12:50<59:06,  1.30it/s, gpt_loss=0.0477, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 1003/5616 [12:50<59:06,  1.30it/s, gpt_loss=0.0604, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 1004/5616 [12:50<1:02:45,  1.22it/s, gpt_loss=0.0604, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 1004/5616 [12:51<1:02:45,  1.22it/s, gpt_loss=0.0794, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 1005/5616 [12:51<59:32,  1.29it/s, gpt_loss=0.0794, lr=4.75e-6]  
+
Train step of epoch 0:  18%|█▊        | 1005/5616 [12:52<59:32,  1.29it/s, gpt_loss=0.0262, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 1006/5616 [12:52<1:01:11,  1.26it/s, gpt_loss=0.0262, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 1006/5616 [12:53<1:01:11,  1.26it/s, gpt_loss=0.0197, lr=4.75e-6]
+
Train step of epoch 0:  18%|█▊        | 1007/5616 [12:53<59:00,  1.30it/s, gpt_loss=0.0197, lr=4.75e-6]  
+
Train step of epoch 0:  18%|█▊        | 1007/5616 [12:53<59:00,  1.30it/s, gpt_loss=0.0725, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1008/5616 [12:53<59:11,  1.30it/s, gpt_loss=0.0725, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1008/5616 [12:54<59:11,  1.30it/s, gpt_loss=0.0479, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1009/5616 [12:54<57:19,  1.34it/s, gpt_loss=0.0479, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1009/5616 [12:55<57:19,  1.34it/s, gpt_loss=0.092, lr=4.74e-6] 
+
Train step of epoch 0:  18%|█▊        | 1010/5616 [12:55<55:51,  1.37it/s, gpt_loss=0.092, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1010/5616 [12:56<55:51,  1.37it/s, gpt_loss=0.00846, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1011/5616 [12:56<54:00,  1.42it/s, gpt_loss=0.00846, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1011/5616 [12:56<54:00,  1.42it/s, gpt_loss=0.0324, lr=4.74e-6] 
+
Train step of epoch 0:  18%|█▊        | 1012/5616 [12:56<56:29,  1.36it/s, gpt_loss=0.0324, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1012/5616 [12:57<56:29,  1.36it/s, gpt_loss=0.0142, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1013/5616 [12:57<56:33,  1.36it/s, gpt_loss=0.0142, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1013/5616 [12:58<56:33,  1.36it/s, gpt_loss=0.0321, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1014/5616 [12:58<54:16,  1.41it/s, gpt_loss=0.0321, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1014/5616 [12:58<54:16,  1.41it/s, gpt_loss=0.0241, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1015/5616 [12:58<53:43,  1.43it/s, gpt_loss=0.0241, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1015/5616 [12:59<53:43,  1.43it/s, gpt_loss=0.0264, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1016/5616 [12:59<54:00,  1.42it/s, gpt_loss=0.0264, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1016/5616 [13:00<54:00,  1.42it/s, gpt_loss=0.0314, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1017/5616 [13:00<54:12,  1.41it/s, gpt_loss=0.0314, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1017/5616 [13:01<54:12,  1.41it/s, gpt_loss=0.011, lr=4.74e-6] 
+
Train step of epoch 0:  18%|█▊        | 1018/5616 [13:01<56:06,  1.37it/s, gpt_loss=0.011, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1018/5616 [13:01<56:06,  1.37it/s, gpt_loss=0.051, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1019/5616 [13:01<58:17,  1.31it/s, gpt_loss=0.051, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1019/5616 [13:02<58:17,  1.31it/s, gpt_loss=0.0179, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1020/5616 [13:02<58:15,  1.31it/s, gpt_loss=0.0179, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1020/5616 [13:03<58:15,  1.31it/s, gpt_loss=0.02, lr=4.74e-6]  
+
Train step of epoch 0:  18%|█▊        | 1021/5616 [13:03<56:14,  1.36it/s, gpt_loss=0.02, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1021/5616 [13:04<56:14,  1.36it/s, gpt_loss=0.0623, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1022/5616 [13:04<56:13,  1.36it/s, gpt_loss=0.0623, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1022/5616 [13:04<56:13,  1.36it/s, gpt_loss=0.0325, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1023/5616 [13:04<58:53,  1.30it/s, gpt_loss=0.0325, lr=4.74e-6]
+
Train step of epoch 0:  18%|█▊        | 1023/5616 [13:05<58:53,  1.30it/s, gpt_loss=0.0518, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1024/5616 [13:05<58:28,  1.31it/s, gpt_loss=0.0518, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1024/5616 [13:06<58:28,  1.31it/s, gpt_loss=0.024, lr=4.73e-6] 
+
Train step of epoch 0:  18%|█▊        | 1025/5616 [13:06<58:38,  1.30it/s, gpt_loss=0.024, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1025/5616 [13:07<58:38,  1.30it/s, gpt_loss=0.0266, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1026/5616 [13:07<56:27,  1.36it/s, gpt_loss=0.0266, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1026/5616 [13:07<56:27,  1.36it/s, gpt_loss=0.031, lr=4.73e-6] 
+
Train step of epoch 0:  18%|█▊        | 1027/5616 [13:07<56:44,  1.35it/s, gpt_loss=0.031, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1027/5616 [13:09<56:44,  1.35it/s, gpt_loss=0.0814, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1028/5616 [13:09<1:06:56,  1.14it/s, gpt_loss=0.0814, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1028/5616 [13:09<1:06:56,  1.14it/s, gpt_loss=0.0289, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1029/5616 [13:09<1:05:20,  1.17it/s, gpt_loss=0.0289, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1029/5616 [13:10<1:05:20,  1.17it/s, gpt_loss=0.0399, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1030/5616 [13:10<1:01:43,  1.24it/s, gpt_loss=0.0399, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1030/5616 [13:11<1:01:43,  1.24it/s, gpt_loss=0.0932, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1031/5616 [13:11<58:25,  1.31it/s, gpt_loss=0.0932, lr=4.73e-6]  
+
Train step of epoch 0:  18%|█▊        | 1031/5616 [13:11<58:25,  1.31it/s, gpt_loss=0.0046, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1032/5616 [13:11<57:49,  1.32it/s, gpt_loss=0.0046, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1032/5616 [13:12<57:49,  1.32it/s, gpt_loss=0.0482, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1033/5616 [13:12<55:35,  1.37it/s, gpt_loss=0.0482, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1033/5616 [13:13<55:35,  1.37it/s, gpt_loss=0.039, lr=4.73e-6] 
+
Train step of epoch 0:  18%|█▊        | 1034/5616 [13:13<56:00,  1.36it/s, gpt_loss=0.039, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1034/5616 [13:14<56:00,  1.36it/s, gpt_loss=0.0588, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1035/5616 [13:14<55:54,  1.37it/s, gpt_loss=0.0588, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1035/5616 [13:14<55:54,  1.37it/s, gpt_loss=0.0136, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1036/5616 [13:14<59:01,  1.29it/s, gpt_loss=0.0136, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1036/5616 [13:15<59:01,  1.29it/s, gpt_loss=0.0364, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1037/5616 [13:15<56:45,  1.34it/s, gpt_loss=0.0364, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1037/5616 [13:16<56:45,  1.34it/s, gpt_loss=0.00271, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1038/5616 [13:16<55:04,  1.39it/s, gpt_loss=0.00271, lr=4.73e-6]
+
Train step of epoch 0:  18%|█▊        | 1038/5616 [13:16<55:04,  1.39it/s, gpt_loss=0.022, lr=4.73e-6]  
+
Train step of epoch 0:  19%|█▊        | 1039/5616 [13:16<53:10,  1.43it/s, gpt_loss=0.022, lr=4.73e-6]
+
Train step of epoch 0:  19%|█▊        | 1039/5616 [13:17<53:10,  1.43it/s, gpt_loss=0.0532, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1040/5616 [13:17<55:58,  1.36it/s, gpt_loss=0.0532, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1040/5616 [13:18<55:58,  1.36it/s, gpt_loss=0.095, lr=4.72e-6] 
+
Train step of epoch 0:  19%|█▊        | 1041/5616 [13:18<54:20,  1.40it/s, gpt_loss=0.095, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1041/5616 [13:19<54:20,  1.40it/s, gpt_loss=0.0146, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1042/5616 [13:19<52:22,  1.46it/s, gpt_loss=0.0146, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1042/5616 [13:19<52:22,  1.46it/s, gpt_loss=0.0694, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1043/5616 [13:19<53:05,  1.44it/s, gpt_loss=0.0694, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1043/5616 [13:20<53:05,  1.44it/s, gpt_loss=0.0715, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1044/5616 [13:20<56:17,  1.35it/s, gpt_loss=0.0715, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1044/5616 [13:21<56:17,  1.35it/s, gpt_loss=0.0673, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1045/5616 [13:21<54:29,  1.40it/s, gpt_loss=0.0673, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1045/5616 [13:21<54:29,  1.40it/s, gpt_loss=0.0324, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1046/5616 [13:21<53:25,  1.43it/s, gpt_loss=0.0324, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1046/5616 [13:22<53:25,  1.43it/s, gpt_loss=0.00935, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1047/5616 [13:22<53:28,  1.42it/s, gpt_loss=0.00935, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1047/5616 [13:23<53:28,  1.42it/s, gpt_loss=0.0248, lr=4.72e-6] 
+
Train step of epoch 0:  19%|█▊        | 1048/5616 [13:23<57:37,  1.32it/s, gpt_loss=0.0248, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1048/5616 [13:24<57:37,  1.32it/s, gpt_loss=0.0653, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1049/5616 [13:24<1:03:56,  1.19it/s, gpt_loss=0.0653, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1049/5616 [13:25<1:03:56,  1.19it/s, gpt_loss=0.0324, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1050/5616 [13:25<1:03:57,  1.19it/s, gpt_loss=0.0324, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1050/5616 [13:26<1:03:57,  1.19it/s, gpt_loss=0.0352, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1051/5616 [13:26<59:39,  1.28it/s, gpt_loss=0.0352, lr=4.72e-6]  
+
Train step of epoch 0:  19%|█▊        | 1051/5616 [13:26<59:39,  1.28it/s, gpt_loss=0.0244, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1052/5616 [13:26<1:00:48,  1.25it/s, gpt_loss=0.0244, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▊        | 1052/5616 [13:27<1:00:48,  1.25it/s, gpt_loss=0.0258, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▉        | 1053/5616 [13:27<1:02:23,  1.22it/s, gpt_loss=0.0258, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▉        | 1053/5616 [13:28<1:02:23,  1.22it/s, gpt_loss=0.0193, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▉        | 1054/5616 [13:28<59:24,  1.28it/s, gpt_loss=0.0193, lr=4.72e-6]  
+
Train step of epoch 0:  19%|█▉        | 1054/5616 [13:29<59:24,  1.28it/s, gpt_loss=0.011, lr=4.72e-6] 
+
Train step of epoch 0:  19%|█▉        | 1055/5616 [13:29<56:16,  1.35it/s, gpt_loss=0.011, lr=4.72e-6]
+
Train step of epoch 0:  19%|█▉        | 1055/5616 [13:29<56:16,  1.35it/s, gpt_loss=0.0137, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1056/5616 [13:29<57:10,  1.33it/s, gpt_loss=0.0137, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1056/5616 [13:30<57:10,  1.33it/s, gpt_loss=0.0375, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1057/5616 [13:30<56:39,  1.34it/s, gpt_loss=0.0375, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1057/5616 [13:31<56:39,  1.34it/s, gpt_loss=0.0444, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1058/5616 [13:31<56:37,  1.34it/s, gpt_loss=0.0444, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1058/5616 [13:32<56:37,  1.34it/s, gpt_loss=0.0486, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1059/5616 [13:32<54:16,  1.40it/s, gpt_loss=0.0486, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1059/5616 [13:32<54:16,  1.40it/s, gpt_loss=0.0576, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1060/5616 [13:32<55:45,  1.36it/s, gpt_loss=0.0576, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1060/5616 [13:33<55:45,  1.36it/s, gpt_loss=0.072, lr=4.71e-6] 
+
Train step of epoch 0:  19%|█▉        | 1061/5616 [13:33<53:46,  1.41it/s, gpt_loss=0.072, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1061/5616 [13:34<53:46,  1.41it/s, gpt_loss=0.0244, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1062/5616 [13:34<54:20,  1.40it/s, gpt_loss=0.0244, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1062/5616 [13:34<54:20,  1.40it/s, gpt_loss=0.107, lr=4.71e-6] 
+
Train step of epoch 0:  19%|█▉        | 1063/5616 [13:34<55:39,  1.36it/s, gpt_loss=0.107, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1063/5616 [13:35<55:39,  1.36it/s, gpt_loss=0.0159, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1064/5616 [13:35<56:31,  1.34it/s, gpt_loss=0.0159, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1064/5616 [13:36<56:31,  1.34it/s, gpt_loss=0.064, lr=4.71e-6] 
+
Train step of epoch 0:  19%|█▉        | 1065/5616 [13:36<56:25,  1.34it/s, gpt_loss=0.064, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1065/5616 [13:37<56:25,  1.34it/s, gpt_loss=0.0337, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1066/5616 [13:37<55:19,  1.37it/s, gpt_loss=0.0337, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1066/5616 [13:37<55:19,  1.37it/s, gpt_loss=0.019, lr=4.71e-6] 
+
Train step of epoch 0:  19%|█▉        | 1067/5616 [13:37<53:31,  1.42it/s, gpt_loss=0.019, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1067/5616 [13:38<53:31,  1.42it/s, gpt_loss=0.0122, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1068/5616 [13:38<56:08,  1.35it/s, gpt_loss=0.0122, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1068/5616 [13:39<56:08,  1.35it/s, gpt_loss=0.041, lr=4.71e-6] 
+
Train step of epoch 0:  19%|█▉        | 1069/5616 [13:39<55:45,  1.36it/s, gpt_loss=0.041, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1069/5616 [13:40<55:45,  1.36it/s, gpt_loss=0.111, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1070/5616 [13:40<56:01,  1.35it/s, gpt_loss=0.111, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1070/5616 [13:40<56:01,  1.35it/s, gpt_loss=0.121, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1071/5616 [13:40<54:23,  1.39it/s, gpt_loss=0.121, lr=4.71e-6]
+
Train step of epoch 0:  19%|█▉        | 1071/5616 [13:41<54:23,  1.39it/s, gpt_loss=0.276, lr=4.7e-6] 
+
Train step of epoch 0:  19%|█▉        | 1072/5616 [13:41<57:49,  1.31it/s, gpt_loss=0.276, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1072/5616 [13:42<57:49,  1.31it/s, gpt_loss=0.00878, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1073/5616 [13:42<57:59,  1.31it/s, gpt_loss=0.00878, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1073/5616 [13:43<57:59,  1.31it/s, gpt_loss=0.0248, lr=4.7e-6] 
+
Train step of epoch 0:  19%|█▉        | 1074/5616 [13:43<55:02,  1.38it/s, gpt_loss=0.0248, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1074/5616 [13:43<55:02,  1.38it/s, gpt_loss=0.0744, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1075/5616 [13:43<54:57,  1.38it/s, gpt_loss=0.0744, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1075/5616 [13:44<54:57,  1.38it/s, gpt_loss=0.0336, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1076/5616 [13:44<56:38,  1.34it/s, gpt_loss=0.0336, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1076/5616 [13:45<56:38,  1.34it/s, gpt_loss=0.0751, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1077/5616 [13:45<56:00,  1.35it/s, gpt_loss=0.0751, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1077/5616 [13:46<56:00,  1.35it/s, gpt_loss=0.0426, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1078/5616 [13:46<56:10,  1.35it/s, gpt_loss=0.0426, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1078/5616 [13:46<56:10,  1.35it/s, gpt_loss=0.0323, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1079/5616 [13:46<54:56,  1.38it/s, gpt_loss=0.0323, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1079/5616 [13:47<54:56,  1.38it/s, gpt_loss=0.0268, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1080/5616 [13:47<56:56,  1.33it/s, gpt_loss=0.0268, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1080/5616 [13:48<56:56,  1.33it/s, gpt_loss=0.0173, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1081/5616 [13:48<55:40,  1.36it/s, gpt_loss=0.0173, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1081/5616 [13:49<55:40,  1.36it/s, gpt_loss=0.1, lr=4.7e-6]   
+
Train step of epoch 0:  19%|█▉        | 1082/5616 [13:49<58:10,  1.30it/s, gpt_loss=0.1, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1082/5616 [13:49<58:10,  1.30it/s, gpt_loss=0.0134, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1083/5616 [13:49<56:30,  1.34it/s, gpt_loss=0.0134, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1083/5616 [13:50<56:30,  1.34it/s, gpt_loss=0.00103, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1084/5616 [13:50<57:05,  1.32it/s, gpt_loss=0.00103, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1084/5616 [13:51<57:05,  1.32it/s, gpt_loss=0.034, lr=4.7e-6]  
+
Train step of epoch 0:  19%|█▉        | 1085/5616 [13:51<58:08,  1.30it/s, gpt_loss=0.034, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1085/5616 [13:52<58:08,  1.30it/s, gpt_loss=0.0521, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1086/5616 [13:52<56:21,  1.34it/s, gpt_loss=0.0521, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1086/5616 [13:52<56:21,  1.34it/s, gpt_loss=0.0223, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1087/5616 [13:52<55:44,  1.35it/s, gpt_loss=0.0223, lr=4.7e-6]
+
Train step of epoch 0:  19%|█▉        | 1087/5616 [13:53<55:44,  1.35it/s, gpt_loss=0.0175, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1088/5616 [13:53<56:27,  1.34it/s, gpt_loss=0.0175, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1088/5616 [13:54<56:27,  1.34it/s, gpt_loss=0.0197, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1089/5616 [13:54<54:47,  1.38it/s, gpt_loss=0.0197, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1089/5616 [13:54<54:47,  1.38it/s, gpt_loss=0.0723, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1090/5616 [13:54<53:38,  1.41it/s, gpt_loss=0.0723, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1090/5616 [13:55<53:38,  1.41it/s, gpt_loss=0.114, lr=4.69e-6] 
+
Train step of epoch 0:  19%|█▉        | 1091/5616 [13:55<53:02,  1.42it/s, gpt_loss=0.114, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1091/5616 [13:56<53:02,  1.42it/s, gpt_loss=0.0906, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1092/5616 [13:56<56:40,  1.33it/s, gpt_loss=0.0906, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1092/5616 [13:57<56:40,  1.33it/s, gpt_loss=0.0503, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1093/5616 [13:57<54:17,  1.39it/s, gpt_loss=0.0503, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1093/5616 [13:57<54:17,  1.39it/s, gpt_loss=0.00332, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1094/5616 [13:57<52:39,  1.43it/s, gpt_loss=0.00332, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1094/5616 [13:58<52:39,  1.43it/s, gpt_loss=0.0075, lr=4.69e-6] 
+
Train step of epoch 0:  19%|█▉        | 1095/5616 [13:58<51:42,  1.46it/s, gpt_loss=0.0075, lr=4.69e-6]
+
Train step of epoch 0:  19%|█▉        | 1095/5616 [13:59<51:42,  1.46it/s, gpt_loss=0.0089, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1096/5616 [13:59<55:33,  1.36it/s, gpt_loss=0.0089, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1096/5616 [14:00<55:33,  1.36it/s, gpt_loss=0.0327, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1097/5616 [14:00<57:36,  1.31it/s, gpt_loss=0.0327, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1097/5616 [14:00<57:36,  1.31it/s, gpt_loss=0.128, lr=4.69e-6] 
+
Train step of epoch 0:  20%|█▉        | 1098/5616 [14:00<57:08,  1.32it/s, gpt_loss=0.128, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1098/5616 [14:01<57:08,  1.32it/s, gpt_loss=0.0524, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1099/5616 [14:01<56:55,  1.32it/s, gpt_loss=0.0524, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1099/5616 [14:02<56:55,  1.32it/s, gpt_loss=0.0395, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1100/5616 [14:02<56:40,  1.33it/s, gpt_loss=0.0395, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1100/5616 [14:03<56:40,  1.33it/s, gpt_loss=0.0511, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1101/5616 [14:03<55:32,  1.35it/s, gpt_loss=0.0511, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1101/5616 [14:03<55:32,  1.35it/s, gpt_loss=0.0212, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1102/5616 [14:03<55:26,  1.36it/s, gpt_loss=0.0212, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1102/5616 [14:04<55:26,  1.36it/s, gpt_loss=0.00734, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1103/5616 [14:04<58:52,  1.28it/s, gpt_loss=0.00734, lr=4.69e-6]
+
Train step of epoch 0:  20%|█▉        | 1103/5616 [14:05<58:52,  1.28it/s, gpt_loss=0.0106, lr=4.68e-6] 
+
Train step of epoch 0:  20%|█▉        | 1104/5616 [14:05<59:50,  1.26it/s, gpt_loss=0.0106, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1104/5616 [14:06<59:50,  1.26it/s, gpt_loss=0.0387, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1105/5616 [14:06<59:43,  1.26it/s, gpt_loss=0.0387, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1105/5616 [14:07<59:43,  1.26it/s, gpt_loss=0.0212, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1106/5616 [14:07<1:04:18,  1.17it/s, gpt_loss=0.0212, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1106/5616 [14:07<1:04:18,  1.17it/s, gpt_loss=0.0271, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1107/5616 [14:07<1:00:28,  1.24it/s, gpt_loss=0.0271, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1107/5616 [14:08<1:00:28,  1.24it/s, gpt_loss=0.0205, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1108/5616 [14:08<58:28,  1.28it/s, gpt_loss=0.0205, lr=4.68e-6]  
+
Train step of epoch 0:  20%|█▉        | 1108/5616 [14:09<58:28,  1.28it/s, gpt_loss=0.0207, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1109/5616 [14:09<54:54,  1.37it/s, gpt_loss=0.0207, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1109/5616 [14:09<54:54,  1.37it/s, gpt_loss=0.0254, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1110/5616 [14:09<53:17,  1.41it/s, gpt_loss=0.0254, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1110/5616 [14:10<53:17,  1.41it/s, gpt_loss=0.0271, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1111/5616 [14:10<59:17,  1.27it/s, gpt_loss=0.0271, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1111/5616 [14:11<59:17,  1.27it/s, gpt_loss=0.0486, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1112/5616 [14:11<1:01:46,  1.22it/s, gpt_loss=0.0486, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1112/5616 [14:12<1:01:46,  1.22it/s, gpt_loss=0.0208, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1113/5616 [14:12<1:00:33,  1.24it/s, gpt_loss=0.0208, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1113/5616 [14:13<1:00:33,  1.24it/s, gpt_loss=0.0373, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1114/5616 [14:13<57:55,  1.30it/s, gpt_loss=0.0373, lr=4.68e-6]  
+
Train step of epoch 0:  20%|█▉        | 1114/5616 [14:14<57:55,  1.30it/s, gpt_loss=0.0173, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1115/5616 [14:14<58:09,  1.29it/s, gpt_loss=0.0173, lr=4.68e-6]
+
Train step of epoch 0:  20%|█▉        | 1115/5616 [14:14<58:09,  1.29it/s, gpt_loss=0.053, lr=4.67e-6] 
+
Train step of epoch 0:  20%|█▉        | 1116/5616 [14:14<1:00:55,  1.23it/s, gpt_loss=0.053, lr=4.67e-6]
+
Train step of epoch 0:  20%|█▉        | 1116/5616 [14:15<1:00:55,  1.23it/s, gpt_loss=0.169, lr=4.67e-6]
+
Train step of epoch 0:  20%|█▉        | 1117/5616 [14:15<59:25,  1.26it/s, gpt_loss=0.169, lr=4.67e-6]  
+
Train step of epoch 0:  20%|█▉        | 1117/5616 [14:16<59:25,  1.26it/s, gpt_loss=0.0507, lr=4.67e-6]
+
Train step of epoch 0:  20%|█▉        | 1118/5616 [14:16<59:51,  1.25it/s, gpt_loss=0.0507, lr=4.67e-6]
+
Train step of epoch 0:  20%|█▉        | 1118/5616 [14:17<59:51,  1.25it/s, gpt_loss=0.0212, lr=4.67e-6]
+
Train step of epoch 0:  20%|█▉        | 1119/5616 [14:17<58:04,  1.29it/s, gpt_loss=0.0212, lr=4.67e-6]
+
Train step of epoch 0:  20%|█▉        | 1119/5616 [14:18<58:04,  1.29it/s, gpt_loss=0.0256, lr=4.67e-6]
+
Train step of epoch 0:  20%|█▉        | 1120/5616 [14:18<1:00:22,  1.24it/s, gpt_loss=0.0256, lr=4.67e-6]
+
Train step of epoch 0:  20%|█▉        | 1120/5616 [14:18<1:00:22,  1.24it/s, gpt_loss=0.0507, lr=4.67e-6]
+
Train step of epoch 0:  20%|█▉        | 1121/5616 [14:18<57:46,  1.30it/s, gpt_loss=0.0507, lr=4.67e-6]  
+
Train step of epoch 0:  20%|█▉        | 1121/5616 [14:19<57:46,  1.30it/s, gpt_loss=0.141, lr=4.67e-6] 
+
Train step of epoch 0:  20%|█▉        | 1122/5616 [14:19<56:54,  1.32it/s, gpt_loss=0.141, lr=4.67e-6]
+
Train step of epoch 0:  20%|█▉        | 1122/5616 [14:20<56:54,  1.32it/s, gpt_loss=0.0851, lr=4.67e-6]
+
Train step of epoch 0:  20%|█▉        | 1123/5616 [14:20<55:59,  1.34it/s, gpt_loss=0.0851, lr=4.67e-6]
+
Train step of epoch 0:  20%|█▉        | 1123/5616 [14:21<55:59,  1.34it/s, gpt_loss=0.0902, lr=4.67e-6]
+
Train step of epoch 0:  20%|██        | 1124/5616 [14:21<1:01:46,  1.21it/s, gpt_loss=0.0902, lr=4.67e-6]
+
Train step of epoch 0:  20%|██        | 1124/5616 [14:22<1:01:46,  1.21it/s, gpt_loss=0.109, lr=4.67e-6] 
+
Train step of epoch 0:  20%|██        | 1125/5616 [14:22<1:01:29,  1.22it/s, gpt_loss=0.109, lr=4.67e-6]
+
Train step of epoch 0:  20%|██        | 1125/5616 [14:22<1:01:29,  1.22it/s, gpt_loss=0.0144, lr=4.67e-6]
+
Train step of epoch 0:  20%|██        | 1126/5616 [14:22<1:01:22,  1.22it/s, gpt_loss=0.0144, lr=4.67e-6]
+
Train step of epoch 0:  20%|██        | 1126/5616 [14:23<1:01:22,  1.22it/s, gpt_loss=0.00132, lr=4.67e-6]
+
Train step of epoch 0:  20%|██        | 1127/5616 [14:23<57:34,  1.30it/s, gpt_loss=0.00132, lr=4.67e-6]  
+
Train step of epoch 0:  20%|██        | 1127/5616 [14:24<57:34,  1.30it/s, gpt_loss=0.115, lr=4.67e-6]  
+
Train step of epoch 0:  20%|██        | 1128/5616 [14:24<56:51,  1.32it/s, gpt_loss=0.115, lr=4.67e-6]
+
Train step of epoch 0:  20%|██        | 1128/5616 [14:25<56:51,  1.32it/s, gpt_loss=0.026, lr=4.67e-6]
+
Train step of epoch 0:  20%|██        | 1129/5616 [14:25<55:21,  1.35it/s, gpt_loss=0.026, lr=4.67e-6]
+
Train step of epoch 0:  20%|██        | 1129/5616 [14:25<55:21,  1.35it/s, gpt_loss=0.0595, lr=4.67e-6]
+
Train step of epoch 0:  20%|██        | 1130/5616 [14:25<53:52,  1.39it/s, gpt_loss=0.0595, lr=4.67e-6]
+
Train step of epoch 0:  20%|██        | 1130/5616 [14:26<53:52,  1.39it/s, gpt_loss=0.129, lr=4.67e-6] 
+
Train step of epoch 0:  20%|██        | 1131/5616 [14:26<52:43,  1.42it/s, gpt_loss=0.129, lr=4.67e-6]
+
Train step of epoch 0:  20%|██        | 1131/5616 [14:27<52:43,  1.42it/s, gpt_loss=0.00284, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1132/5616 [14:27<55:41,  1.34it/s, gpt_loss=0.00284, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1132/5616 [14:27<55:41,  1.34it/s, gpt_loss=0.0611, lr=4.66e-6] 
+
Train step of epoch 0:  20%|██        | 1133/5616 [14:27<54:22,  1.37it/s, gpt_loss=0.0611, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1133/5616 [14:28<54:22,  1.37it/s, gpt_loss=0.0441, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1134/5616 [14:28<55:28,  1.35it/s, gpt_loss=0.0441, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1134/5616 [14:29<55:28,  1.35it/s, gpt_loss=0.00733, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1135/5616 [14:29<54:40,  1.37it/s, gpt_loss=0.00733, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1135/5616 [14:30<54:40,  1.37it/s, gpt_loss=0.0715, lr=4.66e-6] 
+
Train step of epoch 0:  20%|██        | 1136/5616 [14:30<55:02,  1.36it/s, gpt_loss=0.0715, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1136/5616 [14:30<55:02,  1.36it/s, gpt_loss=0.0128, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1137/5616 [14:30<54:54,  1.36it/s, gpt_loss=0.0128, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1137/5616 [14:31<54:54,  1.36it/s, gpt_loss=0.0376, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1138/5616 [14:31<55:33,  1.34it/s, gpt_loss=0.0376, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1138/5616 [14:32<55:33,  1.34it/s, gpt_loss=0.0146, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1139/5616 [14:32<53:22,  1.40it/s, gpt_loss=0.0146, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1139/5616 [14:33<53:22,  1.40it/s, gpt_loss=0.0193, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1140/5616 [14:33<54:17,  1.37it/s, gpt_loss=0.0193, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1140/5616 [14:33<54:17,  1.37it/s, gpt_loss=0.111, lr=4.66e-6] 
+
Train step of epoch 0:  20%|██        | 1141/5616 [14:33<57:00,  1.31it/s, gpt_loss=0.111, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1141/5616 [14:34<57:00,  1.31it/s, gpt_loss=0.0672, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1142/5616 [14:34<1:01:35,  1.21it/s, gpt_loss=0.0672, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1142/5616 [14:35<1:01:35,  1.21it/s, gpt_loss=0.0284, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1143/5616 [14:35<59:23,  1.26it/s, gpt_loss=0.0284, lr=4.66e-6]  
+
Train step of epoch 0:  20%|██        | 1143/5616 [14:36<59:23,  1.26it/s, gpt_loss=0.0737, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1144/5616 [14:36<1:00:40,  1.23it/s, gpt_loss=0.0737, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1144/5616 [14:37<1:00:40,  1.23it/s, gpt_loss=0.0282, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1145/5616 [14:37<58:01,  1.28it/s, gpt_loss=0.0282, lr=4.66e-6]  
+
Train step of epoch 0:  20%|██        | 1145/5616 [14:37<58:01,  1.28it/s, gpt_loss=0.0202, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1146/5616 [14:37<57:30,  1.30it/s, gpt_loss=0.0202, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1146/5616 [14:38<57:30,  1.30it/s, gpt_loss=0.0281, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1147/5616 [14:38<56:28,  1.32it/s, gpt_loss=0.0281, lr=4.66e-6]
+
Train step of epoch 0:  20%|██        | 1147/5616 [14:39<56:28,  1.32it/s, gpt_loss=0.0444, lr=4.65e-6]
+
Train step of epoch 0:  20%|██        | 1148/5616 [14:39<58:35,  1.27it/s, gpt_loss=0.0444, lr=4.65e-6]
+
Train step of epoch 0:  20%|██        | 1148/5616 [14:40<58:35,  1.27it/s, gpt_loss=0.0346, lr=4.65e-6]
+
Train step of epoch 0:  20%|██        | 1149/5616 [14:40<56:36,  1.32it/s, gpt_loss=0.0346, lr=4.65e-6]
+
Train step of epoch 0:  20%|██        | 1149/5616 [14:41<56:36,  1.32it/s, gpt_loss=0.0098, lr=4.65e-6]
+
Train step of epoch 0:  20%|██        | 1150/5616 [14:41<1:04:23,  1.16it/s, gpt_loss=0.0098, lr=4.65e-6]
+
Train step of epoch 0:  20%|██        | 1150/5616 [14:41<1:04:23,  1.16it/s, gpt_loss=0.0405, lr=4.65e-6]
+
Train step of epoch 0:  20%|██        | 1151/5616 [14:41<59:47,  1.24it/s, gpt_loss=0.0405, lr=4.65e-6]  
+
Train step of epoch 0:  20%|██        | 1151/5616 [14:42<59:47,  1.24it/s, gpt_loss=0.042, lr=4.65e-6] 
+
Train step of epoch 0:  21%|██        | 1152/5616 [14:42<58:28,  1.27it/s, gpt_loss=0.042, lr=4.65e-6]
+
Train step of epoch 0:  21%|██        | 1152/5616 [14:43<58:28,  1.27it/s, gpt_loss=0.137, lr=4.65e-6]
+
Train step of epoch 0:  21%|██        | 1153/5616 [14:43<59:18,  1.25it/s, gpt_loss=0.137, lr=4.65e-6]
+
Train step of epoch 0:  21%|██        | 1153/5616 [14:44<59:18,  1.25it/s, gpt_loss=0.0939, lr=4.65e-6]
+
Train step of epoch 0:  21%|██        | 1154/5616 [14:44<56:22,  1.32it/s, gpt_loss=0.0939, lr=4.65e-6]
+
Train step of epoch 0:  21%|██        | 1154/5616 [14:44<56:22,  1.32it/s, gpt_loss=0.102, lr=4.65e-6] 
+
Train step of epoch 0:  21%|██        | 1155/5616 [14:44<56:06,  1.33it/s, gpt_loss=0.102, lr=4.65e-6]
+
Train step of epoch 0:  21%|██        | 1155/5616 [14:45<56:06,  1.33it/s, gpt_loss=0.0306, lr=4.65e-6]
+
Train step of epoch 0:  21%|██        | 1156/5616 [14:45<56:06,  1.32it/s, gpt_loss=0.0306, lr=4.65e-6]
+
Train step of epoch 0:  21%|██        | 1156/5616 [14:46<56:06,  1.32it/s, gpt_loss=0.0462, lr=4.65e-6]
+
Train step of epoch 0:  21%|██        | 1157/5616 [14:46<56:11,  1.32it/s, gpt_loss=0.0462, lr=4.65e-6]
+
Train step of epoch 0:  21%|██        | 1157/5616 [14:47<56:11,  1.32it/s, gpt_loss=0.0558, lr=4.65e-6]
+
Train step of epoch 0:  21%|██        | 1158/5616 [14:47<56:15,  1.32it/s, gpt_loss=0.0558, lr=4.65e-6]
+
Train step of epoch 0:  21%|██        | 1158/5616 [14:47<56:15,  1.32it/s, gpt_loss=0.054, lr=4.65e-6] 
+
Train step of epoch 0:  21%|██        | 1159/5616 [14:47<55:19,  1.34it/s, gpt_loss=0.054, lr=4.65e-6]
+
Train step of epoch 0:  21%|██        | 1159/5616 [14:48<55:19,  1.34it/s, gpt_loss=0.0193, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1160/5616 [14:48<57:10,  1.30it/s, gpt_loss=0.0193, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1160/5616 [14:49<57:10,  1.30it/s, gpt_loss=0.0418, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1161/5616 [14:49<57:53,  1.28it/s, gpt_loss=0.0418, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1161/5616 [14:50<57:53,  1.28it/s, gpt_loss=0.0917, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1162/5616 [14:50<58:13,  1.27it/s, gpt_loss=0.0917, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1162/5616 [14:51<58:13,  1.27it/s, gpt_loss=0.0381, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1163/5616 [14:51<58:15,  1.27it/s, gpt_loss=0.0381, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1163/5616 [14:51<58:15,  1.27it/s, gpt_loss=0.0751, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1164/5616 [14:51<58:29,  1.27it/s, gpt_loss=0.0751, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1164/5616 [14:52<58:29,  1.27it/s, gpt_loss=0.0536, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1165/5616 [14:52<58:51,  1.26it/s, gpt_loss=0.0536, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1165/5616 [14:53<58:51,  1.26it/s, gpt_loss=0.0553, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1166/5616 [14:53<56:04,  1.32it/s, gpt_loss=0.0553, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1166/5616 [14:54<56:04,  1.32it/s, gpt_loss=0.0113, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1167/5616 [14:54<56:35,  1.31it/s, gpt_loss=0.0113, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1167/5616 [14:55<56:35,  1.31it/s, gpt_loss=0.0327, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1168/5616 [14:55<58:55,  1.26it/s, gpt_loss=0.0327, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1168/5616 [14:55<58:55,  1.26it/s, gpt_loss=0.0455, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1169/5616 [14:55<56:58,  1.30it/s, gpt_loss=0.0455, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1169/5616 [14:56<56:58,  1.30it/s, gpt_loss=0.0647, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1170/5616 [14:56<55:43,  1.33it/s, gpt_loss=0.0647, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1170/5616 [14:57<55:43,  1.33it/s, gpt_loss=0.131, lr=4.64e-6] 
+
Train step of epoch 0:  21%|██        | 1171/5616 [14:57<1:07:29,  1.10it/s, gpt_loss=0.131, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1171/5616 [14:58<1:07:29,  1.10it/s, gpt_loss=0.0293, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1172/5616 [14:58<1:05:32,  1.13it/s, gpt_loss=0.0293, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1172/5616 [14:59<1:05:32,  1.13it/s, gpt_loss=0.0852, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1173/5616 [14:59<1:04:13,  1.15it/s, gpt_loss=0.0852, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1173/5616 [15:00<1:04:13,  1.15it/s, gpt_loss=0.0376, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1174/5616 [15:00<1:02:43,  1.18it/s, gpt_loss=0.0376, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1174/5616 [15:00<1:02:43,  1.18it/s, gpt_loss=0.0147, lr=4.64e-6]
+
Train step of epoch 0:  21%|██        | 1175/5616 [15:00<58:56,  1.26it/s, gpt_loss=0.0147, lr=4.64e-6]  
+
Train step of epoch 0:  21%|██        | 1175/5616 [15:01<58:56,  1.26it/s, gpt_loss=0.0679, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1176/5616 [15:01<1:00:16,  1.23it/s, gpt_loss=0.0679, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1176/5616 [15:02<1:00:16,  1.23it/s, gpt_loss=0.105, lr=4.63e-6] 
+
Train step of epoch 0:  21%|██        | 1177/5616 [15:02<59:20,  1.25it/s, gpt_loss=0.105, lr=4.63e-6]  
+
Train step of epoch 0:  21%|██        | 1177/5616 [15:03<59:20,  1.25it/s, gpt_loss=0.0676, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1178/5616 [15:03<57:01,  1.30it/s, gpt_loss=0.0676, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1178/5616 [15:03<57:01,  1.30it/s, gpt_loss=0.0424, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1179/5616 [15:03<55:30,  1.33it/s, gpt_loss=0.0424, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1179/5616 [15:04<55:30,  1.33it/s, gpt_loss=0.202, lr=4.63e-6] 
+
Train step of epoch 0:  21%|██        | 1180/5616 [15:04<57:27,  1.29it/s, gpt_loss=0.202, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1180/5616 [15:05<57:27,  1.29it/s, gpt_loss=0.00716, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1181/5616 [15:05<56:00,  1.32it/s, gpt_loss=0.00716, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1181/5616 [15:06<56:00,  1.32it/s, gpt_loss=0.0532, lr=4.63e-6] 
+
Train step of epoch 0:  21%|██        | 1182/5616 [15:06<54:22,  1.36it/s, gpt_loss=0.0532, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1182/5616 [15:06<54:22,  1.36it/s, gpt_loss=0.0643, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1183/5616 [15:06<53:30,  1.38it/s, gpt_loss=0.0643, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1183/5616 [15:07<53:30,  1.38it/s, gpt_loss=0.0526, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1184/5616 [15:07<56:25,  1.31it/s, gpt_loss=0.0526, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1184/5616 [15:08<56:25,  1.31it/s, gpt_loss=0.0231, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1185/5616 [15:08<57:07,  1.29it/s, gpt_loss=0.0231, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1185/5616 [15:09<57:07,  1.29it/s, gpt_loss=0.0185, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1186/5616 [15:09<57:43,  1.28it/s, gpt_loss=0.0185, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1186/5616 [15:09<57:43,  1.28it/s, gpt_loss=0.0781, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1187/5616 [15:09<54:52,  1.35it/s, gpt_loss=0.0781, lr=4.63e-6]
+
Train step of epoch 0:  21%|██        | 1187/5616 [15:10<54:52,  1.35it/s, gpt_loss=0.0158, lr=4.62e-6]
+
Train step of epoch 0:  21%|██        | 1188/5616 [15:10<58:28,  1.26it/s, gpt_loss=0.0158, lr=4.62e-6]
+
Train step of epoch 0:  21%|██        | 1188/5616 [15:11<58:28,  1.26it/s, gpt_loss=0.0205, lr=4.62e-6]
+
Train step of epoch 0:  21%|██        | 1189/5616 [15:11<58:36,  1.26it/s, gpt_loss=0.0205, lr=4.62e-6]
+
Train step of epoch 0:  21%|██        | 1189/5616 [15:12<58:36,  1.26it/s, gpt_loss=0.111, lr=4.62e-6] 
+
Train step of epoch 0:  21%|██        | 1190/5616 [15:12<56:41,  1.30it/s, gpt_loss=0.111, lr=4.62e-6]
+
Train step of epoch 0:  21%|██        | 1190/5616 [15:13<56:41,  1.30it/s, gpt_loss=0.00857, lr=4.62e-6]
+
Train step of epoch 0:  21%|██        | 1191/5616 [15:13<56:08,  1.31it/s, gpt_loss=0.00857, lr=4.62e-6]
+
Train step of epoch 0:  21%|██        | 1191/5616 [15:13<56:08,  1.31it/s, gpt_loss=0.0105, lr=4.62e-6] 
+
Train step of epoch 0:  21%|██        | 1192/5616 [15:13<55:44,  1.32it/s, gpt_loss=0.0105, lr=4.62e-6]
+
Train step of epoch 0:  21%|██        | 1192/5616 [15:14<55:44,  1.32it/s, gpt_loss=0.017, lr=4.62e-6] 
+
Train step of epoch 0:  21%|██        | 1193/5616 [15:14<56:26,  1.31it/s, gpt_loss=0.017, lr=4.62e-6]
+
Train step of epoch 0:  21%|██        | 1193/5616 [15:15<56:26,  1.31it/s, gpt_loss=0.0579, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1194/5616 [15:15<54:35,  1.35it/s, gpt_loss=0.0579, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1194/5616 [15:16<54:35,  1.35it/s, gpt_loss=0.0684, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1195/5616 [15:16<55:51,  1.32it/s, gpt_loss=0.0684, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1195/5616 [15:16<55:51,  1.32it/s, gpt_loss=0.0658, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1196/5616 [15:16<56:21,  1.31it/s, gpt_loss=0.0658, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1196/5616 [15:17<56:21,  1.31it/s, gpt_loss=0.0743, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1197/5616 [15:17<57:26,  1.28it/s, gpt_loss=0.0743, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1197/5616 [15:18<57:26,  1.28it/s, gpt_loss=0.0689, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1198/5616 [15:18<55:41,  1.32it/s, gpt_loss=0.0689, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1198/5616 [15:19<55:41,  1.32it/s, gpt_loss=0.0122, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1199/5616 [15:19<54:19,  1.36it/s, gpt_loss=0.0122, lr=4.62e-6][2025-06-30 06:53:28,350] [INFO] [logging.py:128:log_dist] [Rank 0] step=300, skipped=0, lr=[4.615560395294549e-06, 4.615560395294549e-06], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 06:53:28,351] [INFO] [timer.py:264:stop] epoch=0/micro_step=1200/global_step=300, RunningAvgSamplesPerSec=42.375648152742095, CurrSamplesPerSec=44.137663982600095, MemAllocated=15.89GB, MaxMemAllocated=51.79GB
+
+
Train step of epoch 0:  21%|██▏       | 1199/5616 [15:19<54:19,  1.36it/s, gpt_loss=0.101, lr=4.62e-6] 
+
Train step of epoch 0:  21%|██▏       | 1200/5616 [15:19<54:05,  1.36it/s, gpt_loss=0.101, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1200/5616 [15:20<54:05,  1.36it/s, gpt_loss=0.00428, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1201/5616 [15:20<56:58,  1.29it/s, gpt_loss=0.00428, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1201/5616 [15:21<56:58,  1.29it/s, gpt_loss=0.0923, lr=4.62e-6] 
+
Train step of epoch 0:  21%|██▏       | 1202/5616 [15:21<54:58,  1.34it/s, gpt_loss=0.0923, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1202/5616 [15:22<54:58,  1.34it/s, gpt_loss=0.0415, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1203/5616 [15:22<56:51,  1.29it/s, gpt_loss=0.0415, lr=4.62e-6]
+
Train step of epoch 0:  21%|██▏       | 1203/5616 [15:22<56:51,  1.29it/s, gpt_loss=0.0176, lr=4.61e-6]
+
Train step of epoch 0:  21%|██▏       | 1204/5616 [15:22<57:25,  1.28it/s, gpt_loss=0.0176, lr=4.61e-6]
+
Train step of epoch 0:  21%|██▏       | 1204/5616 [15:23<57:25,  1.28it/s, gpt_loss=0.0305, lr=4.61e-6]
+
Train step of epoch 0:  21%|██▏       | 1205/5616 [15:23<58:17,  1.26it/s, gpt_loss=0.0305, lr=4.61e-6]
+
Train step of epoch 0:  21%|██▏       | 1205/5616 [15:24<58:17,  1.26it/s, gpt_loss=0.0606, lr=4.61e-6]
+
Train step of epoch 0:  21%|██▏       | 1206/5616 [15:24<57:31,  1.28it/s, gpt_loss=0.0606, lr=4.61e-6]
+
Train step of epoch 0:  21%|██▏       | 1206/5616 [15:25<57:31,  1.28it/s, gpt_loss=0.0432, lr=4.61e-6]
+
Train step of epoch 0:  21%|██▏       | 1207/5616 [15:25<57:54,  1.27it/s, gpt_loss=0.0432, lr=4.61e-6]
+
Train step of epoch 0:  21%|██▏       | 1207/5616 [15:26<57:54,  1.27it/s, gpt_loss=0.00459, lr=4.61e-6]
+
Train step of epoch 0:  22%|██▏       | 1208/5616 [15:26<56:37,  1.30it/s, gpt_loss=0.00459, lr=4.61e-6]
+
Train step of epoch 0:  22%|██▏       | 1208/5616 [15:26<56:37,  1.30it/s, gpt_loss=0.0142, lr=4.61e-6] 
+
Train step of epoch 0:  22%|██▏       | 1209/5616 [15:26<54:52,  1.34it/s, gpt_loss=0.0142, lr=4.61e-6]
+
Train step of epoch 0:  22%|██▏       | 1209/5616 [15:27<54:52,  1.34it/s, gpt_loss=0.0672, lr=4.61e-6]
+
Train step of epoch 0:  22%|██▏       | 1210/5616 [15:27<53:27,  1.37it/s, gpt_loss=0.0672, lr=4.61e-6]
+
Train step of epoch 0:  22%|██▏       | 1210/5616 [15:28<53:27,  1.37it/s, gpt_loss=0.0375, lr=4.61e-6]
+
Train step of epoch 0:  22%|██▏       | 1211/5616 [15:28<55:45,  1.32it/s, gpt_loss=0.0375, lr=4.61e-6]
+
Train step of epoch 0:  22%|██▏       | 1211/5616 [15:29<55:45,  1.32it/s, gpt_loss=0.00245, lr=4.61e-6]
+
Train step of epoch 0:  22%|██▏       | 1212/5616 [15:29<56:05,  1.31it/s, gpt_loss=0.00245, lr=4.61e-6]
+
Train step of epoch 0:  22%|██▏       | 1212/5616 [15:29<56:05,  1.31it/s, gpt_loss=0.0318, lr=4.61e-6] 
+
Train step of epoch 0:  22%|██▏       | 1213/5616 [15:29<54:39,  1.34it/s, gpt_loss=0.0318, lr=4.61e-6]
+
Train step of epoch 0:  22%|██▏       | 1213/5616 [15:30<54:39,  1.34it/s, gpt_loss=0.163, lr=4.61e-6] 
+
Train step of epoch 0:  22%|██▏       | 1214/5616 [15:30<55:15,  1.33it/s, gpt_loss=0.163, lr=4.61e-6]
+
Train step of epoch 0:  22%|██▏       | 1214/5616 [15:31<55:15,  1.33it/s, gpt_loss=0.0857, lr=4.61e-6]
+
Train step of epoch 0:  22%|██▏       | 1215/5616 [15:31<54:43,  1.34it/s, gpt_loss=0.0857, lr=4.61e-6]
+
Train step of epoch 0:  22%|██▏       | 1215/5616 [15:32<54:43,  1.34it/s, gpt_loss=0.0829, lr=4.6e-6] 
+
Train step of epoch 0:  22%|██▏       | 1216/5616 [15:32<55:04,  1.33it/s, gpt_loss=0.0829, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1216/5616 [15:32<55:04,  1.33it/s, gpt_loss=0.0208, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1217/5616 [15:32<54:04,  1.36it/s, gpt_loss=0.0208, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1217/5616 [15:33<54:04,  1.36it/s, gpt_loss=0.0243, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1218/5616 [15:33<54:26,  1.35it/s, gpt_loss=0.0243, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1218/5616 [15:34<54:26,  1.35it/s, gpt_loss=0.0403, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1219/5616 [15:34<59:41,  1.23it/s, gpt_loss=0.0403, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1219/5616 [15:35<59:41,  1.23it/s, gpt_loss=0.012, lr=4.6e-6] 
+
Train step of epoch 0:  22%|██▏       | 1220/5616 [15:35<59:05,  1.24it/s, gpt_loss=0.012, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1220/5616 [15:36<59:05,  1.24it/s, gpt_loss=0.0943, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1221/5616 [15:36<57:34,  1.27it/s, gpt_loss=0.0943, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1221/5616 [15:36<57:34,  1.27it/s, gpt_loss=0.0109, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1222/5616 [15:36<55:45,  1.31it/s, gpt_loss=0.0109, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1222/5616 [15:37<55:45,  1.31it/s, gpt_loss=0.0862, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1223/5616 [15:37<54:11,  1.35it/s, gpt_loss=0.0862, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1223/5616 [15:38<54:11,  1.35it/s, gpt_loss=0.0941, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1224/5616 [15:38<1:01:28,  1.19it/s, gpt_loss=0.0941, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1224/5616 [15:39<1:01:28,  1.19it/s, gpt_loss=0.0206, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1225/5616 [15:39<1:00:32,  1.21it/s, gpt_loss=0.0206, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1225/5616 [15:39<1:00:32,  1.21it/s, gpt_loss=0.0189, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1226/5616 [15:39<56:40,  1.29it/s, gpt_loss=0.0189, lr=4.6e-6]  
+
Train step of epoch 0:  22%|██▏       | 1226/5616 [15:40<56:40,  1.29it/s, gpt_loss=0.0761, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1227/5616 [15:40<54:26,  1.34it/s, gpt_loss=0.0761, lr=4.6e-6]
+
Train step of epoch 0:  22%|██▏       | 1227/5616 [15:41<54:26,  1.34it/s, gpt_loss=0.0138, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1228/5616 [15:41<55:43,  1.31it/s, gpt_loss=0.0138, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1228/5616 [15:42<55:43,  1.31it/s, gpt_loss=0.0632, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1229/5616 [15:42<53:50,  1.36it/s, gpt_loss=0.0632, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1229/5616 [15:42<53:50,  1.36it/s, gpt_loss=0.12, lr=4.59e-6]  
+
Train step of epoch 0:  22%|██▏       | 1230/5616 [15:42<55:04,  1.33it/s, gpt_loss=0.12, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1230/5616 [15:43<55:04,  1.33it/s, gpt_loss=0.00782, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1231/5616 [15:43<54:06,  1.35it/s, gpt_loss=0.00782, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1231/5616 [15:44<54:06,  1.35it/s, gpt_loss=0.0212, lr=4.59e-6] 
+
Train step of epoch 0:  22%|██▏       | 1232/5616 [15:44<56:32,  1.29it/s, gpt_loss=0.0212, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1232/5616 [15:45<56:32,  1.29it/s, gpt_loss=0.0741, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1233/5616 [15:45<53:50,  1.36it/s, gpt_loss=0.0741, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1233/5616 [15:45<53:50,  1.36it/s, gpt_loss=0.036, lr=4.59e-6] 
+
Train step of epoch 0:  22%|██▏       | 1234/5616 [15:45<52:42,  1.39it/s, gpt_loss=0.036, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1234/5616 [15:46<52:42,  1.39it/s, gpt_loss=0.0447, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1235/5616 [15:46<51:42,  1.41it/s, gpt_loss=0.0447, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1235/5616 [15:47<51:42,  1.41it/s, gpt_loss=0.061, lr=4.59e-6] 
+
Train step of epoch 0:  22%|██▏       | 1236/5616 [15:47<53:11,  1.37it/s, gpt_loss=0.061, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1236/5616 [15:47<53:11,  1.37it/s, gpt_loss=0.106, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1237/5616 [15:47<53:33,  1.36it/s, gpt_loss=0.106, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1237/5616 [15:48<53:33,  1.36it/s, gpt_loss=0.0155, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1238/5616 [15:48<54:23,  1.34it/s, gpt_loss=0.0155, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1238/5616 [15:49<54:23,  1.34it/s, gpt_loss=0.0282, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1239/5616 [15:49<52:57,  1.38it/s, gpt_loss=0.0282, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1239/5616 [15:50<52:57,  1.38it/s, gpt_loss=0.013, lr=4.59e-6] 
+
Train step of epoch 0:  22%|██▏       | 1240/5616 [15:50<55:00,  1.33it/s, gpt_loss=0.013, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1240/5616 [15:50<55:00,  1.33it/s, gpt_loss=0.0408, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1241/5616 [15:50<53:31,  1.36it/s, gpt_loss=0.0408, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1241/5616 [15:51<53:31,  1.36it/s, gpt_loss=0.0268, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1242/5616 [15:51<54:56,  1.33it/s, gpt_loss=0.0268, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1242/5616 [15:52<54:56,  1.33it/s, gpt_loss=0.051, lr=4.59e-6] 
+
Train step of epoch 0:  22%|██▏       | 1243/5616 [15:52<53:31,  1.36it/s, gpt_loss=0.051, lr=4.59e-6]
+
Train step of epoch 0:  22%|██▏       | 1243/5616 [15:53<53:31,  1.36it/s, gpt_loss=0.0462, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1244/5616 [15:53<55:17,  1.32it/s, gpt_loss=0.0462, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1244/5616 [15:53<55:17,  1.32it/s, gpt_loss=0.0604, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1245/5616 [15:53<53:55,  1.35it/s, gpt_loss=0.0604, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1245/5616 [15:54<53:55,  1.35it/s, gpt_loss=0.0411, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1246/5616 [15:54<53:51,  1.35it/s, gpt_loss=0.0411, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1246/5616 [15:55<53:51,  1.35it/s, gpt_loss=0.0391, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1247/5616 [15:55<52:25,  1.39it/s, gpt_loss=0.0391, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1247/5616 [15:56<52:25,  1.39it/s, gpt_loss=0.0541, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1248/5616 [15:56<53:49,  1.35it/s, gpt_loss=0.0541, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1248/5616 [15:56<53:49,  1.35it/s, gpt_loss=0.104, lr=4.58e-6] 
+
Train step of epoch 0:  22%|██▏       | 1249/5616 [15:56<51:42,  1.41it/s, gpt_loss=0.104, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1249/5616 [15:57<51:42,  1.41it/s, gpt_loss=0.0665, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1250/5616 [15:57<50:16,  1.45it/s, gpt_loss=0.0665, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1250/5616 [15:58<50:16,  1.45it/s, gpt_loss=0.0725, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1251/5616 [15:58<49:02,  1.48it/s, gpt_loss=0.0725, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1251/5616 [15:58<49:02,  1.48it/s, gpt_loss=0.00811, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1252/5616 [15:58<51:42,  1.41it/s, gpt_loss=0.00811, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1252/5616 [15:59<51:42,  1.41it/s, gpt_loss=0.0646, lr=4.58e-6] 
+
Train step of epoch 0:  22%|██▏       | 1253/5616 [15:59<51:46,  1.40it/s, gpt_loss=0.0646, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1253/5616 [16:00<51:46,  1.40it/s, gpt_loss=0.0611, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1254/5616 [16:00<54:53,  1.32it/s, gpt_loss=0.0611, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1254/5616 [16:01<54:53,  1.32it/s, gpt_loss=0.136, lr=4.58e-6] 
+
Train step of epoch 0:  22%|██▏       | 1255/5616 [16:01<53:12,  1.37it/s, gpt_loss=0.136, lr=4.58e-6]
+
Train step of epoch 0:  22%|██▏       | 1255/5616 [16:02<53:12,  1.37it/s, gpt_loss=0.0691, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1256/5616 [16:02<59:57,  1.21it/s, gpt_loss=0.0691, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1256/5616 [16:02<59:57,  1.21it/s, gpt_loss=0.0682, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1257/5616 [16:02<59:14,  1.23it/s, gpt_loss=0.0682, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1257/5616 [16:03<59:14,  1.23it/s, gpt_loss=0.053, lr=4.57e-6] 
+
Train step of epoch 0:  22%|██▏       | 1258/5616 [16:03<56:29,  1.29it/s, gpt_loss=0.053, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1258/5616 [16:04<56:29,  1.29it/s, gpt_loss=0.0779, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1259/5616 [16:04<55:20,  1.31it/s, gpt_loss=0.0779, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1259/5616 [16:05<55:20,  1.31it/s, gpt_loss=0.036, lr=4.57e-6] 
+
Train step of epoch 0:  22%|██▏       | 1260/5616 [16:05<56:44,  1.28it/s, gpt_loss=0.036, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1260/5616 [16:05<56:44,  1.28it/s, gpt_loss=0.0487, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1261/5616 [16:05<55:26,  1.31it/s, gpt_loss=0.0487, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1261/5616 [16:06<55:26,  1.31it/s, gpt_loss=0.0534, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1262/5616 [16:06<56:45,  1.28it/s, gpt_loss=0.0534, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1262/5616 [16:07<56:45,  1.28it/s, gpt_loss=0.0841, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1263/5616 [16:07<53:45,  1.35it/s, gpt_loss=0.0841, lr=4.57e-6]
+
Train step of epoch 0:  22%|██▏       | 1263/5616 [16:08<53:45,  1.35it/s, gpt_loss=0.0183, lr=4.57e-6]
+
Train step of epoch 0:  23%|██▎       | 1264/5616 [16:08<55:46,  1.30it/s, gpt_loss=0.0183, lr=4.57e-6]
+
Train step of epoch 0:  23%|██▎       | 1264/5616 [16:08<55:46,  1.30it/s, gpt_loss=0.0335, lr=4.57e-6]
+
Train step of epoch 0:  23%|██▎       | 1265/5616 [16:08<52:56,  1.37it/s, gpt_loss=0.0335, lr=4.57e-6]
+
Train step of epoch 0:  23%|██▎       | 1265/5616 [16:09<52:56,  1.37it/s, gpt_loss=0.0865, lr=4.57e-6]
+
Train step of epoch 0:  23%|██▎       | 1266/5616 [16:09<51:52,  1.40it/s, gpt_loss=0.0865, lr=4.57e-6]
+
Train step of epoch 0:  23%|██▎       | 1266/5616 [16:10<51:52,  1.40it/s, gpt_loss=0.0446, lr=4.57e-6]
+
Train step of epoch 0:  23%|██▎       | 1267/5616 [16:10<51:12,  1.42it/s, gpt_loss=0.0446, lr=4.57e-6]
+
Train step of epoch 0:  23%|██▎       | 1267/5616 [16:10<51:12,  1.42it/s, gpt_loss=0.0345, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1268/5616 [16:10<52:32,  1.38it/s, gpt_loss=0.0345, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1268/5616 [16:11<52:32,  1.38it/s, gpt_loss=0.083, lr=4.56e-6] 
+
Train step of epoch 0:  23%|██▎       | 1269/5616 [16:11<52:17,  1.39it/s, gpt_loss=0.083, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1269/5616 [16:12<52:17,  1.39it/s, gpt_loss=0.0106, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1270/5616 [16:12<55:03,  1.32it/s, gpt_loss=0.0106, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1270/5616 [16:13<55:03,  1.32it/s, gpt_loss=0.0732, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1271/5616 [16:13<53:20,  1.36it/s, gpt_loss=0.0732, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1271/5616 [16:14<53:20,  1.36it/s, gpt_loss=0.036, lr=4.56e-6] 
+
Train step of epoch 0:  23%|██▎       | 1272/5616 [16:14<57:36,  1.26it/s, gpt_loss=0.036, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1272/5616 [16:15<57:36,  1.26it/s, gpt_loss=0.0204, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1273/5616 [16:15<1:04:03,  1.13it/s, gpt_loss=0.0204, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1273/5616 [16:15<1:04:03,  1.13it/s, gpt_loss=0.0263, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1274/5616 [16:15<1:00:37,  1.19it/s, gpt_loss=0.0263, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1274/5616 [16:16<1:00:37,  1.19it/s, gpt_loss=0.0292, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1275/5616 [16:16<58:16,  1.24it/s, gpt_loss=0.0292, lr=4.56e-6]  
+
Train step of epoch 0:  23%|██▎       | 1275/5616 [16:17<58:16,  1.24it/s, gpt_loss=0.00733, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1276/5616 [16:17<57:33,  1.26it/s, gpt_loss=0.00733, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1276/5616 [16:18<57:33,  1.26it/s, gpt_loss=0.0246, lr=4.56e-6] 
+
Train step of epoch 0:  23%|██▎       | 1277/5616 [16:18<57:39,  1.25it/s, gpt_loss=0.0246, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1277/5616 [16:19<57:39,  1.25it/s, gpt_loss=0.0233, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1278/5616 [16:19<58:13,  1.24it/s, gpt_loss=0.0233, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1278/5616 [16:19<58:13,  1.24it/s, gpt_loss=0.0547, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1279/5616 [16:19<57:18,  1.26it/s, gpt_loss=0.0547, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1279/5616 [16:20<57:18,  1.26it/s, gpt_loss=0.0605, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1280/5616 [16:20<58:04,  1.24it/s, gpt_loss=0.0605, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1280/5616 [16:21<58:04,  1.24it/s, gpt_loss=0.0953, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1281/5616 [16:21<1:01:15,  1.18it/s, gpt_loss=0.0953, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1281/5616 [16:22<1:01:15,  1.18it/s, gpt_loss=0.0622, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1282/5616 [16:22<1:03:00,  1.15it/s, gpt_loss=0.0622, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1282/5616 [16:23<1:03:00,  1.15it/s, gpt_loss=0.0514, lr=4.56e-6]
+
Train step of epoch 0:  23%|██▎       | 1283/5616 [16:23<57:36,  1.25it/s, gpt_loss=0.0514, lr=4.56e-6]  
+
Train step of epoch 0:  23%|██▎       | 1283/5616 [16:24<57:36,  1.25it/s, gpt_loss=0.133, lr=4.55e-6] 
+
Train step of epoch 0:  23%|██▎       | 1284/5616 [16:24<58:45,  1.23it/s, gpt_loss=0.133, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1284/5616 [16:24<58:45,  1.23it/s, gpt_loss=0.038, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1285/5616 [16:24<59:47,  1.21it/s, gpt_loss=0.038, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1285/5616 [16:25<59:47,  1.21it/s, gpt_loss=0.0267, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1286/5616 [16:25<59:12,  1.22it/s, gpt_loss=0.0267, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1286/5616 [16:26<59:12,  1.22it/s, gpt_loss=0.0198, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1287/5616 [16:26<56:04,  1.29it/s, gpt_loss=0.0198, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1287/5616 [16:27<56:04,  1.29it/s, gpt_loss=0.101, lr=4.55e-6] 
+
Train step of epoch 0:  23%|██▎       | 1288/5616 [16:27<55:48,  1.29it/s, gpt_loss=0.101, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1288/5616 [16:27<55:48,  1.29it/s, gpt_loss=0.0187, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1289/5616 [16:27<53:57,  1.34it/s, gpt_loss=0.0187, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1289/5616 [16:28<53:57,  1.34it/s, gpt_loss=0.0911, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1290/5616 [16:28<53:05,  1.36it/s, gpt_loss=0.0911, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1290/5616 [16:29<53:05,  1.36it/s, gpt_loss=0.133, lr=4.55e-6] 
+
Train step of epoch 0:  23%|██▎       | 1291/5616 [16:29<50:55,  1.42it/s, gpt_loss=0.133, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1291/5616 [16:29<50:55,  1.42it/s, gpt_loss=0.0861, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1292/5616 [16:29<51:29,  1.40it/s, gpt_loss=0.0861, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1292/5616 [16:30<51:29,  1.40it/s, gpt_loss=0.042, lr=4.55e-6] 
+
Train step of epoch 0:  23%|██▎       | 1293/5616 [16:30<51:04,  1.41it/s, gpt_loss=0.042, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1293/5616 [16:31<51:04,  1.41it/s, gpt_loss=0.0729, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1294/5616 [16:31<50:18,  1.43it/s, gpt_loss=0.0729, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1294/5616 [16:32<50:18,  1.43it/s, gpt_loss=0.0165, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1295/5616 [16:32<54:01,  1.33it/s, gpt_loss=0.0165, lr=4.55e-6]
+
Train step of epoch 0:  23%|██▎       | 1295/5616 [16:33<54:01,  1.33it/s, gpt_loss=0.107, lr=4.54e-6] 
+
Train step of epoch 0:  23%|██▎       | 1296/5616 [16:33<56:49,  1.27it/s, gpt_loss=0.107, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1296/5616 [16:33<56:49,  1.27it/s, gpt_loss=0.0282, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1297/5616 [16:33<59:25,  1.21it/s, gpt_loss=0.0282, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1297/5616 [16:34<59:25,  1.21it/s, gpt_loss=0.105, lr=4.54e-6] 
+
Train step of epoch 0:  23%|██▎       | 1298/5616 [16:34<1:00:50,  1.18it/s, gpt_loss=0.105, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1298/5616 [16:35<1:00:50,  1.18it/s, gpt_loss=0.00886, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1299/5616 [16:35<1:01:01,  1.18it/s, gpt_loss=0.00886, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1299/5616 [16:36<1:01:01,  1.18it/s, gpt_loss=0.0238, lr=4.54e-6] 
+
Train step of epoch 0:  23%|██▎       | 1300/5616 [16:36<58:19,  1.23it/s, gpt_loss=0.0238, lr=4.54e-6]  
+
Train step of epoch 0:  23%|██▎       | 1300/5616 [16:37<58:19,  1.23it/s, gpt_loss=0.0156, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1301/5616 [16:37<54:51,  1.31it/s, gpt_loss=0.0156, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1301/5616 [16:37<54:51,  1.31it/s, gpt_loss=0.00429, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1302/5616 [16:37<53:08,  1.35it/s, gpt_loss=0.00429, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1302/5616 [16:38<53:08,  1.35it/s, gpt_loss=0.0208, lr=4.54e-6] 
+
Train step of epoch 0:  23%|██▎       | 1303/5616 [16:38<53:08,  1.35it/s, gpt_loss=0.0208, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1303/5616 [16:39<53:08,  1.35it/s, gpt_loss=0.0776, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1304/5616 [16:39<52:47,  1.36it/s, gpt_loss=0.0776, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1304/5616 [16:40<52:47,  1.36it/s, gpt_loss=0.142, lr=4.54e-6] 
+
Train step of epoch 0:  23%|██▎       | 1305/5616 [16:40<53:23,  1.35it/s, gpt_loss=0.142, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1305/5616 [16:40<53:23,  1.35it/s, gpt_loss=0.0197, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1306/5616 [16:40<53:29,  1.34it/s, gpt_loss=0.0197, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1306/5616 [16:41<53:29,  1.34it/s, gpt_loss=0.0267, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1307/5616 [16:41<52:12,  1.38it/s, gpt_loss=0.0267, lr=4.54e-6]
+
Train step of epoch 0:  23%|██▎       | 1307/5616 [16:42<52:12,  1.38it/s, gpt_loss=0.0853, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1308/5616 [16:42<53:38,  1.34it/s, gpt_loss=0.0853, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1308/5616 [16:43<53:38,  1.34it/s, gpt_loss=0.0431, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1309/5616 [16:43<54:52,  1.31it/s, gpt_loss=0.0431, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1309/5616 [16:43<54:52,  1.31it/s, gpt_loss=0.0146, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1310/5616 [16:43<53:37,  1.34it/s, gpt_loss=0.0146, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1310/5616 [16:44<53:37,  1.34it/s, gpt_loss=0.0212, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1311/5616 [16:44<51:19,  1.40it/s, gpt_loss=0.0212, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1311/5616 [16:45<51:19,  1.40it/s, gpt_loss=0.0112, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1312/5616 [16:45<55:01,  1.30it/s, gpt_loss=0.0112, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1312/5616 [16:45<55:01,  1.30it/s, gpt_loss=0.103, lr=4.53e-6] 
+
Train step of epoch 0:  23%|██▎       | 1313/5616 [16:45<53:22,  1.34it/s, gpt_loss=0.103, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1313/5616 [16:46<53:22,  1.34it/s, gpt_loss=0.154, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1314/5616 [16:46<52:52,  1.36it/s, gpt_loss=0.154, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1314/5616 [16:47<52:52,  1.36it/s, gpt_loss=0.0197, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1315/5616 [16:47<51:29,  1.39it/s, gpt_loss=0.0197, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1315/5616 [16:48<51:29,  1.39it/s, gpt_loss=0.0486, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1316/5616 [16:48<54:28,  1.32it/s, gpt_loss=0.0486, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1316/5616 [16:48<54:28,  1.32it/s, gpt_loss=0.0595, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1317/5616 [16:48<52:54,  1.35it/s, gpt_loss=0.0595, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1317/5616 [16:49<52:54,  1.35it/s, gpt_loss=0.00822, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1318/5616 [16:49<53:48,  1.33it/s, gpt_loss=0.00822, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1318/5616 [16:50<53:48,  1.33it/s, gpt_loss=0.00991, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1319/5616 [16:50<52:08,  1.37it/s, gpt_loss=0.00991, lr=4.53e-6]
+
Train step of epoch 0:  23%|██▎       | 1319/5616 [16:51<52:08,  1.37it/s, gpt_loss=0.0274, lr=4.52e-6] 
+
Train step of epoch 0:  24%|██▎       | 1320/5616 [16:51<52:47,  1.36it/s, gpt_loss=0.0274, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1320/5616 [16:51<52:47,  1.36it/s, gpt_loss=0.00919, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1321/5616 [16:51<50:40,  1.41it/s, gpt_loss=0.00919, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1321/5616 [16:52<50:40,  1.41it/s, gpt_loss=0.0634, lr=4.52e-6] 
+
Train step of epoch 0:  24%|██▎       | 1322/5616 [16:52<51:12,  1.40it/s, gpt_loss=0.0634, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1322/5616 [16:53<51:12,  1.40it/s, gpt_loss=0.0282, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1323/5616 [16:53<50:37,  1.41it/s, gpt_loss=0.0282, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1323/5616 [16:53<50:37,  1.41it/s, gpt_loss=0.022, lr=4.52e-6] 
+
Train step of epoch 0:  24%|██▎       | 1324/5616 [16:53<52:06,  1.37it/s, gpt_loss=0.022, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1324/5616 [16:54<52:06,  1.37it/s, gpt_loss=0.0177, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1325/5616 [16:54<51:05,  1.40it/s, gpt_loss=0.0177, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1325/5616 [16:55<51:05,  1.40it/s, gpt_loss=0.169, lr=4.52e-6] 
+
Train step of epoch 0:  24%|██▎       | 1326/5616 [16:55<54:39,  1.31it/s, gpt_loss=0.169, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1326/5616 [16:56<54:39,  1.31it/s, gpt_loss=0.0175, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1327/5616 [16:56<54:50,  1.30it/s, gpt_loss=0.0175, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1327/5616 [16:57<54:50,  1.30it/s, gpt_loss=0.0258, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1328/5616 [16:57<56:01,  1.28it/s, gpt_loss=0.0258, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1328/5616 [16:57<56:01,  1.28it/s, gpt_loss=0.0176, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1329/5616 [16:57<55:25,  1.29it/s, gpt_loss=0.0176, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1329/5616 [16:58<55:25,  1.29it/s, gpt_loss=0.0576, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1330/5616 [16:58<56:58,  1.25it/s, gpt_loss=0.0576, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1330/5616 [16:59<56:58,  1.25it/s, gpt_loss=0.121, lr=4.52e-6] 
+
Train step of epoch 0:  24%|██▎       | 1331/5616 [16:59<54:41,  1.31it/s, gpt_loss=0.121, lr=4.52e-6]
+
Train step of epoch 0:  24%|██▎       | 1331/5616 [17:00<54:41,  1.31it/s, gpt_loss=0.0801, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▎       | 1332/5616 [17:00<1:04:06,  1.11it/s, gpt_loss=0.0801, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▎       | 1332/5616 [17:01<1:04:06,  1.11it/s, gpt_loss=0.102, lr=4.51e-6] 
+
Train step of epoch 0:  24%|██▎       | 1333/5616 [17:01<1:00:07,  1.19it/s, gpt_loss=0.102, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▎       | 1333/5616 [17:02<1:00:07,  1.19it/s, gpt_loss=0.00836, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1334/5616 [17:02<57:25,  1.24it/s, gpt_loss=0.00836, lr=4.51e-6]  
+
Train step of epoch 0:  24%|██▍       | 1334/5616 [17:02<57:25,  1.24it/s, gpt_loss=0.0448, lr=4.51e-6] 
+
Train step of epoch 0:  24%|██▍       | 1335/5616 [17:02<56:49,  1.26it/s, gpt_loss=0.0448, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1335/5616 [17:03<56:49,  1.26it/s, gpt_loss=0.0445, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1336/5616 [17:03<56:50,  1.25it/s, gpt_loss=0.0445, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1336/5616 [17:04<56:50,  1.25it/s, gpt_loss=0.0411, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1337/5616 [17:04<1:01:07,  1.17it/s, gpt_loss=0.0411, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1337/5616 [17:05<1:01:07,  1.17it/s, gpt_loss=0.111, lr=4.51e-6] 
+
Train step of epoch 0:  24%|██▍       | 1338/5616 [17:05<1:00:30,  1.18it/s, gpt_loss=0.111, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1338/5616 [17:06<1:00:30,  1.18it/s, gpt_loss=0.0108, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1339/5616 [17:06<58:46,  1.21it/s, gpt_loss=0.0108, lr=4.51e-6]  
+
Train step of epoch 0:  24%|██▍       | 1339/5616 [17:06<58:46,  1.21it/s, gpt_loss=0.074, lr=4.51e-6] 
+
Train step of epoch 0:  24%|██▍       | 1340/5616 [17:06<57:46,  1.23it/s, gpt_loss=0.074, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1340/5616 [17:07<57:46,  1.23it/s, gpt_loss=0.039, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1341/5616 [17:07<55:48,  1.28it/s, gpt_loss=0.039, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1341/5616 [17:08<55:48,  1.28it/s, gpt_loss=0.0637, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1342/5616 [17:08<54:41,  1.30it/s, gpt_loss=0.0637, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1342/5616 [17:09<54:41,  1.30it/s, gpt_loss=0.0328, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1343/5616 [17:09<52:38,  1.35it/s, gpt_loss=0.0328, lr=4.51e-6]
+
Train step of epoch 0:  24%|██▍       | 1343/5616 [17:09<52:38,  1.35it/s, gpt_loss=0.0121, lr=4.5e-6] 
+
Train step of epoch 0:  24%|██▍       | 1344/5616 [17:09<52:18,  1.36it/s, gpt_loss=0.0121, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1344/5616 [17:10<52:18,  1.36it/s, gpt_loss=0.0365, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1345/5616 [17:10<55:57,  1.27it/s, gpt_loss=0.0365, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1345/5616 [17:11<55:57,  1.27it/s, gpt_loss=0.0159, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1346/5616 [17:11<54:19,  1.31it/s, gpt_loss=0.0159, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1346/5616 [17:12<54:19,  1.31it/s, gpt_loss=0.112, lr=4.5e-6] 
+
Train step of epoch 0:  24%|██▍       | 1347/5616 [17:12<53:39,  1.33it/s, gpt_loss=0.112, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1347/5616 [17:13<53:39,  1.33it/s, gpt_loss=0.0643, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1348/5616 [17:13<56:35,  1.26it/s, gpt_loss=0.0643, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1348/5616 [17:13<56:35,  1.26it/s, gpt_loss=0.0464, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1349/5616 [17:13<59:00,  1.21it/s, gpt_loss=0.0464, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1349/5616 [17:14<59:00,  1.21it/s, gpt_loss=0.0353, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1350/5616 [17:14<55:48,  1.27it/s, gpt_loss=0.0353, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1350/5616 [17:15<55:48,  1.27it/s, gpt_loss=0.0403, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1351/5616 [17:15<53:42,  1.32it/s, gpt_loss=0.0403, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1351/5616 [17:16<53:42,  1.32it/s, gpt_loss=0.13, lr=4.5e-6]  
+
Train step of epoch 0:  24%|██▍       | 1352/5616 [17:16<54:02,  1.32it/s, gpt_loss=0.13, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1352/5616 [17:16<54:02,  1.32it/s, gpt_loss=0.0111, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1353/5616 [17:16<53:39,  1.32it/s, gpt_loss=0.0111, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1353/5616 [17:17<53:39,  1.32it/s, gpt_loss=0.0423, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1354/5616 [17:17<54:15,  1.31it/s, gpt_loss=0.0423, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1354/5616 [17:18<54:15,  1.31it/s, gpt_loss=0.0486, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1355/5616 [17:18<52:12,  1.36it/s, gpt_loss=0.0486, lr=4.5e-6]
+
Train step of epoch 0:  24%|██▍       | 1355/5616 [17:19<52:12,  1.36it/s, gpt_loss=0.0599, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1356/5616 [17:19<55:42,  1.27it/s, gpt_loss=0.0599, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1356/5616 [17:19<55:42,  1.27it/s, gpt_loss=0.022, lr=4.49e-6] 
+
Train step of epoch 0:  24%|██▍       | 1357/5616 [17:19<53:17,  1.33it/s, gpt_loss=0.022, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1357/5616 [17:20<53:17,  1.33it/s, gpt_loss=0.0447, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1358/5616 [17:20<51:14,  1.38it/s, gpt_loss=0.0447, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1358/5616 [17:21<51:14,  1.38it/s, gpt_loss=0.111, lr=4.49e-6] 
+
Train step of epoch 0:  24%|██▍       | 1359/5616 [17:21<53:47,  1.32it/s, gpt_loss=0.111, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1359/5616 [17:22<53:47,  1.32it/s, gpt_loss=0.0226, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1360/5616 [17:22<53:50,  1.32it/s, gpt_loss=0.0226, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1360/5616 [17:22<53:50,  1.32it/s, gpt_loss=0.0837, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1361/5616 [17:22<52:42,  1.35it/s, gpt_loss=0.0837, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1361/5616 [17:23<52:42,  1.35it/s, gpt_loss=0.0363, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1362/5616 [17:23<52:51,  1.34it/s, gpt_loss=0.0363, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1362/5616 [17:24<52:51,  1.34it/s, gpt_loss=0.0287, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1363/5616 [17:24<52:37,  1.35it/s, gpt_loss=0.0287, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1363/5616 [17:25<52:37,  1.35it/s, gpt_loss=0.0934, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1364/5616 [17:25<52:24,  1.35it/s, gpt_loss=0.0934, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1364/5616 [17:25<52:24,  1.35it/s, gpt_loss=0.0188, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1365/5616 [17:25<50:07,  1.41it/s, gpt_loss=0.0188, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1365/5616 [17:26<50:07,  1.41it/s, gpt_loss=0.0523, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1366/5616 [17:26<50:20,  1.41it/s, gpt_loss=0.0523, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1366/5616 [17:27<50:20,  1.41it/s, gpt_loss=0.0835, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1367/5616 [17:27<50:37,  1.40it/s, gpt_loss=0.0835, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1367/5616 [17:28<50:37,  1.40it/s, gpt_loss=0.0443, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1368/5616 [17:28<54:30,  1.30it/s, gpt_loss=0.0443, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1368/5616 [17:28<54:30,  1.30it/s, gpt_loss=0.00663, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1369/5616 [17:28<52:13,  1.36it/s, gpt_loss=0.00663, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1369/5616 [17:29<52:13,  1.36it/s, gpt_loss=0.0638, lr=4.49e-6] 
+
Train step of epoch 0:  24%|██▍       | 1370/5616 [17:29<51:01,  1.39it/s, gpt_loss=0.0638, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1370/5616 [17:30<51:01,  1.39it/s, gpt_loss=0.00728, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1371/5616 [17:30<50:02,  1.41it/s, gpt_loss=0.00728, lr=4.49e-6]
+
Train step of epoch 0:  24%|██▍       | 1371/5616 [17:30<50:02,  1.41it/s, gpt_loss=0.0387, lr=4.48e-6] 
+
Train step of epoch 0:  24%|██▍       | 1372/5616 [17:30<53:25,  1.32it/s, gpt_loss=0.0387, lr=4.48e-6]
+
Train step of epoch 0:  24%|██▍       | 1372/5616 [17:31<53:25,  1.32it/s, gpt_loss=0.0383, lr=4.48e-6]
+
Train step of epoch 0:  24%|██▍       | 1373/5616 [17:31<53:10,  1.33it/s, gpt_loss=0.0383, lr=4.48e-6]
+
Train step of epoch 0:  24%|██▍       | 1373/5616 [17:32<53:10,  1.33it/s, gpt_loss=0.082, lr=4.48e-6] 
+
Train step of epoch 0:  24%|██▍       | 1374/5616 [17:32<53:25,  1.32it/s, gpt_loss=0.082, lr=4.48e-6]
+
Train step of epoch 0:  24%|██▍       | 1374/5616 [17:33<53:25,  1.32it/s, gpt_loss=0.169, lr=4.48e-6]
+
Train step of epoch 0:  24%|██▍       | 1375/5616 [17:33<53:44,  1.32it/s, gpt_loss=0.169, lr=4.48e-6]
+
Train step of epoch 0:  24%|██▍       | 1375/5616 [17:33<53:44,  1.32it/s, gpt_loss=0.0195, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1376/5616 [17:33<53:30,  1.32it/s, gpt_loss=0.0195, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1376/5616 [17:34<53:30,  1.32it/s, gpt_loss=0.0521, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1377/5616 [17:34<52:09,  1.35it/s, gpt_loss=0.0521, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1377/5616 [17:35<52:09,  1.35it/s, gpt_loss=0.0132, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1378/5616 [17:35<53:58,  1.31it/s, gpt_loss=0.0132, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1378/5616 [17:36<53:58,  1.31it/s, gpt_loss=0.014, lr=4.48e-6] 
+
Train step of epoch 0:  25%|██▍       | 1379/5616 [17:36<52:27,  1.35it/s, gpt_loss=0.014, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1379/5616 [17:37<52:27,  1.35it/s, gpt_loss=0.0351, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1380/5616 [17:37<55:00,  1.28it/s, gpt_loss=0.0351, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1380/5616 [17:37<55:00,  1.28it/s, gpt_loss=0.0192, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1381/5616 [17:37<52:09,  1.35it/s, gpt_loss=0.0192, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1381/5616 [17:38<52:09,  1.35it/s, gpt_loss=0.0459, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1382/5616 [17:38<50:11,  1.41it/s, gpt_loss=0.0459, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1382/5616 [17:39<50:11,  1.41it/s, gpt_loss=0.0104, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1383/5616 [17:39<50:02,  1.41it/s, gpt_loss=0.0104, lr=4.48e-6]
+
Train step of epoch 0:  25%|██▍       | 1383/5616 [17:39<50:02,  1.41it/s, gpt_loss=0.0272, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1384/5616 [17:39<53:25,  1.32it/s, gpt_loss=0.0272, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1384/5616 [17:40<53:25,  1.32it/s, gpt_loss=0.0579, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1385/5616 [17:40<52:45,  1.34it/s, gpt_loss=0.0579, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1385/5616 [17:41<52:45,  1.34it/s, gpt_loss=0.0337, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1386/5616 [17:41<53:44,  1.31it/s, gpt_loss=0.0337, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1386/5616 [17:42<53:44,  1.31it/s, gpt_loss=0.144, lr=4.47e-6] 
+
Train step of epoch 0:  25%|██▍       | 1387/5616 [17:42<51:39,  1.36it/s, gpt_loss=0.144, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1387/5616 [17:42<51:39,  1.36it/s, gpt_loss=0.0211, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1388/5616 [17:42<53:17,  1.32it/s, gpt_loss=0.0211, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1388/5616 [17:43<53:17,  1.32it/s, gpt_loss=0.154, lr=4.47e-6] 
+
Train step of epoch 0:  25%|██▍       | 1389/5616 [17:43<52:05,  1.35it/s, gpt_loss=0.154, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1389/5616 [17:44<52:05,  1.35it/s, gpt_loss=0.0969, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1390/5616 [17:44<51:13,  1.38it/s, gpt_loss=0.0969, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1390/5616 [17:45<51:13,  1.38it/s, gpt_loss=0.0242, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1391/5616 [17:45<50:43,  1.39it/s, gpt_loss=0.0242, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1391/5616 [17:45<50:43,  1.39it/s, gpt_loss=0.0678, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1392/5616 [17:45<51:56,  1.36it/s, gpt_loss=0.0678, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1392/5616 [17:46<51:56,  1.36it/s, gpt_loss=0.0733, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1393/5616 [17:46<51:10,  1.38it/s, gpt_loss=0.0733, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1393/5616 [17:47<51:10,  1.38it/s, gpt_loss=0.0232, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1394/5616 [17:47<50:35,  1.39it/s, gpt_loss=0.0232, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1394/5616 [17:48<50:35,  1.39it/s, gpt_loss=0.0518, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1395/5616 [17:48<59:35,  1.18it/s, gpt_loss=0.0518, lr=4.47e-6]
+
Train step of epoch 0:  25%|██▍       | 1395/5616 [17:49<59:35,  1.18it/s, gpt_loss=0.0123, lr=4.46e-6]
+
Train step of epoch 0:  25%|██▍       | 1396/5616 [17:49<1:00:08,  1.17it/s, gpt_loss=0.0123, lr=4.46e-6]
+
Train step of epoch 0:  25%|██▍       | 1396/5616 [17:49<1:00:08,  1.17it/s, gpt_loss=0.0446, lr=4.46e-6]
+
Train step of epoch 0:  25%|██▍       | 1397/5616 [17:49<55:31,  1.27it/s, gpt_loss=0.0446, lr=4.46e-6]  
+
Train step of epoch 0:  25%|██▍       | 1397/5616 [17:50<55:31,  1.27it/s, gpt_loss=0.00782, lr=4.46e-6]
+
Train step of epoch 0:  25%|██▍       | 1398/5616 [17:50<53:26,  1.32it/s, gpt_loss=0.00782, lr=4.46e-6]
+
Train step of epoch 0:  25%|██▍       | 1398/5616 [17:51<53:26,  1.32it/s, gpt_loss=0.0232, lr=4.46e-6] 
+
Train step of epoch 0:  25%|██▍       | 1399/5616 [17:51<51:48,  1.36it/s, gpt_loss=0.0232, lr=4.46e-6]
+
Train step of epoch 0:  25%|██▍       | 1399/5616 [17:52<51:48,  1.36it/s, gpt_loss=0.00994, lr=4.46e-6]
+
Train step of epoch 0:  25%|██▍       | 1400/5616 [17:52<52:35,  1.34it/s, gpt_loss=0.00994, lr=4.46e-6]
+
Train step of epoch 0:  25%|██▍       | 1400/5616 [17:52<52:35,  1.34it/s, gpt_loss=0.143, lr=4.46e-6]  
+
Train step of epoch 0:  25%|██▍       | 1401/5616 [17:52<53:49,  1.31it/s, gpt_loss=0.143, lr=4.46e-6]
+
Train step of epoch 0:  25%|██▍       | 1401/5616 [17:53<53:49,  1.31it/s, gpt_loss=0.0703, lr=4.46e-6]
+
Train step of epoch 0:  25%|██▍       | 1402/5616 [17:53<52:51,  1.33it/s, gpt_loss=0.0703, lr=4.46e-6]
+
Train step of epoch 0:  25%|██▍       | 1402/5616 [17:54<52:51,  1.33it/s, gpt_loss=0.0545, lr=4.46e-6]
+
Train step of epoch 0:  25%|██▍       | 1403/5616 [17:54<51:20,  1.37it/s, gpt_loss=0.0545, lr=4.46e-6]
+
Train step of epoch 0:  25%|██▍       | 1403/5616 [17:54<51:20,  1.37it/s, gpt_loss=0.0101, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1404/5616 [17:54<52:12,  1.34it/s, gpt_loss=0.0101, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1404/5616 [17:55<52:12,  1.34it/s, gpt_loss=0.0723, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1405/5616 [17:55<51:19,  1.37it/s, gpt_loss=0.0723, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1405/5616 [17:56<51:19,  1.37it/s, gpt_loss=0.103, lr=4.45e-6] 
+
Train step of epoch 0:  25%|██▌       | 1406/5616 [17:56<51:46,  1.36it/s, gpt_loss=0.103, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1406/5616 [17:57<51:46,  1.36it/s, gpt_loss=0.00224, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1407/5616 [17:57<53:36,  1.31it/s, gpt_loss=0.00224, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1407/5616 [17:58<53:36,  1.31it/s, gpt_loss=0.0173, lr=4.45e-6] 
+
Train step of epoch 0:  25%|██▌       | 1408/5616 [17:58<53:20,  1.31it/s, gpt_loss=0.0173, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1408/5616 [17:58<53:20,  1.31it/s, gpt_loss=0.038, lr=4.45e-6] 
+
Train step of epoch 0:  25%|██▌       | 1409/5616 [17:58<54:17,  1.29it/s, gpt_loss=0.038, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1409/5616 [17:59<54:17,  1.29it/s, gpt_loss=0.00897, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1410/5616 [17:59<52:26,  1.34it/s, gpt_loss=0.00897, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1410/5616 [18:00<52:26,  1.34it/s, gpt_loss=0.0135, lr=4.45e-6] 
+
Train step of epoch 0:  25%|██▌       | 1411/5616 [18:00<51:53,  1.35it/s, gpt_loss=0.0135, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1411/5616 [18:01<51:53,  1.35it/s, gpt_loss=0.0733, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1412/5616 [18:01<53:42,  1.30it/s, gpt_loss=0.0733, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1412/5616 [18:01<53:42,  1.30it/s, gpt_loss=0.00324, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1413/5616 [18:01<51:08,  1.37it/s, gpt_loss=0.00324, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1413/5616 [18:02<51:08,  1.37it/s, gpt_loss=0.0272, lr=4.45e-6] 
+
Train step of epoch 0:  25%|██▌       | 1414/5616 [18:02<50:58,  1.37it/s, gpt_loss=0.0272, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1414/5616 [18:03<50:58,  1.37it/s, gpt_loss=0.0109, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1415/5616 [18:03<53:41,  1.30it/s, gpt_loss=0.0109, lr=4.45e-6]
+
Train step of epoch 0:  25%|██▌       | 1415/5616 [18:04<53:41,  1.30it/s, gpt_loss=0.0473, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1416/5616 [18:04<54:46,  1.28it/s, gpt_loss=0.0473, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1416/5616 [18:05<54:46,  1.28it/s, gpt_loss=0.129, lr=4.44e-6] 
+
Train step of epoch 0:  25%|██▌       | 1417/5616 [18:05<57:46,  1.21it/s, gpt_loss=0.129, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1417/5616 [18:05<57:46,  1.21it/s, gpt_loss=0.212, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1418/5616 [18:05<57:12,  1.22it/s, gpt_loss=0.212, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1418/5616 [18:06<57:12,  1.22it/s, gpt_loss=0.0361, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1419/5616 [18:06<56:37,  1.24it/s, gpt_loss=0.0361, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1419/5616 [18:07<56:37,  1.24it/s, gpt_loss=0.109, lr=4.44e-6] 
+
Train step of epoch 0:  25%|██▌       | 1420/5616 [18:07<55:33,  1.26it/s, gpt_loss=0.109, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1420/5616 [18:08<55:33,  1.26it/s, gpt_loss=0.0209, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1421/5616 [18:08<55:27,  1.26it/s, gpt_loss=0.0209, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1421/5616 [18:08<55:27,  1.26it/s, gpt_loss=0.0142, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1422/5616 [18:08<53:15,  1.31it/s, gpt_loss=0.0142, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1422/5616 [18:09<53:15,  1.31it/s, gpt_loss=0.0403, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1423/5616 [18:09<51:57,  1.34it/s, gpt_loss=0.0403, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1423/5616 [18:10<51:57,  1.34it/s, gpt_loss=0.0251, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1424/5616 [18:10<52:29,  1.33it/s, gpt_loss=0.0251, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1424/5616 [18:11<52:29,  1.33it/s, gpt_loss=0.00888, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1425/5616 [18:11<51:28,  1.36it/s, gpt_loss=0.00888, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1425/5616 [18:11<51:28,  1.36it/s, gpt_loss=0.00829, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1426/5616 [18:11<52:22,  1.33it/s, gpt_loss=0.00829, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1426/5616 [18:12<52:22,  1.33it/s, gpt_loss=0.00348, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1427/5616 [18:12<51:15,  1.36it/s, gpt_loss=0.00348, lr=4.44e-6]
+
Train step of epoch 0:  25%|██▌       | 1427/5616 [18:13<51:15,  1.36it/s, gpt_loss=0.0338, lr=4.43e-6] 
+
Train step of epoch 0:  25%|██▌       | 1428/5616 [18:13<54:53,  1.27it/s, gpt_loss=0.0338, lr=4.43e-6]
+
Train step of epoch 0:  25%|██▌       | 1428/5616 [18:14<54:53,  1.27it/s, gpt_loss=0.0653, lr=4.43e-6]
+
Train step of epoch 0:  25%|██▌       | 1429/5616 [18:14<52:56,  1.32it/s, gpt_loss=0.0653, lr=4.43e-6]
+
Train step of epoch 0:  25%|██▌       | 1429/5616 [18:14<52:56,  1.32it/s, gpt_loss=0.00987, lr=4.43e-6]
+
Train step of epoch 0:  25%|██▌       | 1430/5616 [18:14<51:07,  1.36it/s, gpt_loss=0.00987, lr=4.43e-6]
+
Train step of epoch 0:  25%|██▌       | 1430/5616 [18:15<51:07,  1.36it/s, gpt_loss=0.016, lr=4.43e-6]  
+
Train step of epoch 0:  25%|██▌       | 1431/5616 [18:15<50:04,  1.39it/s, gpt_loss=0.016, lr=4.43e-6]
+
Train step of epoch 0:  25%|██▌       | 1431/5616 [18:16<50:04,  1.39it/s, gpt_loss=0.1, lr=4.43e-6]  
+
Train step of epoch 0:  25%|██▌       | 1432/5616 [18:16<51:27,  1.36it/s, gpt_loss=0.1, lr=4.43e-6]
+
Train step of epoch 0:  25%|██▌       | 1432/5616 [18:17<51:27,  1.36it/s, gpt_loss=0.015, lr=4.43e-6]
+
Train step of epoch 0:  26%|██▌       | 1433/5616 [18:17<53:04,  1.31it/s, gpt_loss=0.015, lr=4.43e-6]
+
Train step of epoch 0:  26%|██▌       | 1433/5616 [18:17<53:04,  1.31it/s, gpt_loss=0.0207, lr=4.43e-6]
+
Train step of epoch 0:  26%|██▌       | 1434/5616 [18:17<51:32,  1.35it/s, gpt_loss=0.0207, lr=4.43e-6]
+
Train step of epoch 0:  26%|██▌       | 1434/5616 [18:18<51:32,  1.35it/s, gpt_loss=0.00465, lr=4.43e-6]
+
Train step of epoch 0:  26%|██▌       | 1435/5616 [18:18<51:40,  1.35it/s, gpt_loss=0.00465, lr=4.43e-6]
+
Train step of epoch 0:  26%|██▌       | 1435/5616 [18:19<51:40,  1.35it/s, gpt_loss=0.0325, lr=4.43e-6] 
+
Train step of epoch 0:  26%|██▌       | 1436/5616 [18:19<52:14,  1.33it/s, gpt_loss=0.0325, lr=4.43e-6]
+
Train step of epoch 0:  26%|██▌       | 1436/5616 [18:20<52:14,  1.33it/s, gpt_loss=0.0154, lr=4.43e-6]
+
Train step of epoch 0:  26%|██▌       | 1437/5616 [18:20<54:12,  1.29it/s, gpt_loss=0.0154, lr=4.43e-6]
+
Train step of epoch 0:  26%|██▌       | 1437/5616 [18:20<54:12,  1.29it/s, gpt_loss=0.0404, lr=4.43e-6]
+
Train step of epoch 0:  26%|██▌       | 1438/5616 [18:20<55:26,  1.26it/s, gpt_loss=0.0404, lr=4.43e-6]
+
Train step of epoch 0:  26%|██▌       | 1438/5616 [18:21<55:26,  1.26it/s, gpt_loss=0.111, lr=4.43e-6] 
+
Train step of epoch 0:  26%|██▌       | 1439/5616 [18:21<53:33,  1.30it/s, gpt_loss=0.111, lr=4.43e-6]
+
Train step of epoch 0:  26%|██▌       | 1439/5616 [18:22<53:33,  1.30it/s, gpt_loss=0.0112, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1440/5616 [18:22<58:58,  1.18it/s, gpt_loss=0.0112, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1440/5616 [18:23<58:58,  1.18it/s, gpt_loss=0.0512, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1441/5616 [18:23<59:13,  1.17it/s, gpt_loss=0.0512, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1441/5616 [18:24<59:13,  1.17it/s, gpt_loss=0.00653, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1442/5616 [18:24<55:03,  1.26it/s, gpt_loss=0.00653, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1442/5616 [18:25<55:03,  1.26it/s, gpt_loss=0.0123, lr=4.42e-6] 
+
Train step of epoch 0:  26%|██▌       | 1443/5616 [18:25<55:35,  1.25it/s, gpt_loss=0.0123, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1443/5616 [18:25<55:35,  1.25it/s, gpt_loss=0.155, lr=4.42e-6] 
+
Train step of epoch 0:  26%|██▌       | 1444/5616 [18:25<54:01,  1.29it/s, gpt_loss=0.155, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1444/5616 [18:26<54:01,  1.29it/s, gpt_loss=0.0156, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1445/5616 [18:26<1:00:48,  1.14it/s, gpt_loss=0.0156, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1445/5616 [18:27<1:00:48,  1.14it/s, gpt_loss=0.255, lr=4.42e-6] 
+
Train step of epoch 0:  26%|██▌       | 1446/5616 [18:27<55:39,  1.25it/s, gpt_loss=0.255, lr=4.42e-6]  
+
Train step of epoch 0:  26%|██▌       | 1446/5616 [18:28<55:39,  1.25it/s, gpt_loss=0.00906, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1447/5616 [18:28<54:36,  1.27it/s, gpt_loss=0.00906, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1447/5616 [18:29<54:36,  1.27it/s, gpt_loss=0.0598, lr=4.42e-6] 
+
Train step of epoch 0:  26%|██▌       | 1448/5616 [18:29<54:57,  1.26it/s, gpt_loss=0.0598, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1448/5616 [18:29<54:57,  1.26it/s, gpt_loss=0.00812, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1449/5616 [18:29<53:56,  1.29it/s, gpt_loss=0.00812, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1449/5616 [18:30<53:56,  1.29it/s, gpt_loss=0.0285, lr=4.42e-6] 
+
Train step of epoch 0:  26%|██▌       | 1450/5616 [18:30<59:24,  1.17it/s, gpt_loss=0.0285, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1450/5616 [18:31<59:24,  1.17it/s, gpt_loss=0.118, lr=4.42e-6] 
+
Train step of epoch 0:  26%|██▌       | 1451/5616 [18:31<57:41,  1.20it/s, gpt_loss=0.118, lr=4.42e-6]
+
Train step of epoch 0:  26%|██▌       | 1451/5616 [18:32<57:41,  1.20it/s, gpt_loss=0.125, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1452/5616 [18:32<55:11,  1.26it/s, gpt_loss=0.125, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1452/5616 [18:33<55:11,  1.26it/s, gpt_loss=0.0449, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1453/5616 [18:33<53:39,  1.29it/s, gpt_loss=0.0449, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1453/5616 [18:33<53:39,  1.29it/s, gpt_loss=0.0644, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1454/5616 [18:33<53:27,  1.30it/s, gpt_loss=0.0644, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1454/5616 [18:34<53:27,  1.30it/s, gpt_loss=0.0253, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1455/5616 [18:34<51:12,  1.35it/s, gpt_loss=0.0253, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1455/5616 [18:35<51:12,  1.35it/s, gpt_loss=0.0594, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1456/5616 [18:35<50:31,  1.37it/s, gpt_loss=0.0594, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1456/5616 [18:35<50:31,  1.37it/s, gpt_loss=0.0842, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1457/5616 [18:35<51:29,  1.35it/s, gpt_loss=0.0842, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1457/5616 [18:36<51:29,  1.35it/s, gpt_loss=0.0233, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1458/5616 [18:36<49:59,  1.39it/s, gpt_loss=0.0233, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1458/5616 [18:37<49:59,  1.39it/s, gpt_loss=0.0132, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1459/5616 [18:37<50:09,  1.38it/s, gpt_loss=0.0132, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1459/5616 [18:38<50:09,  1.38it/s, gpt_loss=0.0844, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1460/5616 [18:38<51:16,  1.35it/s, gpt_loss=0.0844, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1460/5616 [18:38<51:16,  1.35it/s, gpt_loss=0.0233, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1461/5616 [18:38<50:09,  1.38it/s, gpt_loss=0.0233, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1461/5616 [18:39<50:09,  1.38it/s, gpt_loss=0.00773, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1462/5616 [18:39<50:08,  1.38it/s, gpt_loss=0.00773, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1462/5616 [18:40<50:08,  1.38it/s, gpt_loss=0.0571, lr=4.41e-6] 
+
Train step of epoch 0:  26%|██▌       | 1463/5616 [18:40<50:10,  1.38it/s, gpt_loss=0.0571, lr=4.41e-6]
+
Train step of epoch 0:  26%|██▌       | 1463/5616 [18:40<50:10,  1.38it/s, gpt_loss=0.00978, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1464/5616 [18:40<50:05,  1.38it/s, gpt_loss=0.00978, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1464/5616 [18:41<50:05,  1.38it/s, gpt_loss=0.085, lr=4.4e-6]  
+
Train step of epoch 0:  26%|██▌       | 1465/5616 [18:41<48:38,  1.42it/s, gpt_loss=0.085, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1465/5616 [18:42<48:38,  1.42it/s, gpt_loss=0.0779, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1466/5616 [18:42<47:56,  1.44it/s, gpt_loss=0.0779, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1466/5616 [18:43<47:56,  1.44it/s, gpt_loss=0.0543, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1467/5616 [18:43<48:39,  1.42it/s, gpt_loss=0.0543, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1467/5616 [18:43<48:39,  1.42it/s, gpt_loss=0.00819, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1468/5616 [18:43<50:29,  1.37it/s, gpt_loss=0.00819, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1468/5616 [18:44<50:29,  1.37it/s, gpt_loss=0.0202, lr=4.4e-6] 
+
Train step of epoch 0:  26%|██▌       | 1469/5616 [18:44<49:19,  1.40it/s, gpt_loss=0.0202, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1469/5616 [18:45<49:19,  1.40it/s, gpt_loss=0.0293, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1470/5616 [18:45<49:32,  1.39it/s, gpt_loss=0.0293, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1470/5616 [18:45<49:32,  1.39it/s, gpt_loss=0.05, lr=4.4e-6]  
+
Train step of epoch 0:  26%|██▌       | 1471/5616 [18:45<50:02,  1.38it/s, gpt_loss=0.05, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1471/5616 [18:46<50:02,  1.38it/s, gpt_loss=0.00528, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1472/5616 [18:46<51:06,  1.35it/s, gpt_loss=0.00528, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1472/5616 [18:47<51:06,  1.35it/s, gpt_loss=0.0302, lr=4.4e-6] 
+
Train step of epoch 0:  26%|██▌       | 1473/5616 [18:47<49:40,  1.39it/s, gpt_loss=0.0302, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1473/5616 [18:48<49:40,  1.39it/s, gpt_loss=0.0682, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1474/5616 [18:48<53:48,  1.28it/s, gpt_loss=0.0682, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▌       | 1474/5616 [18:49<53:48,  1.28it/s, gpt_loss=0.0584, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▋       | 1475/5616 [18:49<52:30,  1.31it/s, gpt_loss=0.0584, lr=4.4e-6]
+
Train step of epoch 0:  26%|██▋       | 1475/5616 [18:49<52:30,  1.31it/s, gpt_loss=0.0456, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1476/5616 [18:49<51:51,  1.33it/s, gpt_loss=0.0456, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1476/5616 [18:50<51:51,  1.33it/s, gpt_loss=0.0174, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1477/5616 [18:50<50:46,  1.36it/s, gpt_loss=0.0174, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1477/5616 [18:51<50:46,  1.36it/s, gpt_loss=0.0392, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1478/5616 [18:51<50:37,  1.36it/s, gpt_loss=0.0392, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1478/5616 [18:51<50:37,  1.36it/s, gpt_loss=0.0496, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1479/5616 [18:51<48:41,  1.42it/s, gpt_loss=0.0496, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1479/5616 [18:52<48:41,  1.42it/s, gpt_loss=0.134, lr=4.39e-6] 
+
Train step of epoch 0:  26%|██▋       | 1480/5616 [18:52<49:58,  1.38it/s, gpt_loss=0.134, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1480/5616 [18:53<49:58,  1.38it/s, gpt_loss=0.0165, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1481/5616 [18:53<48:50,  1.41it/s, gpt_loss=0.0165, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1481/5616 [18:53<48:50,  1.41it/s, gpt_loss=0.103, lr=4.39e-6] 
+
Train step of epoch 0:  26%|██▋       | 1482/5616 [18:53<48:44,  1.41it/s, gpt_loss=0.103, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1482/5616 [18:54<48:44,  1.41it/s, gpt_loss=0.0839, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1483/5616 [18:54<48:19,  1.43it/s, gpt_loss=0.0839, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1483/5616 [18:55<48:19,  1.43it/s, gpt_loss=0.0885, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1484/5616 [18:55<49:25,  1.39it/s, gpt_loss=0.0885, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1484/5616 [18:56<49:25,  1.39it/s, gpt_loss=0.004, lr=4.39e-6] 
+
Train step of epoch 0:  26%|██▋       | 1485/5616 [18:56<48:39,  1.41it/s, gpt_loss=0.004, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1485/5616 [18:56<48:39,  1.41it/s, gpt_loss=0.0571, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1486/5616 [18:56<47:54,  1.44it/s, gpt_loss=0.0571, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1486/5616 [18:57<47:54,  1.44it/s, gpt_loss=0.0519, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1487/5616 [18:57<48:25,  1.42it/s, gpt_loss=0.0519, lr=4.39e-6]
+
Train step of epoch 0:  26%|██▋       | 1487/5616 [18:58<48:25,  1.42it/s, gpt_loss=0.0077, lr=4.38e-6]
+
Train step of epoch 0:  26%|██▋       | 1488/5616 [18:58<49:58,  1.38it/s, gpt_loss=0.0077, lr=4.38e-6]
+
Train step of epoch 0:  26%|██▋       | 1488/5616 [18:58<49:58,  1.38it/s, gpt_loss=0.0182, lr=4.38e-6]
+
Train step of epoch 0:  27%|██▋       | 1489/5616 [18:58<48:35,  1.42it/s, gpt_loss=0.0182, lr=4.38e-6]
+
Train step of epoch 0:  27%|██▋       | 1489/5616 [18:59<48:35,  1.42it/s, gpt_loss=0.0236, lr=4.38e-6]
+
Train step of epoch 0:  27%|██▋       | 1490/5616 [18:59<50:18,  1.37it/s, gpt_loss=0.0236, lr=4.38e-6]
+
Train step of epoch 0:  27%|██▋       | 1490/5616 [19:00<50:18,  1.37it/s, gpt_loss=0.0397, lr=4.38e-6]
+
Train step of epoch 0:  27%|██▋       | 1491/5616 [19:00<51:28,  1.34it/s, gpt_loss=0.0397, lr=4.38e-6]
+
Train step of epoch 0:  27%|██▋       | 1491/5616 [19:01<51:28,  1.34it/s, gpt_loss=0.0238, lr=4.38e-6]
+
Train step of epoch 0:  27%|██▋       | 1492/5616 [19:01<53:30,  1.28it/s, gpt_loss=0.0238, lr=4.38e-6]
+
Train step of epoch 0:  27%|██▋       | 1492/5616 [19:02<53:30,  1.28it/s, gpt_loss=0.097, lr=4.38e-6] 
+
Train step of epoch 0:  27%|██▋       | 1493/5616 [19:02<53:12,  1.29it/s, gpt_loss=0.097, lr=4.38e-6]
+
Train step of epoch 0:  27%|██▋       | 1493/5616 [19:02<53:12,  1.29it/s, gpt_loss=0.0271, lr=4.38e-6]
+
Train step of epoch 0:  27%|██▋       | 1494/5616 [19:02<54:08,  1.27it/s, gpt_loss=0.0271, lr=4.38e-6]
+
Train step of epoch 0:  27%|██▋       | 1494/5616 [19:03<54:08,  1.27it/s, gpt_loss=0.0166, lr=4.38e-6]
+
Train step of epoch 0:  27%|██▋       | 1495/5616 [19:03<52:07,  1.32it/s, gpt_loss=0.0166, lr=4.38e-6]
+
Train step of epoch 0:  27%|██▋       | 1495/5616 [19:04<52:07,  1.32it/s, gpt_loss=0.0337, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1496/5616 [19:04<51:25,  1.34it/s, gpt_loss=0.0337, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1496/5616 [19:05<51:25,  1.34it/s, gpt_loss=0.00582, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1497/5616 [19:05<50:33,  1.36it/s, gpt_loss=0.00582, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1497/5616 [19:05<50:33,  1.36it/s, gpt_loss=0.0528, lr=4.37e-6] 
+
Train step of epoch 0:  27%|██▋       | 1498/5616 [19:05<50:33,  1.36it/s, gpt_loss=0.0528, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1498/5616 [19:06<50:33,  1.36it/s, gpt_loss=0.0402, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1499/5616 [19:06<49:32,  1.39it/s, gpt_loss=0.0402, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1499/5616 [19:07<49:32,  1.39it/s, gpt_loss=0.0294, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1500/5616 [19:07<50:54,  1.35it/s, gpt_loss=0.0294, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1500/5616 [19:07<50:54,  1.35it/s, gpt_loss=0.0415, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1501/5616 [19:07<49:40,  1.38it/s, gpt_loss=0.0415, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1501/5616 [19:08<49:40,  1.38it/s, gpt_loss=0.00966, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1502/5616 [19:08<50:42,  1.35it/s, gpt_loss=0.00966, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1502/5616 [19:09<50:42,  1.35it/s, gpt_loss=0.0173, lr=4.37e-6] 
+
Train step of epoch 0:  27%|██▋       | 1503/5616 [19:09<50:31,  1.36it/s, gpt_loss=0.0173, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1503/5616 [19:10<50:31,  1.36it/s, gpt_loss=0.0409, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1504/5616 [19:10<50:36,  1.35it/s, gpt_loss=0.0409, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1504/5616 [19:10<50:36,  1.35it/s, gpt_loss=0.0118, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1505/5616 [19:10<49:49,  1.38it/s, gpt_loss=0.0118, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1505/5616 [19:11<49:49,  1.38it/s, gpt_loss=0.0612, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1506/5616 [19:11<49:06,  1.39it/s, gpt_loss=0.0612, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1506/5616 [19:12<49:06,  1.39it/s, gpt_loss=0.0299, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1507/5616 [19:12<47:18,  1.45it/s, gpt_loss=0.0299, lr=4.37e-6]
+
Train step of epoch 0:  27%|██▋       | 1507/5616 [19:12<47:18,  1.45it/s, gpt_loss=0.0158, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1508/5616 [19:12<47:50,  1.43it/s, gpt_loss=0.0158, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1508/5616 [19:13<47:50,  1.43it/s, gpt_loss=0.102, lr=4.36e-6] 
+
Train step of epoch 0:  27%|██▋       | 1509/5616 [19:13<47:36,  1.44it/s, gpt_loss=0.102, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1509/5616 [19:14<47:36,  1.44it/s, gpt_loss=0.0832, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1510/5616 [19:14<47:17,  1.45it/s, gpt_loss=0.0832, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1510/5616 [19:15<47:17,  1.45it/s, gpt_loss=0.111, lr=4.36e-6] 
+
Train step of epoch 0:  27%|██▋       | 1511/5616 [19:15<46:55,  1.46it/s, gpt_loss=0.111, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1511/5616 [19:15<46:55,  1.46it/s, gpt_loss=0.0503, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1512/5616 [19:15<48:57,  1.40it/s, gpt_loss=0.0503, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1512/5616 [19:16<48:57,  1.40it/s, gpt_loss=0.0337, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1513/5616 [19:16<48:18,  1.42it/s, gpt_loss=0.0337, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1513/5616 [19:17<48:18,  1.42it/s, gpt_loss=0.0461, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1514/5616 [19:17<48:06,  1.42it/s, gpt_loss=0.0461, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1514/5616 [19:17<48:06,  1.42it/s, gpt_loss=0.0393, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1515/5616 [19:17<47:53,  1.43it/s, gpt_loss=0.0393, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1515/5616 [19:18<47:53,  1.43it/s, gpt_loss=0.0222, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1516/5616 [19:18<50:31,  1.35it/s, gpt_loss=0.0222, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1516/5616 [19:19<50:31,  1.35it/s, gpt_loss=0.0535, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1517/5616 [19:19<50:41,  1.35it/s, gpt_loss=0.0535, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1517/5616 [19:20<50:41,  1.35it/s, gpt_loss=0.0813, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1518/5616 [19:20<57:57,  1.18it/s, gpt_loss=0.0813, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1518/5616 [19:21<57:57,  1.18it/s, gpt_loss=0.0154, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1519/5616 [19:21<56:24,  1.21it/s, gpt_loss=0.0154, lr=4.36e-6]
+
Train step of epoch 0:  27%|██▋       | 1519/5616 [19:22<56:24,  1.21it/s, gpt_loss=0.0335, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1520/5616 [19:22<57:52,  1.18it/s, gpt_loss=0.0335, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1520/5616 [19:22<57:52,  1.18it/s, gpt_loss=0.0992, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1521/5616 [19:22<55:55,  1.22it/s, gpt_loss=0.0992, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1521/5616 [19:23<55:55,  1.22it/s, gpt_loss=0.0181, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1522/5616 [19:23<54:18,  1.26it/s, gpt_loss=0.0181, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1522/5616 [19:24<54:18,  1.26it/s, gpt_loss=0.0895, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1523/5616 [19:24<56:57,  1.20it/s, gpt_loss=0.0895, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1523/5616 [19:25<56:57,  1.20it/s, gpt_loss=0.021, lr=4.35e-6] 
+
Train step of epoch 0:  27%|██▋       | 1524/5616 [19:25<57:12,  1.19it/s, gpt_loss=0.021, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1524/5616 [19:26<57:12,  1.19it/s, gpt_loss=0.0587, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1525/5616 [19:26<57:03,  1.19it/s, gpt_loss=0.0587, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1525/5616 [19:27<57:03,  1.19it/s, gpt_loss=0.0513, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1526/5616 [19:27<58:32,  1.16it/s, gpt_loss=0.0513, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1526/5616 [19:27<58:32,  1.16it/s, gpt_loss=0.00532, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1527/5616 [19:27<54:31,  1.25it/s, gpt_loss=0.00532, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1527/5616 [19:28<54:31,  1.25it/s, gpt_loss=0.0374, lr=4.35e-6] 
+
Train step of epoch 0:  27%|██▋       | 1528/5616 [19:28<54:05,  1.26it/s, gpt_loss=0.0374, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1528/5616 [19:29<54:05,  1.26it/s, gpt_loss=0.0316, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1529/5616 [19:29<54:49,  1.24it/s, gpt_loss=0.0316, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1529/5616 [19:30<54:49,  1.24it/s, gpt_loss=0.0508, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1530/5616 [19:30<51:55,  1.31it/s, gpt_loss=0.0508, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1530/5616 [19:30<51:55,  1.31it/s, gpt_loss=0.0145, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1531/5616 [19:30<50:45,  1.34it/s, gpt_loss=0.0145, lr=4.35e-6]
+
Train step of epoch 0:  27%|██▋       | 1531/5616 [19:31<50:45,  1.34it/s, gpt_loss=0.0526, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1532/5616 [19:31<52:38,  1.29it/s, gpt_loss=0.0526, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1532/5616 [19:32<52:38,  1.29it/s, gpt_loss=0.0357, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1533/5616 [19:32<51:01,  1.33it/s, gpt_loss=0.0357, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1533/5616 [19:33<51:01,  1.33it/s, gpt_loss=0.0469, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1534/5616 [19:33<51:19,  1.33it/s, gpt_loss=0.0469, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1534/5616 [19:33<51:19,  1.33it/s, gpt_loss=0.098, lr=4.34e-6] 
+
Train step of epoch 0:  27%|██▋       | 1535/5616 [19:33<49:47,  1.37it/s, gpt_loss=0.098, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1535/5616 [19:34<49:47,  1.37it/s, gpt_loss=0.036, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1536/5616 [19:34<50:44,  1.34it/s, gpt_loss=0.036, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1536/5616 [19:35<50:44,  1.34it/s, gpt_loss=0.116, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1537/5616 [19:35<50:25,  1.35it/s, gpt_loss=0.116, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1537/5616 [19:36<50:25,  1.35it/s, gpt_loss=0.0693, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1538/5616 [19:36<49:38,  1.37it/s, gpt_loss=0.0693, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1538/5616 [19:36<49:38,  1.37it/s, gpt_loss=0.0244, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1539/5616 [19:36<50:56,  1.33it/s, gpt_loss=0.0244, lr=4.34e-6]
+
Train step of epoch 0:  27%|██▋       | 1539/5616 [19:37<50:56,  1.33it/s, gpt_loss=0.0182, lr=4.33e-6]
+
Train step of epoch 0:  27%|██▋       | 1540/5616 [19:37<50:57,  1.33it/s, gpt_loss=0.0182, lr=4.33e-6]
+
Train step of epoch 0:  27%|██▋       | 1540/5616 [19:38<50:57,  1.33it/s, gpt_loss=0.0259, lr=4.33e-6]
+
Train step of epoch 0:  27%|██▋       | 1541/5616 [19:38<50:25,  1.35it/s, gpt_loss=0.0259, lr=4.33e-6]
+
Train step of epoch 0:  27%|██▋       | 1541/5616 [19:38<50:25,  1.35it/s, gpt_loss=0.0115, lr=4.33e-6]
+
Train step of epoch 0:  27%|██▋       | 1542/5616 [19:38<48:50,  1.39it/s, gpt_loss=0.0115, lr=4.33e-6]
+
Train step of epoch 0:  27%|██▋       | 1542/5616 [19:39<48:50,  1.39it/s, gpt_loss=0.028, lr=4.33e-6] 
+
Train step of epoch 0:  27%|██▋       | 1543/5616 [19:39<48:43,  1.39it/s, gpt_loss=0.028, lr=4.33e-6]
+
Train step of epoch 0:  27%|██▋       | 1543/5616 [19:40<48:43,  1.39it/s, gpt_loss=0.015, lr=4.33e-6]
+
Train step of epoch 0:  27%|██▋       | 1544/5616 [19:40<49:57,  1.36it/s, gpt_loss=0.015, lr=4.33e-6]
+
Train step of epoch 0:  27%|██▋       | 1544/5616 [19:41<49:57,  1.36it/s, gpt_loss=0.0259, lr=4.33e-6]
+
Train step of epoch 0:  28%|██��       | 1545/5616 [19:41<48:54,  1.39it/s, gpt_loss=0.0259, lr=4.33e-6]
+
Train step of epoch 0:  28%|██▊       | 1545/5616 [19:41<48:54,  1.39it/s, gpt_loss=0.0576, lr=4.33e-6]
+
Train step of epoch 0:  28%|██▊       | 1546/5616 [19:41<47:35,  1.43it/s, gpt_loss=0.0576, lr=4.33e-6]
+
Train step of epoch 0:  28%|██▊       | 1546/5616 [19:42<47:35,  1.43it/s, gpt_loss=0.0978, lr=4.33e-6]
+
Train step of epoch 0:  28%|██▊       | 1547/5616 [19:42<46:25,  1.46it/s, gpt_loss=0.0978, lr=4.33e-6]
+
Train step of epoch 0:  28%|██▊       | 1547/5616 [19:43<46:25,  1.46it/s, gpt_loss=0.0843, lr=4.33e-6]
+
Train step of epoch 0:  28%|██▊       | 1548/5616 [19:43<48:07,  1.41it/s, gpt_loss=0.0843, lr=4.33e-6]
+
Train step of epoch 0:  28%|██▊       | 1548/5616 [19:43<48:07,  1.41it/s, gpt_loss=0.104, lr=4.33e-6] 
+
Train step of epoch 0:  28%|██▊       | 1549/5616 [19:43<47:51,  1.42it/s, gpt_loss=0.104, lr=4.33e-6]
+
Train step of epoch 0:  28%|██▊       | 1549/5616 [19:44<47:51,  1.42it/s, gpt_loss=0.0486, lr=4.33e-6]
+
Train step of epoch 0:  28%|██▊       | 1550/5616 [19:44<47:04,  1.44it/s, gpt_loss=0.0486, lr=4.33e-6]
+
Train step of epoch 0:  28%|██▊       | 1550/5616 [19:45<47:04,  1.44it/s, gpt_loss=0.0366, lr=4.33e-6]
+
Train step of epoch 0:  28%|██▊       | 1551/5616 [19:45<53:13,  1.27it/s, gpt_loss=0.0366, lr=4.33e-6]
+
Train step of epoch 0:  28%|██▊       | 1551/5616 [19:46<53:13,  1.27it/s, gpt_loss=0.0305, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1552/5616 [19:46<58:42,  1.15it/s, gpt_loss=0.0305, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1552/5616 [19:47<58:42,  1.15it/s, gpt_loss=0.031, lr=4.32e-6] 
+
Train step of epoch 0:  28%|██▊       | 1553/5616 [19:47<56:36,  1.20it/s, gpt_loss=0.031, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1553/5616 [19:48<56:36,  1.20it/s, gpt_loss=0.0446, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1554/5616 [19:48<54:48,  1.24it/s, gpt_loss=0.0446, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1554/5616 [19:48<54:48,  1.24it/s, gpt_loss=0.0518, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1555/5616 [19:48<52:38,  1.29it/s, gpt_loss=0.0518, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1555/5616 [19:49<52:38,  1.29it/s, gpt_loss=0.115, lr=4.32e-6] 
+
Train step of epoch 0:  28%|██▊       | 1556/5616 [19:49<52:57,  1.28it/s, gpt_loss=0.115, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1556/5616 [19:50<52:57,  1.28it/s, gpt_loss=0.0694, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1557/5616 [19:50<53:00,  1.28it/s, gpt_loss=0.0694, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1557/5616 [19:51<53:00,  1.28it/s, gpt_loss=0.0718, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1558/5616 [19:51<58:58,  1.15it/s, gpt_loss=0.0718, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1558/5616 [19:52<58:58,  1.15it/s, gpt_loss=0.0696, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1559/5616 [19:52<54:59,  1.23it/s, gpt_loss=0.0696, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1559/5616 [19:53<54:59,  1.23it/s, gpt_loss=0.0637, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1560/5616 [19:53<54:52,  1.23it/s, gpt_loss=0.0637, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1560/5616 [19:53<54:52,  1.23it/s, gpt_loss=0.0506, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1561/5616 [19:53<51:59,  1.30it/s, gpt_loss=0.0506, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1561/5616 [19:54<51:59,  1.30it/s, gpt_loss=0.0393, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1562/5616 [19:54<50:45,  1.33it/s, gpt_loss=0.0393, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1562/5616 [19:55<50:45,  1.33it/s, gpt_loss=0.0833, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1563/5616 [19:55<59:20,  1.14it/s, gpt_loss=0.0833, lr=4.32e-6]
+
Train step of epoch 0:  28%|██▊       | 1563/5616 [19:56<59:20,  1.14it/s, gpt_loss=0.246, lr=4.31e-6] 
+
Train step of epoch 0:  28%|██▊       | 1564/5616 [19:56<58:42,  1.15it/s, gpt_loss=0.246, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1564/5616 [19:57<58:42,  1.15it/s, gpt_loss=0.101, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1565/5616 [19:57<57:01,  1.18it/s, gpt_loss=0.101, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1565/5616 [19:57<57:01,  1.18it/s, gpt_loss=0.155, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1566/5616 [19:57<53:49,  1.25it/s, gpt_loss=0.155, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1566/5616 [19:58<53:49,  1.25it/s, gpt_loss=0.0141, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1567/5616 [19:58<53:41,  1.26it/s, gpt_loss=0.0141, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1567/5616 [19:59<53:41,  1.26it/s, gpt_loss=0.0273, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1568/5616 [19:59<52:58,  1.27it/s, gpt_loss=0.0273, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1568/5616 [20:00<52:58,  1.27it/s, gpt_loss=0.0178, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1569/5616 [20:00<53:21,  1.26it/s, gpt_loss=0.0178, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1569/5616 [20:00<53:21,  1.26it/s, gpt_loss=0.0262, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1570/5616 [20:00<51:51,  1.30it/s, gpt_loss=0.0262, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1570/5616 [20:01<51:51,  1.30it/s, gpt_loss=0.0358, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1571/5616 [20:01<51:35,  1.31it/s, gpt_loss=0.0358, lr=4.31e-6]
+
Train step of epoch 0:  28%|██▊       | 1571/5616 [20:02<51:35,  1.31it/s, gpt_loss=0.0404, lr=4.3e-6] 
+
Train step of epoch 0:  28%|██▊       | 1572/5616 [20:02<51:11,  1.32it/s, gpt_loss=0.0404, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1572/5616 [20:03<51:11,  1.32it/s, gpt_loss=0.177, lr=4.3e-6] 
+
Train step of epoch 0:  28%|██▊       | 1573/5616 [20:03<51:21,  1.31it/s, gpt_loss=0.177, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1573/5616 [20:04<51:21,  1.31it/s, gpt_loss=0.0598, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1574/5616 [20:04<52:30,  1.28it/s, gpt_loss=0.0598, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1574/5616 [20:04<52:30,  1.28it/s, gpt_loss=0.0937, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1575/5616 [20:04<50:13,  1.34it/s, gpt_loss=0.0937, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1575/5616 [20:05<50:13,  1.34it/s, gpt_loss=0.0406, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1576/5616 [20:05<53:10,  1.27it/s, gpt_loss=0.0406, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1576/5616 [20:06<53:10,  1.27it/s, gpt_loss=0.0101, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1577/5616 [20:06<52:52,  1.27it/s, gpt_loss=0.0101, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1577/5616 [20:07<52:52,  1.27it/s, gpt_loss=0.062, lr=4.3e-6] 
+
Train step of epoch 0:  28%|██▊       | 1578/5616 [20:07<51:36,  1.30it/s, gpt_loss=0.062, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1578/5616 [20:08<51:36,  1.30it/s, gpt_loss=0.0682, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1579/5616 [20:08<53:47,  1.25it/s, gpt_loss=0.0682, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1579/5616 [20:08<53:47,  1.25it/s, gpt_loss=0.015, lr=4.3e-6] 
+
Train step of epoch 0:  28%|██▊       | 1580/5616 [20:08<53:58,  1.25it/s, gpt_loss=0.015, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1580/5616 [20:09<53:58,  1.25it/s, gpt_loss=0.15, lr=4.3e-6] 
+
Train step of epoch 0:  28%|██▊       | 1581/5616 [20:09<54:48,  1.23it/s, gpt_loss=0.15, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1581/5616 [20:10<54:48,  1.23it/s, gpt_loss=0.0268, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1582/5616 [20:10<54:14,  1.24it/s, gpt_loss=0.0268, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1582/5616 [20:11<54:14,  1.24it/s, gpt_loss=0.00559, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1583/5616 [20:11<53:18,  1.26it/s, gpt_loss=0.00559, lr=4.3e-6]
+
Train step of epoch 0:  28%|██▊       | 1583/5616 [20:12<53:18,  1.26it/s, gpt_loss=0.0196, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1584/5616 [20:12<53:31,  1.26it/s, gpt_loss=0.0196, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1584/5616 [20:12<53:31,  1.26it/s, gpt_loss=0.0256, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1585/5616 [20:12<51:04,  1.32it/s, gpt_loss=0.0256, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1585/5616 [20:13<51:04,  1.32it/s, gpt_loss=0.0482, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1586/5616 [20:13<49:28,  1.36it/s, gpt_loss=0.0482, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1586/5616 [20:14<49:28,  1.36it/s, gpt_loss=0.0131, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1587/5616 [20:14<48:25,  1.39it/s, gpt_loss=0.0131, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1587/5616 [20:14<48:25,  1.39it/s, gpt_loss=0.0151, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1588/5616 [20:14<50:00,  1.34it/s, gpt_loss=0.0151, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1588/5616 [20:15<50:00,  1.34it/s, gpt_loss=0.0639, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1589/5616 [20:15<50:33,  1.33it/s, gpt_loss=0.0639, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1589/5616 [20:16<50:33,  1.33it/s, gpt_loss=0.00872, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1590/5616 [20:16<48:32,  1.38it/s, gpt_loss=0.00872, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1590/5616 [20:17<48:32,  1.38it/s, gpt_loss=0.0558, lr=4.29e-6] 
+
Train step of epoch 0:  28%|██▊       | 1591/5616 [20:17<50:40,  1.32it/s, gpt_loss=0.0558, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1591/5616 [20:17<50:40,  1.32it/s, gpt_loss=0.0298, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1592/5616 [20:17<52:52,  1.27it/s, gpt_loss=0.0298, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1592/5616 [20:18<52:52,  1.27it/s, gpt_loss=0.113, lr=4.29e-6] 
+
Train step of epoch 0:  28%|██▊       | 1593/5616 [20:18<53:14,  1.26it/s, gpt_loss=0.113, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1593/5616 [20:19<53:14,  1.26it/s, gpt_loss=0.0595, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1594/5616 [20:19<49:51,  1.34it/s, gpt_loss=0.0595, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1594/5616 [20:20<49:51,  1.34it/s, gpt_loss=0.131, lr=4.29e-6] 
+
Train step of epoch 0:  28%|██▊       | 1595/5616 [20:20<49:52,  1.34it/s, gpt_loss=0.131, lr=4.29e-6]
+
Train step of epoch 0:  28%|██▊       | 1595/5616 [20:21<49:52,  1.34it/s, gpt_loss=0.0599, lr=4.28e-6]
+
Train step of epoch 0:  28%|██▊       | 1596/5616 [20:21<52:22,  1.28it/s, gpt_loss=0.0599, lr=4.28e-6]
+
Train step of epoch 0:  28%|██▊       | 1596/5616 [20:21<52:22,  1.28it/s, gpt_loss=0.0873, lr=4.28e-6]
+
Train step of epoch 0:  28%|██▊       | 1597/5616 [20:21<49:53,  1.34it/s, gpt_loss=0.0873, lr=4.28e-6]
+
Train step of epoch 0:  28%|██▊       | 1597/5616 [20:22<49:53,  1.34it/s, gpt_loss=0.027, lr=4.28e-6] 
+
Train step of epoch 0:  28%|██▊       | 1598/5616 [20:22<49:09,  1.36it/s, gpt_loss=0.027, lr=4.28e-6]
+
Train step of epoch 0:  28%|██▊       | 1598/5616 [20:23<49:09,  1.36it/s, gpt_loss=0.123, lr=4.28e-6]
+
Train step of epoch 0:  28%|██▊       | 1599/5616 [20:23<49:02,  1.37it/s, gpt_loss=0.123, lr=4.28e-6][2025-06-30 06:58:32,552] [INFO] [logging.py:128:log_dist] [Rank 0] step=400, skipped=0, lr=[4.2783025268729185e-06, 4.2783025268729185e-06], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 06:58:32,553] [INFO] [timer.py:264:stop] epoch=0/micro_step=1600/global_step=400, RunningAvgSamplesPerSec=42.43175575955272, CurrSamplesPerSec=43.256889638919006, MemAllocated=15.88GB, MaxMemAllocated=51.79GB
+
+
Train step of epoch 0:  28%|██▊       | 1599/5616 [20:24<49:02,  1.37it/s, gpt_loss=0.0457, lr=4.28e-6]
+
Train step of epoch 0:  28%|██▊       | 1600/5616 [20:24<52:21,  1.28it/s, gpt_loss=0.0457, lr=4.28e-6]
+
Train step of epoch 0:  28%|██▊       | 1600/5616 [20:24<52:21,  1.28it/s, gpt_loss=0.0235, lr=4.28e-6]
+
Train step of epoch 0:  29%|██▊       | 1601/5616 [20:24<55:03,  1.22it/s, gpt_loss=0.0235, lr=4.28e-6]
+
Train step of epoch 0:  29%|██▊       | 1601/5616 [20:25<55:03,  1.22it/s, gpt_loss=0.0227, lr=4.28e-6]
+
Train step of epoch 0:  29%|██▊       | 1602/5616 [20:25<53:25,  1.25it/s, gpt_loss=0.0227, lr=4.28e-6]
+
Train step of epoch 0:  29%|██▊       | 1602/5616 [20:26<53:25,  1.25it/s, gpt_loss=0.0106, lr=4.28e-6]
+
Train step of epoch 0:  29%|██▊       | 1603/5616 [20:26<52:53,  1.26it/s, gpt_loss=0.0106, lr=4.28e-6]
+
Train step of epoch 0:  29%|██▊       | 1603/5616 [20:27<52:53,  1.26it/s, gpt_loss=0.0709, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1604/5616 [20:27<52:38,  1.27it/s, gpt_loss=0.0709, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1604/5616 [20:27<52:38,  1.27it/s, gpt_loss=0.0405, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1605/5616 [20:27<50:06,  1.33it/s, gpt_loss=0.0405, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1605/5616 [20:28<50:06,  1.33it/s, gpt_loss=0.0244, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1606/5616 [20:28<50:00,  1.34it/s, gpt_loss=0.0244, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1606/5616 [20:29<50:00,  1.34it/s, gpt_loss=0.0889, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1607/5616 [20:29<49:01,  1.36it/s, gpt_loss=0.0889, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1607/5616 [20:30<49:01,  1.36it/s, gpt_loss=0.0548, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1608/5616 [20:30<50:47,  1.32it/s, gpt_loss=0.0548, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1608/5616 [20:30<50:47,  1.32it/s, gpt_loss=0.0306, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1609/5616 [20:30<49:14,  1.36it/s, gpt_loss=0.0306, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1609/5616 [20:31<49:14,  1.36it/s, gpt_loss=0.0402, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1610/5616 [20:31<48:22,  1.38it/s, gpt_loss=0.0402, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1610/5616 [20:32<48:22,  1.38it/s, gpt_loss=0.0301, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1611/5616 [20:32<48:20,  1.38it/s, gpt_loss=0.0301, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1611/5616 [20:32<48:20,  1.38it/s, gpt_loss=0.0599, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1612/5616 [20:32<47:48,  1.40it/s, gpt_loss=0.0599, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1612/5616 [20:33<47:48,  1.40it/s, gpt_loss=0.00711, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1613/5616 [20:33<48:24,  1.38it/s, gpt_loss=0.00711, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1613/5616 [20:34<48:24,  1.38it/s, gpt_loss=0.0489, lr=4.27e-6] 
+
Train step of epoch 0:  29%|██▊       | 1614/5616 [20:34<49:55,  1.34it/s, gpt_loss=0.0489, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▊       | 1614/5616 [20:35<49:55,  1.34it/s, gpt_loss=0.03, lr=4.27e-6]  
+
Train step of epoch 0:  29%|██▉       | 1615/5616 [20:35<53:16,  1.25it/s, gpt_loss=0.03, lr=4.27e-6]
+
Train step of epoch 0:  29%|██▉       | 1615/5616 [20:36<53:16,  1.25it/s, gpt_loss=0.0404, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1616/5616 [20:36<52:50,  1.26it/s, gpt_loss=0.0404, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1616/5616 [20:36<52:50,  1.26it/s, gpt_loss=0.0686, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1617/5616 [20:36<51:01,  1.31it/s, gpt_loss=0.0686, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1617/5616 [20:37<51:01,  1.31it/s, gpt_loss=0.0322, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1618/5616 [20:37<48:44,  1.37it/s, gpt_loss=0.0322, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1618/5616 [20:38<48:44,  1.37it/s, gpt_loss=0.0153, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1619/5616 [20:38<48:41,  1.37it/s, gpt_loss=0.0153, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1619/5616 [20:39<48:41,  1.37it/s, gpt_loss=0.045, lr=4.26e-6] 
+
Train step of epoch 0:  29%|██▉       | 1620/5616 [20:39<51:06,  1.30it/s, gpt_loss=0.045, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1620/5616 [20:39<51:06,  1.30it/s, gpt_loss=0.106, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1621/5616 [20:39<48:40,  1.37it/s, gpt_loss=0.106, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1621/5616 [20:40<48:40,  1.37it/s, gpt_loss=0.129, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1622/5616 [20:40<49:33,  1.34it/s, gpt_loss=0.129, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1622/5616 [20:41<49:33,  1.34it/s, gpt_loss=0.0188, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1623/5616 [20:41<49:38,  1.34it/s, gpt_loss=0.0188, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1623/5616 [20:42<49:38,  1.34it/s, gpt_loss=0.0973, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1624/5616 [20:42<49:35,  1.34it/s, gpt_loss=0.0973, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1624/5616 [20:42<49:35,  1.34it/s, gpt_loss=0.0139, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1625/5616 [20:42<48:15,  1.38it/s, gpt_loss=0.0139, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1625/5616 [20:43<48:15,  1.38it/s, gpt_loss=0.0237, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1626/5616 [20:43<47:23,  1.40it/s, gpt_loss=0.0237, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1626/5616 [20:44<47:23,  1.40it/s, gpt_loss=0.0682, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1627/5616 [20:44<48:42,  1.36it/s, gpt_loss=0.0682, lr=4.26e-6]
+
Train step of epoch 0:  29%|██▉       | 1627/5616 [20:44<48:42,  1.36it/s, gpt_loss=0.0998, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1628/5616 [20:44<48:37,  1.37it/s, gpt_loss=0.0998, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1628/5616 [20:45<48:37,  1.37it/s, gpt_loss=0.00299, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1629/5616 [20:45<47:55,  1.39it/s, gpt_loss=0.00299, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1629/5616 [20:46<47:55,  1.39it/s, gpt_loss=0.00315, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1630/5616 [20:46<47:17,  1.40it/s, gpt_loss=0.00315, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1630/5616 [20:46<47:17,  1.40it/s, gpt_loss=0.0227, lr=4.25e-6] 
+
Train step of epoch 0:  29%|██▉       | 1631/5616 [20:46<46:52,  1.42it/s, gpt_loss=0.0227, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1631/5616 [20:47<46:52,  1.42it/s, gpt_loss=0.0743, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1632/5616 [20:47<46:53,  1.42it/s, gpt_loss=0.0743, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1632/5616 [20:48<46:53,  1.42it/s, gpt_loss=0.0227, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1633/5616 [20:48<47:10,  1.41it/s, gpt_loss=0.0227, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1633/5616 [20:49<47:10,  1.41it/s, gpt_loss=0.0421, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1634/5616 [20:49<46:31,  1.43it/s, gpt_loss=0.0421, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1634/5616 [20:49<46:31,  1.43it/s, gpt_loss=0.0721, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1635/5616 [20:49<46:25,  1.43it/s, gpt_loss=0.0721, lr=4.25e-6]
+
Train step of epoch 0:  29%|██▉       | 1635/5616 [20:50<46:25,  1.43it/s, gpt_loss=0.00359, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1636/5616 [20:50<48:32,  1.37it/s, gpt_loss=0.00359, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1636/5616 [20:51<48:32,  1.37it/s, gpt_loss=0.01, lr=4.24e-6]   
+
Train step of epoch 0:  29%|██▉       | 1637/5616 [20:51<47:55,  1.38it/s, gpt_loss=0.01, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1637/5616 [20:51<47:55,  1.38it/s, gpt_loss=0.00855, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1638/5616 [20:51<46:32,  1.42it/s, gpt_loss=0.00855, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1638/5616 [20:52<46:32,  1.42it/s, gpt_loss=0.0251, lr=4.24e-6] 
+
Train step of epoch 0:  29%|██▉       | 1639/5616 [20:52<45:31,  1.46it/s, gpt_loss=0.0251, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1639/5616 [20:53<45:31,  1.46it/s, gpt_loss=0.0835, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1640/5616 [20:53<46:37,  1.42it/s, gpt_loss=0.0835, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1640/5616 [20:54<46:37,  1.42it/s, gpt_loss=0.0539, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1641/5616 [20:54<54:22,  1.22it/s, gpt_loss=0.0539, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1641/5616 [20:55<54:22,  1.22it/s, gpt_loss=0.00735, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1642/5616 [20:55<52:39,  1.26it/s, gpt_loss=0.00735, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1642/5616 [20:56<52:39,  1.26it/s, gpt_loss=0.069, lr=4.24e-6]  
+
Train step of epoch 0:  29%|██▉       | 1643/5616 [20:56<57:02,  1.16it/s, gpt_loss=0.069, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1643/5616 [20:57<57:02,  1.16it/s, gpt_loss=0.0116, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1644/5616 [20:57<57:06,  1.16it/s, gpt_loss=0.0116, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1644/5616 [20:57<57:06,  1.16it/s, gpt_loss=0.0467, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1645/5616 [20:57<55:01,  1.20it/s, gpt_loss=0.0467, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1645/5616 [20:58<55:01,  1.20it/s, gpt_loss=0.0149, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1646/5616 [20:58<52:12,  1.27it/s, gpt_loss=0.0149, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1646/5616 [20:59<52:12,  1.27it/s, gpt_loss=0.0215, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1647/5616 [20:59<50:13,  1.32it/s, gpt_loss=0.0215, lr=4.24e-6]
+
Train step of epoch 0:  29%|██▉       | 1647/5616 [21:00<50:13,  1.32it/s, gpt_loss=0.133, lr=4.23e-6] 
+
Train step of epoch 0:  29%|██▉       | 1648/5616 [21:00<51:19,  1.29it/s, gpt_loss=0.133, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1648/5616 [21:00<51:19,  1.29it/s, gpt_loss=0.0656, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1649/5616 [21:00<52:26,  1.26it/s, gpt_loss=0.0656, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1649/5616 [21:01<52:26,  1.26it/s, gpt_loss=0.0225, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1650/5616 [21:01<51:53,  1.27it/s, gpt_loss=0.0225, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1650/5616 [21:02<51:53,  1.27it/s, gpt_loss=0.0474, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1651/5616 [21:02<51:58,  1.27it/s, gpt_loss=0.0474, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1651/5616 [21:03<51:58,  1.27it/s, gpt_loss=0.0247, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1652/5616 [21:03<52:00,  1.27it/s, gpt_loss=0.0247, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1652/5616 [21:03<52:00,  1.27it/s, gpt_loss=0.0382, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1653/5616 [21:03<51:07,  1.29it/s, gpt_loss=0.0382, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1653/5616 [21:04<51:07,  1.29it/s, gpt_loss=0.112, lr=4.23e-6] 
+
Train step of epoch 0:  29%|██▉       | 1654/5616 [21:04<50:00,  1.32it/s, gpt_loss=0.112, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1654/5616 [21:05<50:00,  1.32it/s, gpt_loss=0.0703, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1655/5616 [21:05<49:48,  1.33it/s, gpt_loss=0.0703, lr=4.23e-6]
+
Train step of epoch 0:  29%|██▉       | 1655/5616 [21:06<49:48,  1.33it/s, gpt_loss=0.0187, lr=4.22e-6]
+
Train step of epoch 0:  29%|██▉       | 1656/5616 [21:06<50:53,  1.30it/s, gpt_loss=0.0187, lr=4.22e-6]
+
Train step of epoch 0:  29%|██▉       | 1656/5616 [21:06<50:53,  1.30it/s, gpt_loss=0.0302, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1657/5616 [21:06<48:52,  1.35it/s, gpt_loss=0.0302, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1657/5616 [21:07<48:52,  1.35it/s, gpt_loss=0.0269, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1658/5616 [21:07<47:47,  1.38it/s, gpt_loss=0.0269, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1658/5616 [21:08<47:47,  1.38it/s, gpt_loss=0.0114, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1659/5616 [21:08<49:06,  1.34it/s, gpt_loss=0.0114, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1659/5616 [21:09<49:06,  1.34it/s, gpt_loss=0.0288, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1660/5616 [21:09<50:15,  1.31it/s, gpt_loss=0.0288, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1660/5616 [21:09<50:15,  1.31it/s, gpt_loss=0.0849, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1661/5616 [21:09<49:14,  1.34it/s, gpt_loss=0.0849, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1661/5616 [21:10<49:14,  1.34it/s, gpt_loss=0.0764, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1662/5616 [21:10<48:54,  1.35it/s, gpt_loss=0.0764, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1662/5616 [21:11<48:54,  1.35it/s, gpt_loss=0.0576, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1663/5616 [21:11<47:52,  1.38it/s, gpt_loss=0.0576, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1663/5616 [21:12<47:52,  1.38it/s, gpt_loss=0.0449, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1664/5616 [21:12<48:39,  1.35it/s, gpt_loss=0.0449, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1664/5616 [21:12<48:39,  1.35it/s, gpt_loss=0.0203, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1665/5616 [21:12<50:52,  1.29it/s, gpt_loss=0.0203, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1665/5616 [21:13<50:52,  1.29it/s, gpt_loss=0.0156, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1666/5616 [21:13<50:47,  1.30it/s, gpt_loss=0.0156, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1666/5616 [21:14<50:47,  1.30it/s, gpt_loss=0.0763, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1667/5616 [21:14<48:22,  1.36it/s, gpt_loss=0.0763, lr=4.22e-6]
+
Train step of epoch 0:  30%|██▉       | 1667/5616 [21:15<48:22,  1.36it/s, gpt_loss=0.145, lr=4.21e-6] 
+
Train step of epoch 0:  30%|██▉       | 1668/5616 [21:15<48:52,  1.35it/s, gpt_loss=0.145, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1668/5616 [21:15<48:52,  1.35it/s, gpt_loss=0.0329, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1669/5616 [21:15<47:46,  1.38it/s, gpt_loss=0.0329, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1669/5616 [21:16<47:46,  1.38it/s, gpt_loss=0.212, lr=4.21e-6] 
+
Train step of epoch 0:  30%|██▉       | 1670/5616 [21:16<48:05,  1.37it/s, gpt_loss=0.212, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1670/5616 [21:17<48:05,  1.37it/s, gpt_loss=0.0509, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1671/5616 [21:17<56:27,  1.16it/s, gpt_loss=0.0509, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1671/5616 [21:18<56:27,  1.16it/s, gpt_loss=0.0532, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1672/5616 [21:18<56:18,  1.17it/s, gpt_loss=0.0532, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1672/5616 [21:19<56:18,  1.17it/s, gpt_loss=0.0416, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1673/5616 [21:19<54:18,  1.21it/s, gpt_loss=0.0416, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1673/5616 [21:20<54:18,  1.21it/s, gpt_loss=0.0959, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1674/5616 [21:20<53:18,  1.23it/s, gpt_loss=0.0959, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1674/5616 [21:20<53:18,  1.23it/s, gpt_loss=0.0445, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1675/5616 [21:20<52:09,  1.26it/s, gpt_loss=0.0445, lr=4.21e-6]
+
Train step of epoch 0:  30%|██▉       | 1675/5616 [21:21<52:09,  1.26it/s, gpt_loss=0.0166, lr=4.2e-6] 
+
Train step of epoch 0:  30%|██▉       | 1676/5616 [21:21<57:15,  1.15it/s, gpt_loss=0.0166, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1676/5616 [21:22<57:15,  1.15it/s, gpt_loss=0.0449, lr=4.2e-6]
+
Train step of epoch 0:  30%|██��       | 1677/5616 [21:22<57:16,  1.15it/s, gpt_loss=0.0449, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1677/5616 [21:23<57:16,  1.15it/s, gpt_loss=0.0104, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1678/5616 [21:23<54:02,  1.21it/s, gpt_loss=0.0104, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1678/5616 [21:24<54:02,  1.21it/s, gpt_loss=0.0278, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1679/5616 [21:24<51:56,  1.26it/s, gpt_loss=0.0278, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1679/5616 [21:25<51:56,  1.26it/s, gpt_loss=0.0514, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1680/5616 [21:25<52:35,  1.25it/s, gpt_loss=0.0514, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1680/5616 [21:25<52:35,  1.25it/s, gpt_loss=0.0864, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1681/5616 [21:25<50:28,  1.30it/s, gpt_loss=0.0864, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1681/5616 [21:26<50:28,  1.30it/s, gpt_loss=0.0464, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1682/5616 [21:26<51:51,  1.26it/s, gpt_loss=0.0464, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1682/5616 [21:27<51:51,  1.26it/s, gpt_loss=0.0267, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1683/5616 [21:27<54:05,  1.21it/s, gpt_loss=0.0267, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1683/5616 [21:28<54:05,  1.21it/s, gpt_loss=0.0822, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1684/5616 [21:28<55:11,  1.19it/s, gpt_loss=0.0822, lr=4.2e-6]
+
Train step of epoch 0:  30%|██▉       | 1684/5616 [21:29<55:11,  1.19it/s, gpt_loss=0.0826, lr=4.2e-6]
+
Train step of epoch 0:  30%|███       | 1685/5616 [21:29<53:22,  1.23it/s, gpt_loss=0.0826, lr=4.2e-6]
+
Train step of epoch 0:  30%|███       | 1685/5616 [21:29<53:22,  1.23it/s, gpt_loss=0.0607, lr=4.2e-6]
+
Train step of epoch 0:  30%|███       | 1686/5616 [21:29<51:53,  1.26it/s, gpt_loss=0.0607, lr=4.2e-6]
+
Train step of epoch 0:  30%|███       | 1686/5616 [21:30<51:53,  1.26it/s, gpt_loss=0.0196, lr=4.2e-6]
+
Train step of epoch 0:  30%|███       | 1687/5616 [21:30<49:38,  1.32it/s, gpt_loss=0.0196, lr=4.2e-6]
+
Train step of epoch 0:  30%|███       | 1687/5616 [21:31<49:38,  1.32it/s, gpt_loss=0.0921, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1688/5616 [21:31<50:32,  1.30it/s, gpt_loss=0.0921, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1688/5616 [21:32<50:32,  1.30it/s, gpt_loss=0.0253, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1689/5616 [21:32<49:13,  1.33it/s, gpt_loss=0.0253, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1689/5616 [21:32<49:13,  1.33it/s, gpt_loss=0.0192, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1690/5616 [21:32<50:11,  1.30it/s, gpt_loss=0.0192, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1690/5616 [21:33<50:11,  1.30it/s, gpt_loss=0.0671, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1691/5616 [21:33<49:47,  1.31it/s, gpt_loss=0.0671, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1691/5616 [21:34<49:47,  1.31it/s, gpt_loss=0.0376, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1692/5616 [21:34<50:32,  1.29it/s, gpt_loss=0.0376, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1692/5616 [21:35<50:32,  1.29it/s, gpt_loss=0.0473, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1693/5616 [21:35<49:29,  1.32it/s, gpt_loss=0.0473, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1693/5616 [21:35<49:29,  1.32it/s, gpt_loss=0.0107, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1694/5616 [21:35<47:40,  1.37it/s, gpt_loss=0.0107, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1694/5616 [21:36<47:40,  1.37it/s, gpt_loss=0.0332, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1695/5616 [21:36<49:10,  1.33it/s, gpt_loss=0.0332, lr=4.19e-6]
+
Train step of epoch 0:  30%|███       | 1695/5616 [21:37<49:10,  1.33it/s, gpt_loss=0.0783, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1696/5616 [21:37<49:50,  1.31it/s, gpt_loss=0.0783, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1696/5616 [21:38<49:50,  1.31it/s, gpt_loss=0.0251, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1697/5616 [21:38<48:23,  1.35it/s, gpt_loss=0.0251, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1697/5616 [21:38<48:23,  1.35it/s, gpt_loss=0.0334, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1698/5616 [21:38<48:10,  1.36it/s, gpt_loss=0.0334, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1698/5616 [21:39<48:10,  1.36it/s, gpt_loss=0.0352, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1699/5616 [21:39<51:02,  1.28it/s, gpt_loss=0.0352, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1699/5616 [21:40<51:02,  1.28it/s, gpt_loss=0.0293, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1700/5616 [21:40<51:30,  1.27it/s, gpt_loss=0.0293, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1700/5616 [21:41<51:30,  1.27it/s, gpt_loss=0.0489, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1701/5616 [21:41<51:10,  1.27it/s, gpt_loss=0.0489, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1701/5616 [21:42<51:10,  1.27it/s, gpt_loss=0.0858, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1702/5616 [21:42<51:24,  1.27it/s, gpt_loss=0.0858, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1702/5616 [21:42<51:24,  1.27it/s, gpt_loss=0.00794, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1703/5616 [21:42<50:27,  1.29it/s, gpt_loss=0.00794, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1703/5616 [21:43<50:27,  1.29it/s, gpt_loss=0.0162, lr=4.18e-6] 
+
Train step of epoch 0:  30%|███       | 1704/5616 [21:43<50:28,  1.29it/s, gpt_loss=0.0162, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1704/5616 [21:44<50:28,  1.29it/s, gpt_loss=0.041, lr=4.18e-6] 
+
Train step of epoch 0:  30%|███       | 1705/5616 [21:44<50:52,  1.28it/s, gpt_loss=0.041, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1705/5616 [21:45<50:52,  1.28it/s, gpt_loss=0.0222, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1706/5616 [21:45<50:43,  1.28it/s, gpt_loss=0.0222, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1706/5616 [21:45<50:43,  1.28it/s, gpt_loss=0.0895, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1707/5616 [21:45<49:12,  1.32it/s, gpt_loss=0.0895, lr=4.18e-6]
+
Train step of epoch 0:  30%|███       | 1707/5616 [21:46<49:12,  1.32it/s, gpt_loss=0.0137, lr=4.17e-6]
+
Train step of epoch 0:  30%|███       | 1708/5616 [21:46<51:15,  1.27it/s, gpt_loss=0.0137, lr=4.17e-6]
+
Train step of epoch 0:  30%|███       | 1708/5616 [21:47<51:15,  1.27it/s, gpt_loss=0.0351, lr=4.17e-6]
+
Train step of epoch 0:  30%|███       | 1709/5616 [21:47<52:16,  1.25it/s, gpt_loss=0.0351, lr=4.17e-6]
+
Train step of epoch 0:  30%|███       | 1709/5616 [21:48<52:16,  1.25it/s, gpt_loss=0.0397, lr=4.17e-6]
+
Train step of epoch 0:  30%|███       | 1710/5616 [21:48<49:40,  1.31it/s, gpt_loss=0.0397, lr=4.17e-6]
+
Train step of epoch 0:  30%|███       | 1710/5616 [21:48<49:40,  1.31it/s, gpt_loss=0.0791, lr=4.17e-6]
+
Train step of epoch 0:  30%|███       | 1711/5616 [21:48<49:01,  1.33it/s, gpt_loss=0.0791, lr=4.17e-6]
+
Train step of epoch 0:  30%|███       | 1711/5616 [21:49<49:01,  1.33it/s, gpt_loss=0.0636, lr=4.17e-6]
+
Train step of epoch 0:  30%|███       | 1712/5616 [21:49<48:47,  1.33it/s, gpt_loss=0.0636, lr=4.17e-6]
+
Train step of epoch 0:  30%|███       | 1712/5616 [21:50<48:47,  1.33it/s, gpt_loss=0.0308, lr=4.17e-6]
+
Train step of epoch 0:  31%|███       | 1713/5616 [21:50<48:33,  1.34it/s, gpt_loss=0.0308, lr=4.17e-6]
+
Train step of epoch 0:  31%|███       | 1713/5616 [21:51<48:33,  1.34it/s, gpt_loss=0.0502, lr=4.17e-6]
+
Train step of epoch 0:  31%|███       | 1714/5616 [21:51<49:36,  1.31it/s, gpt_loss=0.0502, lr=4.17e-6]
+
Train step of epoch 0:  31%|███       | 1714/5616 [21:51<49:36,  1.31it/s, gpt_loss=0.0758, lr=4.17e-6]
+
Train step of epoch 0:  31%|███       | 1715/5616 [21:51<48:56,  1.33it/s, gpt_loss=0.0758, lr=4.17e-6]
+
Train step of epoch 0:  31%|███       | 1715/5616 [21:52<48:56,  1.33it/s, gpt_loss=0.0297, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1716/5616 [21:52<49:19,  1.32it/s, gpt_loss=0.0297, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1716/5616 [21:53<49:19,  1.32it/s, gpt_loss=0.019, lr=4.16e-6] 
+
Train step of epoch 0:  31%|███       | 1717/5616 [21:53<49:36,  1.31it/s, gpt_loss=0.019, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1717/5616 [21:54<49:36,  1.31it/s, gpt_loss=0.00143, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1718/5616 [21:54<48:08,  1.35it/s, gpt_loss=0.00143, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1718/5616 [21:54<48:08,  1.35it/s, gpt_loss=0.0431, lr=4.16e-6] 
+
Train step of epoch 0:  31%|███       | 1719/5616 [21:54<48:02,  1.35it/s, gpt_loss=0.0431, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1719/5616 [21:55<48:02,  1.35it/s, gpt_loss=0.0525, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1720/5616 [21:55<48:43,  1.33it/s, gpt_loss=0.0525, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1720/5616 [21:56<48:43,  1.33it/s, gpt_loss=0.00541, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1721/5616 [21:56<50:44,  1.28it/s, gpt_loss=0.00541, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1721/5616 [21:57<50:44,  1.28it/s, gpt_loss=0.0115, lr=4.16e-6] 
+
Train step of epoch 0:  31%|███       | 1722/5616 [21:57<49:02,  1.32it/s, gpt_loss=0.0115, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1722/5616 [21:57<49:02,  1.32it/s, gpt_loss=0.0135, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1723/5616 [21:57<49:02,  1.32it/s, gpt_loss=0.0135, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1723/5616 [21:58<49:02,  1.32it/s, gpt_loss=0.108, lr=4.16e-6] 
+
Train step of epoch 0:  31%|███       | 1724/5616 [21:58<49:19,  1.31it/s, gpt_loss=0.108, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1724/5616 [21:59<49:19,  1.31it/s, gpt_loss=0.104, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1725/5616 [21:59<47:36,  1.36it/s, gpt_loss=0.104, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1725/5616 [22:00<47:36,  1.36it/s, gpt_loss=0.0126, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1726/5616 [22:00<46:02,  1.41it/s, gpt_loss=0.0126, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1726/5616 [22:00<46:02,  1.41it/s, gpt_loss=0.0157, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1727/5616 [22:00<45:23,  1.43it/s, gpt_loss=0.0157, lr=4.16e-6]
+
Train step of epoch 0:  31%|███       | 1727/5616 [22:01<45:23,  1.43it/s, gpt_loss=0.102, lr=4.15e-6] 
+
Train step of epoch 0:  31%|███       | 1728/5616 [22:01<46:42,  1.39it/s, gpt_loss=0.102, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1728/5616 [22:02<46:42,  1.39it/s, gpt_loss=0.0361, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1729/5616 [22:02<46:04,  1.41it/s, gpt_loss=0.0361, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1729/5616 [22:02<46:04,  1.41it/s, gpt_loss=0.0346, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1730/5616 [22:02<45:39,  1.42it/s, gpt_loss=0.0346, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1730/5616 [22:03<45:39,  1.42it/s, gpt_loss=0.0208, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1731/5616 [22:03<48:58,  1.32it/s, gpt_loss=0.0208, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1731/5616 [22:04<48:58,  1.32it/s, gpt_loss=0.0484, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1732/5616 [22:04<49:00,  1.32it/s, gpt_loss=0.0484, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1732/5616 [22:05<49:00,  1.32it/s, gpt_loss=0.155, lr=4.15e-6] 
+
Train step of epoch 0:  31%|███       | 1733/5616 [22:05<47:42,  1.36it/s, gpt_loss=0.155, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1733/5616 [22:05<47:42,  1.36it/s, gpt_loss=0.042, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1734/5616 [22:05<46:22,  1.40it/s, gpt_loss=0.042, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1734/5616 [22:06<46:22,  1.40it/s, gpt_loss=0.0182, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1735/5616 [22:06<44:58,  1.44it/s, gpt_loss=0.0182, lr=4.15e-6]
+
Train step of epoch 0:  31%|███       | 1735/5616 [22:07<44:58,  1.44it/s, gpt_loss=0.186, lr=4.14e-6] 
+
Train step of epoch 0:  31%|███       | 1736/5616 [22:07<47:53,  1.35it/s, gpt_loss=0.186, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1736/5616 [22:08<47:53,  1.35it/s, gpt_loss=0.155, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1737/5616 [22:08<46:43,  1.38it/s, gpt_loss=0.155, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1737/5616 [22:08<46:43,  1.38it/s, gpt_loss=0.12, lr=4.14e-6] 
+
Train step of epoch 0:  31%|███       | 1738/5616 [22:08<46:19,  1.40it/s, gpt_loss=0.12, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1738/5616 [22:09<46:19,  1.40it/s, gpt_loss=0.00124, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1739/5616 [22:09<45:02,  1.43it/s, gpt_loss=0.00124, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1739/5616 [22:10<45:02,  1.43it/s, gpt_loss=0.0096, lr=4.14e-6] 
+
Train step of epoch 0:  31%|███       | 1740/5616 [22:10<47:30,  1.36it/s, gpt_loss=0.0096, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1740/5616 [22:11<47:30,  1.36it/s, gpt_loss=0.067, lr=4.14e-6] 
+
Train step of epoch 0:  31%|███       | 1741/5616 [22:11<49:30,  1.30it/s, gpt_loss=0.067, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1741/5616 [22:11<49:30,  1.30it/s, gpt_loss=0.00567, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1742/5616 [22:11<49:59,  1.29it/s, gpt_loss=0.00567, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1742/5616 [22:12<49:59,  1.29it/s, gpt_loss=0.00498, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1743/5616 [22:12<47:16,  1.37it/s, gpt_loss=0.00498, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1743/5616 [22:13<47:16,  1.37it/s, gpt_loss=0.133, lr=4.14e-6]  
+
Train step of epoch 0:  31%|███       | 1744/5616 [22:13<49:15,  1.31it/s, gpt_loss=0.133, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1744/5616 [22:14<49:15,  1.31it/s, gpt_loss=0.077, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1745/5616 [22:14<48:02,  1.34it/s, gpt_loss=0.077, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1745/5616 [22:14<48:02,  1.34it/s, gpt_loss=0.0377, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1746/5616 [22:14<48:30,  1.33it/s, gpt_loss=0.0377, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1746/5616 [22:15<48:30,  1.33it/s, gpt_loss=0.0295, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1747/5616 [22:15<52:01,  1.24it/s, gpt_loss=0.0295, lr=4.14e-6]
+
Train step of epoch 0:  31%|███       | 1747/5616 [22:16<52:01,  1.24it/s, gpt_loss=0.146, lr=4.13e-6] 
+
Train step of epoch 0:  31%|███       | 1748/5616 [22:16<53:16,  1.21it/s, gpt_loss=0.146, lr=4.13e-6]
+
Train step of epoch 0:  31%|███       | 1748/5616 [22:17<53:16,  1.21it/s, gpt_loss=0.11, lr=4.13e-6] 
+
Train step of epoch 0:  31%|███       | 1749/5616 [22:17<56:13,  1.15it/s, gpt_loss=0.11, lr=4.13e-6]
+
Train step of epoch 0:  31%|███       | 1749/5616 [22:18<56:13,  1.15it/s, gpt_loss=0.0054, lr=4.13e-6]
+
Train step of epoch 0:  31%|███       | 1750/5616 [22:18<52:55,  1.22it/s, gpt_loss=0.0054, lr=4.13e-6]
+
Train step of epoch 0:  31%|███       | 1750/5616 [22:18<52:55,  1.22it/s, gpt_loss=0.0672, lr=4.13e-6]
+
Train step of epoch 0:  31%|███       | 1751/5616 [22:18<49:59,  1.29it/s, gpt_loss=0.0672, lr=4.13e-6]
+
Train step of epoch 0:  31%|███       | 1751/5616 [22:19<49:59,  1.29it/s, gpt_loss=0.0603, lr=4.13e-6]
+
Train step of epoch 0:  31%|███       | 1752/5616 [22:19<50:50,  1.27it/s, gpt_loss=0.0603, lr=4.13e-6]
+
Train step of epoch 0:  31%|███       | 1752/5616 [22:20<50:50,  1.27it/s, gpt_loss=0.0159, lr=4.13e-6]
+
Train step of epoch 0:  31%|███       | 1753/5616 [22:20<48:24,  1.33it/s, gpt_loss=0.0159, lr=4.13e-6]
+
Train step of epoch 0:  31%|███       | 1753/5616 [22:21<48:24,  1.33it/s, gpt_loss=0.0912, lr=4.13e-6]
+
Train step of epoch 0:  31%|███       | 1754/5616 [22:21<48:38,  1.32it/s, gpt_loss=0.0912, lr=4.13e-6]
+
Train step of epoch 0:  31%|███       | 1754/5616 [22:21<48:38,  1.32it/s, gpt_loss=0.0101, lr=4.13e-6]
+
Train step of epoch 0:  31%|███▏      | 1755/5616 [22:21<46:31,  1.38it/s, gpt_loss=0.0101, lr=4.13e-6]
+
Train step of epoch 0:  31%|███▏      | 1755/5616 [22:22<46:31,  1.38it/s, gpt_loss=0.0666, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1756/5616 [22:22<46:32,  1.38it/s, gpt_loss=0.0666, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1756/5616 [22:23<46:32,  1.38it/s, gpt_loss=0.137, lr=4.12e-6] 
+
Train step of epoch 0:  31%|███▏      | 1757/5616 [22:23<45:42,  1.41it/s, gpt_loss=0.137, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1757/5616 [22:23<45:42,  1.41it/s, gpt_loss=0.0439, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1758/5616 [22:23<44:46,  1.44it/s, gpt_loss=0.0439, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1758/5616 [22:24<44:46,  1.44it/s, gpt_loss=0.00271, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1759/5616 [22:24<46:42,  1.38it/s, gpt_loss=0.00271, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1759/5616 [22:25<46:42,  1.38it/s, gpt_loss=0.00765, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1760/5616 [22:25<47:05,  1.36it/s, gpt_loss=0.00765, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1760/5616 [22:26<47:05,  1.36it/s, gpt_loss=0.0533, lr=4.12e-6] 
+
Train step of epoch 0:  31%|███▏      | 1761/5616 [22:26<47:37,  1.35it/s, gpt_loss=0.0533, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1761/5616 [22:26<47:37,  1.35it/s, gpt_loss=0.00255, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1762/5616 [22:26<47:52,  1.34it/s, gpt_loss=0.00255, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1762/5616 [22:28<47:52,  1.34it/s, gpt_loss=0.0859, lr=4.12e-6] 
+
Train step of epoch 0:  31%|███▏      | 1763/5616 [22:28<56:29,  1.14it/s, gpt_loss=0.0859, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1763/5616 [22:28<56:29,  1.14it/s, gpt_loss=0.0163, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1764/5616 [22:28<54:16,  1.18it/s, gpt_loss=0.0163, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1764/5616 [22:29<54:16,  1.18it/s, gpt_loss=0.0357, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1765/5616 [22:29<51:21,  1.25it/s, gpt_loss=0.0357, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1765/5616 [22:30<51:21,  1.25it/s, gpt_loss=0.0354, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1766/5616 [22:30<49:10,  1.30it/s, gpt_loss=0.0354, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1766/5616 [22:31<49:10,  1.30it/s, gpt_loss=0.0403, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1767/5616 [22:31<48:49,  1.31it/s, gpt_loss=0.0403, lr=4.12e-6]
+
Train step of epoch 0:  31%|███▏      | 1767/5616 [22:31<48:49,  1.31it/s, gpt_loss=0.115, lr=4.11e-6] 
+
Train step of epoch 0:  31%|███▏      | 1768/5616 [22:31<47:59,  1.34it/s, gpt_loss=0.115, lr=4.11e-6]
+
Train step of epoch 0:  31%|███▏      | 1768/5616 [22:32<47:59,  1.34it/s, gpt_loss=0.0433, lr=4.11e-6]
+
Train step of epoch 0:  31%|███▏      | 1769/5616 [22:32<48:15,  1.33it/s, gpt_loss=0.0433, lr=4.11e-6]
+
Train step of epoch 0:  31%|███▏      | 1769/5616 [22:33<48:15,  1.33it/s, gpt_loss=0.0479, lr=4.11e-6]
+
Train step of epoch 0:  32%|███▏      | 1770/5616 [22:33<47:27,  1.35it/s, gpt_loss=0.0479, lr=4.11e-6]
+
Train step of epoch 0:  32%|███▏      | 1770/5616 [22:34<47:27,  1.35it/s, gpt_loss=0.0766, lr=4.11e-6]
+
Train step of epoch 0:  32%|███▏      | 1771/5616 [22:34<47:42,  1.34it/s, gpt_loss=0.0766, lr=4.11e-6]
+
Train step of epoch 0:  32%|███▏      | 1771/5616 [22:34<47:42,  1.34it/s, gpt_loss=0.0143, lr=4.11e-6]
+
Train step of epoch 0:  32%|███▏      | 1772/5616 [22:34<48:56,  1.31it/s, gpt_loss=0.0143, lr=4.11e-6]
+
Train step of epoch 0:  32%|███▏      | 1772/5616 [22:35<48:56,  1.31it/s, gpt_loss=0.0268, lr=4.11e-6]
+
Train step of epoch 0:  32%|███▏      | 1773/5616 [22:35<49:02,  1.31it/s, gpt_loss=0.0268, lr=4.11e-6]
+
Train step of epoch 0:  32%|███▏      | 1773/5616 [22:36<49:02,  1.31it/s, gpt_loss=0.0408, lr=4.11e-6]
+
Train step of epoch 0:  32%|███▏      | 1774/5616 [22:36<47:14,  1.36it/s, gpt_loss=0.0408, lr=4.11e-6]
+
Train step of epoch 0:  32%|███▏      | 1774/5616 [22:37<47:14,  1.36it/s, gpt_loss=0.0145, lr=4.11e-6]
+
Train step of epoch 0:  32%|███▏      | 1775/5616 [22:37<52:01,  1.23it/s, gpt_loss=0.0145, lr=4.11e-6]
+
Train step of epoch 0:  32%|███▏      | 1775/5616 [22:38<52:01,  1.23it/s, gpt_loss=0.105, lr=4.1e-6]  
+
Train step of epoch 0:  32%|███▏      | 1776/5616 [22:38<52:52,  1.21it/s, gpt_loss=0.105, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1776/5616 [22:38<52:52,  1.21it/s, gpt_loss=0.184, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1777/5616 [22:38<52:38,  1.22it/s, gpt_loss=0.184, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1777/5616 [22:39<52:38,  1.22it/s, gpt_loss=0.0759, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1778/5616 [22:39<49:58,  1.28it/s, gpt_loss=0.0759, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1778/5616 [22:40<49:58,  1.28it/s, gpt_loss=0.0134, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1779/5616 [22:40<48:18,  1.32it/s, gpt_loss=0.0134, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1779/5616 [22:41<48:18,  1.32it/s, gpt_loss=0.0118, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1780/5616 [22:41<51:19,  1.25it/s, gpt_loss=0.0118, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1780/5616 [22:42<51:19,  1.25it/s, gpt_loss=0.00998, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1781/5616 [22:42<50:45,  1.26it/s, gpt_loss=0.00998, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1781/5616 [22:42<50:45,  1.26it/s, gpt_loss=0.0231, lr=4.1e-6] 
+
Train step of epoch 0:  32%|███▏      | 1782/5616 [22:42<49:24,  1.29it/s, gpt_loss=0.0231, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1782/5616 [22:43<49:24,  1.29it/s, gpt_loss=0.0521, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1783/5616 [22:43<47:06,  1.36it/s, gpt_loss=0.0521, lr=4.1e-6]
+
Train step of epoch 0:  32%|███▏      | 1783/5616 [22:44<47:06,  1.36it/s, gpt_loss=0.0543, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1784/5616 [22:44<54:57,  1.16it/s, gpt_loss=0.0543, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1784/5616 [22:45<54:57,  1.16it/s, gpt_loss=0.0783, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1785/5616 [22:45<54:40,  1.17it/s, gpt_loss=0.0783, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1785/5616 [22:46<54:40,  1.17it/s, gpt_loss=0.0371, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1786/5616 [22:46<52:14,  1.22it/s, gpt_loss=0.0371, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1786/5616 [22:46<52:14,  1.22it/s, gpt_loss=0.0107, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1787/5616 [22:46<51:34,  1.24it/s, gpt_loss=0.0107, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1787/5616 [22:47<51:34,  1.24it/s, gpt_loss=0.0234, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1788/5616 [22:47<50:49,  1.26it/s, gpt_loss=0.0234, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1788/5616 [22:48<50:49,  1.26it/s, gpt_loss=0.0708, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1789/5616 [22:48<53:45,  1.19it/s, gpt_loss=0.0708, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1789/5616 [22:49<53:45,  1.19it/s, gpt_loss=0.0422, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1790/5616 [22:49<49:53,  1.28it/s, gpt_loss=0.0422, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1790/5616 [22:49<49:53,  1.28it/s, gpt_loss=0.0531, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1791/5616 [22:49<48:46,  1.31it/s, gpt_loss=0.0531, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1791/5616 [22:50<48:46,  1.31it/s, gpt_loss=0.0684, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1792/5616 [22:50<50:55,  1.25it/s, gpt_loss=0.0684, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1792/5616 [22:51<50:55,  1.25it/s, gpt_loss=0.0137, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1793/5616 [22:51<52:35,  1.21it/s, gpt_loss=0.0137, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1793/5616 [22:52<52:35,  1.21it/s, gpt_loss=0.0289, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1794/5616 [22:52<50:14,  1.27it/s, gpt_loss=0.0289, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1794/5616 [22:53<50:14,  1.27it/s, gpt_loss=0.019, lr=4.09e-6] 
+
Train step of epoch 0:  32%|███▏      | 1795/5616 [22:53<49:48,  1.28it/s, gpt_loss=0.019, lr=4.09e-6]
+
Train step of epoch 0:  32%|███▏      | 1795/5616 [22:54<49:48,  1.28it/s, gpt_loss=0.0705, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1796/5616 [22:54<51:23,  1.24it/s, gpt_loss=0.0705, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1796/5616 [22:54<51:23,  1.24it/s, gpt_loss=0.0629, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1797/5616 [22:54<48:36,  1.31it/s, gpt_loss=0.0629, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1797/5616 [22:55<48:36,  1.31it/s, gpt_loss=0.136, lr=4.08e-6] 
+
Train step of epoch 0:  32%|███▏      | 1798/5616 [22:55<47:24,  1.34it/s, gpt_loss=0.136, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1798/5616 [22:56<47:24,  1.34it/s, gpt_loss=0.0599, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1799/5616 [22:56<50:37,  1.26it/s, gpt_loss=0.0599, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1799/5616 [22:57<50:37,  1.26it/s, gpt_loss=0.0605, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1800/5616 [22:57<48:54,  1.30it/s, gpt_loss=0.0605, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1800/5616 [22:57<48:54,  1.30it/s, gpt_loss=0.0272, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1801/5616 [22:57<47:17,  1.34it/s, gpt_loss=0.0272, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1801/5616 [22:58<47:17,  1.34it/s, gpt_loss=0.0382, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1802/5616 [22:58<48:08,  1.32it/s, gpt_loss=0.0382, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1802/5616 [22:59<48:08,  1.32it/s, gpt_loss=0.0337, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1803/5616 [22:59<47:04,  1.35it/s, gpt_loss=0.0337, lr=4.08e-6]
+
Train step of epoch 0:  32%|███▏      | 1803/5616 [23:00<47:04,  1.35it/s, gpt_loss=0.0867, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1804/5616 [23:00<48:34,  1.31it/s, gpt_loss=0.0867, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1804/5616 [23:00<48:34,  1.31it/s, gpt_loss=0.0228, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1805/5616 [23:00<48:34,  1.31it/s, gpt_loss=0.0228, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1805/5616 [23:01<48:34,  1.31it/s, gpt_loss=0.0906, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1806/5616 [23:01<46:23,  1.37it/s, gpt_loss=0.0906, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1806/5616 [23:02<46:23,  1.37it/s, gpt_loss=0.0511, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1807/5616 [23:02<46:53,  1.35it/s, gpt_loss=0.0511, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1807/5616 [23:02<46:53,  1.35it/s, gpt_loss=0.0312, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1808/5616 [23:02<47:02,  1.35it/s, gpt_loss=0.0312, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1808/5616 [23:03<47:02,  1.35it/s, gpt_loss=0.185, lr=4.07e-6] 
+
Train step of epoch 0:  32%|███▏      | 1809/5616 [23:03<47:21,  1.34it/s, gpt_loss=0.185, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1809/5616 [23:04<47:21,  1.34it/s, gpt_loss=0.0294, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1810/5616 [23:04<45:54,  1.38it/s, gpt_loss=0.0294, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1810/5616 [23:05<45:54,  1.38it/s, gpt_loss=0.104, lr=4.07e-6] 
+
Train step of epoch 0:  32%|███▏      | 1811/5616 [23:05<52:48,  1.20it/s, gpt_loss=0.104, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1811/5616 [23:06<52:48,  1.20it/s, gpt_loss=0.0942, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1812/5616 [23:06<52:11,  1.21it/s, gpt_loss=0.0942, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1812/5616 [23:07<52:11,  1.21it/s, gpt_loss=0.0435, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1813/5616 [23:07<51:07,  1.24it/s, gpt_loss=0.0435, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1813/5616 [23:07<51:07,  1.24it/s, gpt_loss=0.0258, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1814/5616 [23:07<49:24,  1.28it/s, gpt_loss=0.0258, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1814/5616 [23:08<49:24,  1.28it/s, gpt_loss=0.0512, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1815/5616 [23:08<47:50,  1.32it/s, gpt_loss=0.0512, lr=4.07e-6]
+
Train step of epoch 0:  32%|███▏      | 1815/5616 [23:09<47:50,  1.32it/s, gpt_loss=0.121, lr=4.06e-6] 
+
Train step of epoch 0:  32%|███▏      | 1816/5616 [23:09<48:45,  1.30it/s, gpt_loss=0.121, lr=4.06e-6]
+
Train step of epoch 0:  32%|███▏      | 1816/5616 [23:09<48:45,  1.30it/s, gpt_loss=0.00416, lr=4.06e-6]
+
Train step of epoch 0:  32%|███▏      | 1817/5616 [23:09<47:33,  1.33it/s, gpt_loss=0.00416, lr=4.06e-6]
+
Train step of epoch 0:  32%|███▏      | 1817/5616 [23:10<47:33,  1.33it/s, gpt_loss=0.0208, lr=4.06e-6] 
+
Train step of epoch 0:  32%|███▏      | 1818/5616 [23:10<48:40,  1.30it/s, gpt_loss=0.0208, lr=4.06e-6]
+
Train step of epoch 0:  32%|███▏      | 1818/5616 [23:11<48:40,  1.30it/s, gpt_loss=0.0452, lr=4.06e-6]
+
Train step of epoch 0:  32%|███▏      | 1819/5616 [23:11<47:11,  1.34it/s, gpt_loss=0.0452, lr=4.06e-6]
+
Train step of epoch 0:  32%|███▏      | 1819/5616 [23:12<47:11,  1.34it/s, gpt_loss=0.0654, lr=4.06e-6]
+
Train step of epoch 0:  32%|███▏      | 1820/5616 [23:12<47:19,  1.34it/s, gpt_loss=0.0654, lr=4.06e-6]
+
Train step of epoch 0:  32%|███▏      | 1820/5616 [23:13<47:19,  1.34it/s, gpt_loss=0.053, lr=4.06e-6] 
+
Train step of epoch 0:  32%|███▏      | 1821/5616 [23:13<53:34,  1.18it/s, gpt_loss=0.053, lr=4.06e-6]
+
Train step of epoch 0:  32%|███▏      | 1821/5616 [23:14<53:34,  1.18it/s, gpt_loss=0.0621, lr=4.06e-6]
+
Train step of epoch 0:  32%|███▏      | 1822/5616 [23:14<52:46,  1.20it/s, gpt_loss=0.0621, lr=4.06e-6]
+
Train step of epoch 0:  32%|███▏      | 1822/5616 [23:14<52:46,  1.20it/s, gpt_loss=0.134, lr=4.06e-6] 
+
Train step of epoch 0:  32%|███▏      | 1823/5616 [23:14<49:03,  1.29it/s, gpt_loss=0.134, lr=4.06e-6]
+
Train step of epoch 0:  32%|███▏      | 1823/5616 [23:15<49:03,  1.29it/s, gpt_loss=0.0414, lr=4.05e-6]
+
Train step of epoch 0:  32%|███▏      | 1824/5616 [23:15<49:35,  1.27it/s, gpt_loss=0.0414, lr=4.05e-6]
+
Train step of epoch 0:  32%|███▏      | 1824/5616 [23:16<49:35,  1.27it/s, gpt_loss=0.00182, lr=4.05e-6]
+
Train step of epoch 0:  32%|███▏      | 1825/5616 [23:16<50:13,  1.26it/s, gpt_loss=0.00182, lr=4.05e-6]
+
Train step of epoch 0:  32%|███▏      | 1825/5616 [23:17<50:13,  1.26it/s, gpt_loss=0.0429, lr=4.05e-6] 
+
Train step of epoch 0:  33%|███▎      | 1826/5616 [23:17<48:48,  1.29it/s, gpt_loss=0.0429, lr=4.05e-6]
+
Train step of epoch 0:  33%|███▎      | 1826/5616 [23:17<48:48,  1.29it/s, gpt_loss=0.0173, lr=4.05e-6]
+
Train step of epoch 0:  33%|███▎      | 1827/5616 [23:17<47:46,  1.32it/s, gpt_loss=0.0173, lr=4.05e-6]
+
Train step of epoch 0:  33%|███▎      | 1827/5616 [23:18<47:46,  1.32it/s, gpt_loss=0.0561, lr=4.05e-6]
+
Train step of epoch 0:  33%|███▎      | 1828/5616 [23:18<47:30,  1.33it/s, gpt_loss=0.0561, lr=4.05e-6]
+
Train step of epoch 0:  33%|███▎      | 1828/5616 [23:19<47:30,  1.33it/s, gpt_loss=0.0175, lr=4.05e-6]
+
Train step of epoch 0:  33%|███▎      | 1829/5616 [23:19<46:24,  1.36it/s, gpt_loss=0.0175, lr=4.05e-6]
+
Train step of epoch 0:  33%|███▎      | 1829/5616 [23:20<46:24,  1.36it/s, gpt_loss=0.00384, lr=4.05e-6]
+
Train step of epoch 0:  33%|███▎      | 1830/5616 [23:20<48:37,  1.30it/s, gpt_loss=0.00384, lr=4.05e-6]
+
Train step of epoch 0:  33%|███▎      | 1830/5616 [23:20<48:37,  1.30it/s, gpt_loss=0.053, lr=4.05e-6]  
+
Train step of epoch 0:  33%|███▎      | 1831/5616 [23:20<47:32,  1.33it/s, gpt_loss=0.053, lr=4.05e-6]
+
Train step of epoch 0:  33%|███▎      | 1831/5616 [23:21<47:32,  1.33it/s, gpt_loss=0.0275, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1832/5616 [23:21<48:22,  1.30it/s, gpt_loss=0.0275, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1832/5616 [23:22<48:22,  1.30it/s, gpt_loss=0.13, lr=4.04e-6]  
+
Train step of epoch 0:  33%|███▎      | 1833/5616 [23:22<47:09,  1.34it/s, gpt_loss=0.13, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1833/5616 [23:23<47:09,  1.34it/s, gpt_loss=0.00656, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1834/5616 [23:23<45:51,  1.37it/s, gpt_loss=0.00656, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1834/5616 [23:23<45:51,  1.37it/s, gpt_loss=0.00269, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1835/5616 [23:23<45:17,  1.39it/s, gpt_loss=0.00269, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1835/5616 [23:24<45:17,  1.39it/s, gpt_loss=0.102, lr=4.04e-6]  
+
Train step of epoch 0:  33%|███▎      | 1836/5616 [23:24<46:00,  1.37it/s, gpt_loss=0.102, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1836/5616 [23:25<46:00,  1.37it/s, gpt_loss=0.00529, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1837/5616 [23:25<45:23,  1.39it/s, gpt_loss=0.00529, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1837/5616 [23:25<45:23,  1.39it/s, gpt_loss=0.031, lr=4.04e-6]  
+
Train step of epoch 0:  33%|███▎      | 1838/5616 [23:25<44:48,  1.41it/s, gpt_loss=0.031, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1838/5616 [23:26<44:48,  1.41it/s, gpt_loss=0.0525, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1839/5616 [23:26<44:00,  1.43it/s, gpt_loss=0.0525, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1839/5616 [23:27<44:00,  1.43it/s, gpt_loss=0.0748, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1840/5616 [23:27<48:12,  1.31it/s, gpt_loss=0.0748, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1840/5616 [23:28<48:12,  1.31it/s, gpt_loss=0.0549, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1841/5616 [23:28<48:42,  1.29it/s, gpt_loss=0.0549, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1841/5616 [23:29<48:42,  1.29it/s, gpt_loss=0.00444, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1842/5616 [23:29<49:46,  1.26it/s, gpt_loss=0.00444, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1842/5616 [23:29<49:46,  1.26it/s, gpt_loss=0.208, lr=4.04e-6]  
+
Train step of epoch 0:  33%|███▎      | 1843/5616 [23:29<48:57,  1.28it/s, gpt_loss=0.208, lr=4.04e-6]
+
Train step of epoch 0:  33%|███▎      | 1843/5616 [23:30<48:57,  1.28it/s, gpt_loss=0.0454, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1844/5616 [23:30<48:42,  1.29it/s, gpt_loss=0.0454, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1844/5616 [23:31<48:42,  1.29it/s, gpt_loss=0.0193, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1845/5616 [23:31<48:04,  1.31it/s, gpt_loss=0.0193, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1845/5616 [23:32<48:04,  1.31it/s, gpt_loss=0.0524, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1846/5616 [23:32<46:23,  1.35it/s, gpt_loss=0.0524, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1846/5616 [23:32<46:23,  1.35it/s, gpt_loss=0.064, lr=4.03e-6] 
+
Train step of epoch 0:  33%|███▎      | 1847/5616 [23:32<45:30,  1.38it/s, gpt_loss=0.064, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1847/5616 [23:33<45:30,  1.38it/s, gpt_loss=0.0237, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1848/5616 [23:33<45:53,  1.37it/s, gpt_loss=0.0237, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1848/5616 [23:34<45:53,  1.37it/s, gpt_loss=0.0687, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1849/5616 [23:34<44:14,  1.42it/s, gpt_loss=0.0687, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1849/5616 [23:35<44:14,  1.42it/s, gpt_loss=0.00762, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1850/5616 [23:35<51:31,  1.22it/s, gpt_loss=0.00762, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1850/5616 [23:35<51:31,  1.22it/s, gpt_loss=0.0203, lr=4.03e-6] 
+
Train step of epoch 0:  33%|███▎      | 1851/5616 [23:35<49:42,  1.26it/s, gpt_loss=0.0203, lr=4.03e-6]
+
Train step of epoch 0:  33%|███▎      | 1851/5616 [23:36<49:42,  1.26it/s, gpt_loss=0.0641, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1852/5616 [23:36<48:41,  1.29it/s, gpt_loss=0.0641, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1852/5616 [23:37<48:41,  1.29it/s, gpt_loss=0.107, lr=4.02e-6] 
+
Train step of epoch 0:  33%|███▎      | 1853/5616 [23:37<46:14,  1.36it/s, gpt_loss=0.107, lr=4.02e-6]
+
Train step of epoch 0:  33%|██���▎      | 1853/5616 [23:38<46:14,  1.36it/s, gpt_loss=0.198, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1854/5616 [23:38<48:11,  1.30it/s, gpt_loss=0.198, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1854/5616 [23:38<48:11,  1.30it/s, gpt_loss=0.0471, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1855/5616 [23:38<49:06,  1.28it/s, gpt_loss=0.0471, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1855/5616 [23:39<49:06,  1.28it/s, gpt_loss=0.0264, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1856/5616 [23:39<48:22,  1.30it/s, gpt_loss=0.0264, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1856/5616 [23:40<48:22,  1.30it/s, gpt_loss=0.0938, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1857/5616 [23:40<48:03,  1.30it/s, gpt_loss=0.0938, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1857/5616 [23:41<48:03,  1.30it/s, gpt_loss=0.0825, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1858/5616 [23:41<47:24,  1.32it/s, gpt_loss=0.0825, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1858/5616 [23:41<47:24,  1.32it/s, gpt_loss=0.0243, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1859/5616 [23:41<45:45,  1.37it/s, gpt_loss=0.0243, lr=4.02e-6]
+
Train step of epoch 0:  33%|███▎      | 1859/5616 [23:42<45:45,  1.37it/s, gpt_loss=0.0649, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1860/5616 [23:42<45:27,  1.38it/s, gpt_loss=0.0649, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1860/5616 [23:43<45:27,  1.38it/s, gpt_loss=0.0365, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1861/5616 [23:43<45:17,  1.38it/s, gpt_loss=0.0365, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1861/5616 [23:43<45:17,  1.38it/s, gpt_loss=0.0561, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1862/5616 [23:43<44:34,  1.40it/s, gpt_loss=0.0561, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1862/5616 [23:44<44:34,  1.40it/s, gpt_loss=0.0262, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1863/5616 [23:44<43:29,  1.44it/s, gpt_loss=0.0262, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1863/5616 [23:45<43:29,  1.44it/s, gpt_loss=0.0326, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1864/5616 [23:45<44:29,  1.41it/s, gpt_loss=0.0326, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1864/5616 [23:46<44:29,  1.41it/s, gpt_loss=0.0207, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1865/5616 [23:46<43:42,  1.43it/s, gpt_loss=0.0207, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1865/5616 [23:46<43:42,  1.43it/s, gpt_loss=0.0319, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1866/5616 [23:46<42:47,  1.46it/s, gpt_loss=0.0319, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1866/5616 [23:47<42:47,  1.46it/s, gpt_loss=0.0297, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1867/5616 [23:47<43:23,  1.44it/s, gpt_loss=0.0297, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1867/5616 [23:48<43:23,  1.44it/s, gpt_loss=0.124, lr=4.01e-6] 
+
Train step of epoch 0:  33%|███▎      | 1868/5616 [23:48<45:47,  1.36it/s, gpt_loss=0.124, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1868/5616 [23:49<45:47,  1.36it/s, gpt_loss=0.0231, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1869/5616 [23:49<46:55,  1.33it/s, gpt_loss=0.0231, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1869/5616 [23:49<46:55,  1.33it/s, gpt_loss=0.0902, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1870/5616 [23:49<45:26,  1.37it/s, gpt_loss=0.0902, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1870/5616 [23:50<45:26,  1.37it/s, gpt_loss=0.0453, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1871/5616 [23:50<44:48,  1.39it/s, gpt_loss=0.0453, lr=4.01e-6]
+
Train step of epoch 0:  33%|███▎      | 1871/5616 [23:51<44:48,  1.39it/s, gpt_loss=0.0291, lr=4e-6]   
+
Train step of epoch 0:  33%|███▎      | 1872/5616 [23:51<50:12,  1.24it/s, gpt_loss=0.0291, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1872/5616 [23:52<50:12,  1.24it/s, gpt_loss=0.0821, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1873/5616 [23:52<51:25,  1.21it/s, gpt_loss=0.0821, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1873/5616 [23:52<51:25,  1.21it/s, gpt_loss=0.0193, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1874/5616 [23:52<48:48,  1.28it/s, gpt_loss=0.0193, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1874/5616 [23:53<48:48,  1.28it/s, gpt_loss=0.0905, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1875/5616 [23:53<46:53,  1.33it/s, gpt_loss=0.0905, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1875/5616 [23:54<46:53,  1.33it/s, gpt_loss=0.0174, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1876/5616 [23:54<48:15,  1.29it/s, gpt_loss=0.0174, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1876/5616 [23:55<48:15,  1.29it/s, gpt_loss=0.0243, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1877/5616 [23:55<48:24,  1.29it/s, gpt_loss=0.0243, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1877/5616 [23:56<48:24,  1.29it/s, gpt_loss=0.0648, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1878/5616 [23:56<48:06,  1.30it/s, gpt_loss=0.0648, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1878/5616 [23:56<48:06,  1.30it/s, gpt_loss=0.0189, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1879/5616 [23:56<48:09,  1.29it/s, gpt_loss=0.0189, lr=4e-6]
+
Train step of epoch 0:  33%|███▎      | 1879/5616 [23:57<48:09,  1.29it/s, gpt_loss=0.0177, lr=3.99e-6]
+
Train step of epoch 0:  33%|███▎      | 1880/5616 [23:57<48:30,  1.28it/s, gpt_loss=0.0177, lr=3.99e-6]
+
Train step of epoch 0:  33%|███▎      | 1880/5616 [23:58<48:30,  1.28it/s, gpt_loss=0.0674, lr=3.99e-6]
+
Train step of epoch 0:  33%|███▎      | 1881/5616 [23:58<49:17,  1.26it/s, gpt_loss=0.0674, lr=3.99e-6]
+
Train step of epoch 0:  33%|███▎      | 1881/5616 [23:59<49:17,  1.26it/s, gpt_loss=0.0596, lr=3.99e-6]
+
Train step of epoch 0:  34%|███▎      | 1882/5616 [23:59<47:03,  1.32it/s, gpt_loss=0.0596, lr=3.99e-6]
+
Train step of epoch 0:  34%|███▎      | 1882/5616 [23:59<47:03,  1.32it/s, gpt_loss=0.0564, lr=3.99e-6]
+
Train step of epoch 0:  34%|███▎      | 1883/5616 [23:59<47:54,  1.30it/s, gpt_loss=0.0564, lr=3.99e-6]
+
Train step of epoch 0:  34%|███▎      | 1883/5616 [24:00<47:54,  1.30it/s, gpt_loss=0.0626, lr=3.99e-6]
+
Train step of epoch 0:  34%|███▎      | 1884/5616 [24:00<47:37,  1.31it/s, gpt_loss=0.0626, lr=3.99e-6]
+
Train step of epoch 0:  34%|███▎      | 1884/5616 [24:01<47:37,  1.31it/s, gpt_loss=0.0141, lr=3.99e-6]
+
Train step of epoch 0:  34%|███▎      | 1885/5616 [24:01<45:09,  1.38it/s, gpt_loss=0.0141, lr=3.99e-6]
+
Train step of epoch 0:  34%|███▎      | 1885/5616 [24:02<45:09,  1.38it/s, gpt_loss=0.0154, lr=3.99e-6]
+
Train step of epoch 0:  34%|███▎      | 1886/5616 [24:02<52:22,  1.19it/s, gpt_loss=0.0154, lr=3.99e-6]
+
Train step of epoch 0:  34%|███▎      | 1886/5616 [24:03<52:22,  1.19it/s, gpt_loss=0.0388, lr=3.99e-6]
+
Train step of epoch 0:  34%|███▎      | 1887/5616 [24:03<49:44,  1.25it/s, gpt_loss=0.0388, lr=3.99e-6]
+
Train step of epoch 0:  34%|███▎      | 1887/5616 [24:04<49:44,  1.25it/s, gpt_loss=0.0454, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1888/5616 [24:04<51:58,  1.20it/s, gpt_loss=0.0454, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1888/5616 [24:04<51:58,  1.20it/s, gpt_loss=0.0422, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1889/5616 [24:04<48:18,  1.29it/s, gpt_loss=0.0422, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1889/5616 [24:05<48:18,  1.29it/s, gpt_loss=0.0312, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1890/5616 [24:05<47:38,  1.30it/s, gpt_loss=0.0312, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1890/5616 [24:06<47:38,  1.30it/s, gpt_loss=0.0265, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1891/5616 [24:06<49:41,  1.25it/s, gpt_loss=0.0265, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1891/5616 [24:06<49:41,  1.25it/s, gpt_loss=0.0202, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1892/5616 [24:06<48:02,  1.29it/s, gpt_loss=0.0202, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1892/5616 [24:07<48:02,  1.29it/s, gpt_loss=0.0243, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1893/5616 [24:07<49:02,  1.27it/s, gpt_loss=0.0243, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1893/5616 [24:08<49:02,  1.27it/s, gpt_loss=0.0469, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1894/5616 [24:08<48:14,  1.29it/s, gpt_loss=0.0469, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1894/5616 [24:09<48:14,  1.29it/s, gpt_loss=0.0186, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1895/5616 [24:09<47:12,  1.31it/s, gpt_loss=0.0186, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▎      | 1895/5616 [24:10<47:12,  1.31it/s, gpt_loss=0.0516, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▍      | 1896/5616 [24:10<48:21,  1.28it/s, gpt_loss=0.0516, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▍      | 1896/5616 [24:11<48:21,  1.28it/s, gpt_loss=0.0277, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▍      | 1897/5616 [24:11<53:04,  1.17it/s, gpt_loss=0.0277, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▍      | 1897/5616 [24:11<53:04,  1.17it/s, gpt_loss=0.0173, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▍      | 1898/5616 [24:11<52:34,  1.18it/s, gpt_loss=0.0173, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▍      | 1898/5616 [24:12<52:34,  1.18it/s, gpt_loss=0.0214, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▍      | 1899/5616 [24:12<49:31,  1.25it/s, gpt_loss=0.0214, lr=3.98e-6]
+
Train step of epoch 0:  34%|███▍      | 1899/5616 [24:13<49:31,  1.25it/s, gpt_loss=0.157, lr=3.97e-6] 
+
Train step of epoch 0:  34%|███▍      | 1900/5616 [24:13<50:33,  1.23it/s, gpt_loss=0.157, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1900/5616 [24:14<50:33,  1.23it/s, gpt_loss=0.198, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1901/5616 [24:14<48:23,  1.28it/s, gpt_loss=0.198, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1901/5616 [24:15<48:23,  1.28it/s, gpt_loss=0.011, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1902/5616 [24:15<53:17,  1.16it/s, gpt_loss=0.011, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1902/5616 [24:15<53:17,  1.16it/s, gpt_loss=0.0638, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1903/5616 [24:15<48:56,  1.26it/s, gpt_loss=0.0638, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1903/5616 [24:16<48:56,  1.26it/s, gpt_loss=0.0341, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1904/5616 [24:16<51:05,  1.21it/s, gpt_loss=0.0341, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1904/5616 [24:17<51:05,  1.21it/s, gpt_loss=0.0346, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1905/5616 [24:17<49:51,  1.24it/s, gpt_loss=0.0346, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1905/5616 [24:18<49:51,  1.24it/s, gpt_loss=0.0578, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1906/5616 [24:18<47:36,  1.30it/s, gpt_loss=0.0578, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1906/5616 [24:18<47:36,  1.30it/s, gpt_loss=0.0709, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1907/5616 [24:18<45:50,  1.35it/s, gpt_loss=0.0709, lr=3.97e-6]
+
Train step of epoch 0:  34%|███▍      | 1907/5616 [24:19<45:50,  1.35it/s, gpt_loss=0.00737, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1908/5616 [24:19<46:09,  1.34it/s, gpt_loss=0.00737, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1908/5616 [24:20<46:09,  1.34it/s, gpt_loss=0.0152, lr=3.96e-6] 
+
Train step of epoch 0:  34%|███▍      | 1909/5616 [24:20<45:38,  1.35it/s, gpt_loss=0.0152, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1909/5616 [24:21<45:38,  1.35it/s, gpt_loss=0.0337, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1910/5616 [24:21<44:06,  1.40it/s, gpt_loss=0.0337, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1910/5616 [24:21<44:06,  1.40it/s, gpt_loss=0.0704, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1911/5616 [24:21<46:38,  1.32it/s, gpt_loss=0.0704, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1911/5616 [24:22<46:38,  1.32it/s, gpt_loss=0.0144, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1912/5616 [24:22<46:39,  1.32it/s, gpt_loss=0.0144, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1912/5616 [24:23<46:39,  1.32it/s, gpt_loss=0.0915, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1913/5616 [24:23<45:47,  1.35it/s, gpt_loss=0.0915, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1913/5616 [24:24<45:47,  1.35it/s, gpt_loss=0.0163, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1914/5616 [24:24<44:28,  1.39it/s, gpt_loss=0.0163, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1914/5616 [24:24<44:28,  1.39it/s, gpt_loss=0.0571, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1915/5616 [24:24<45:07,  1.37it/s, gpt_loss=0.0571, lr=3.96e-6]
+
Train step of epoch 0:  34%|███▍      | 1915/5616 [24:25<45:07,  1.37it/s, gpt_loss=0.116, lr=3.95e-6] 
+
Train step of epoch 0:  34%|███▍      | 1916/5616 [24:25<47:23,  1.30it/s, gpt_loss=0.116, lr=3.95e-6]
+
Train step of epoch 0:  34%|███▍      | 1916/5616 [24:26<47:23,  1.30it/s, gpt_loss=0.02, lr=3.95e-6] 
+
Train step of epoch 0:  34%|███▍      | 1917/5616 [24:26<47:42,  1.29it/s, gpt_loss=0.02, lr=3.95e-6]
+
Train step of epoch 0:  34%|███▍      | 1917/5616 [24:27<47:42,  1.29it/s, gpt_loss=0.12, lr=3.95e-6]
+
Train step of epoch 0:  34%|███▍      | 1918/5616 [24:27<45:58,  1.34it/s, gpt_loss=0.12, lr=3.95e-6]
+
Train step of epoch 0:  34%|███▍      | 1918/5616 [24:27<45:58,  1.34it/s, gpt_loss=0.066, lr=3.95e-6]
+
Train step of epoch 0:  34%|███▍      | 1919/5616 [24:27<44:46,  1.38it/s, gpt_loss=0.066, lr=3.95e-6]
+
Train step of epoch 0:  34%|███▍      | 1919/5616 [24:28<44:46,  1.38it/s, gpt_loss=0.0647, lr=3.95e-6]
+
Train step of epoch 0:  34%|███▍      | 1920/5616 [24:28<45:36,  1.35it/s, gpt_loss=0.0647, lr=3.95e-6]
+
Train step of epoch 0:  34%|███▍      | 1920/5616 [24:29<45:36,  1.35it/s, gpt_loss=0.0122, lr=3.95e-6]
+
Train step of epoch 0:  34%|███▍      | 1921/5616 [24:29<45:14,  1.36it/s, gpt_loss=0.0122, lr=3.95e-6]
+
Train step of epoch 0:  34%|███▍      | 1921/5616 [24:30<45:14,  1.36it/s, gpt_loss=0.139, lr=3.95e-6] 
+
Train step of epoch 0:  34%|███▍      | 1922/5616 [24:30<48:04,  1.28it/s, gpt_loss=0.139, lr=3.95e-6]
+
Train step of epoch 0:  34%|███▍      | 1922/5616 [24:30<48:04,  1.28it/s, gpt_loss=0.0891, lr=3.95e-6]
+
Train step of epoch 0:  34%|███▍      | 1923/5616 [24:30<46:28,  1.32it/s, gpt_loss=0.0891, lr=3.95e-6]
+
Train step of epoch 0:  34%|███▍      | 1923/5616 [24:31<46:28,  1.32it/s, gpt_loss=0.0167, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1924/5616 [24:31<46:21,  1.33it/s, gpt_loss=0.0167, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1924/5616 [24:32<46:21,  1.33it/s, gpt_loss=0.014, lr=3.94e-6] 
+
Train step of epoch 0:  34%|███▍      | 1925/5616 [24:32<44:12,  1.39it/s, gpt_loss=0.014, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1925/5616 [24:33<44:12,  1.39it/s, gpt_loss=0.15, lr=3.94e-6] 
+
Train step of epoch 0:  34%|███▍      | 1926/5616 [24:33<49:07,  1.25it/s, gpt_loss=0.15, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1926/5616 [24:34<49:07,  1.25it/s, gpt_loss=0.00819, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1927/5616 [24:34<48:57,  1.26it/s, gpt_loss=0.00819, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1927/5616 [24:34<48:57,  1.26it/s, gpt_loss=0.0401, lr=3.94e-6] 
+
Train step of epoch 0:  34%|███▍      | 1928/5616 [24:34<49:31,  1.24it/s, gpt_loss=0.0401, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1928/5616 [24:35<49:31,  1.24it/s, gpt_loss=0.00257, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1929/5616 [24:35<49:33,  1.24it/s, gpt_loss=0.00257, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1929/5616 [24:36<49:33,  1.24it/s, gpt_loss=0.0126, lr=3.94e-6] 
+
Train step of epoch 0:  34%|███▍      | 1930/5616 [24:36<47:11,  1.30it/s, gpt_loss=0.0126, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1930/5616 [24:37<47:11,  1.30it/s, gpt_loss=0.0302, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1931/5616 [24:37<47:40,  1.29it/s, gpt_loss=0.0302, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1931/5616 [24:38<47:40,  1.29it/s, gpt_loss=0.101, lr=3.94e-6] 
+
Train step of epoch 0:  34%|███▍      | 1932/5616 [24:38<50:31,  1.22it/s, gpt_loss=0.101, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1932/5616 [24:38<50:31,  1.22it/s, gpt_loss=0.00647, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1933/5616 [24:38<48:16,  1.27it/s, gpt_loss=0.00647, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1933/5616 [24:39<48:16,  1.27it/s, gpt_loss=0.0415, lr=3.94e-6] 
+
Train step of epoch 0:  34%|███▍      | 1934/5616 [24:39<45:01,  1.36it/s, gpt_loss=0.0415, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1934/5616 [24:40<45:01,  1.36it/s, gpt_loss=0.0495, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1935/5616 [24:40<43:51,  1.40it/s, gpt_loss=0.0495, lr=3.94e-6]
+
Train step of epoch 0:  34%|███▍      | 1935/5616 [24:41<43:51,  1.40it/s, gpt_loss=0.0814, lr=3.93e-6]
+
Train step of epoch 0:  34%|███▍      | 1936/5616 [24:41<48:18,  1.27it/s, gpt_loss=0.0814, lr=3.93e-6]
+
Train step of epoch 0:  34%|███▍      | 1936/5616 [24:41<48:18,  1.27it/s, gpt_loss=0.0328, lr=3.93e-6]
+
Train step of epoch 0:  34%|███▍      | 1937/5616 [24:41<46:25,  1.32it/s, gpt_loss=0.0328, lr=3.93e-6]
+
Train step of epoch 0:  34%|███▍      | 1937/5616 [24:42<46:25,  1.32it/s, gpt_loss=0.0113, lr=3.93e-6]
+
Train step of epoch 0:  35%|███▍      | 1938/5616 [24:42<44:49,  1.37it/s, gpt_loss=0.0113, lr=3.93e-6]
+
Train step of epoch 0:  35%|███▍      | 1938/5616 [24:43<44:49,  1.37it/s, gpt_loss=0.051, lr=3.93e-6] 
+
Train step of epoch 0:  35%|███▍      | 1939/5616 [24:43<46:50,  1.31it/s, gpt_loss=0.051, lr=3.93e-6]
+
Train step of epoch 0:  35%|███▍      | 1939/5616 [24:44<46:50,  1.31it/s, gpt_loss=0.0461, lr=3.93e-6]
+
Train step of epoch 0:  35%|███▍      | 1940/5616 [24:44<48:46,  1.26it/s, gpt_loss=0.0461, lr=3.93e-6]
+
Train step of epoch 0:  35%|███▍      | 1940/5616 [24:44<48:46,  1.26it/s, gpt_loss=0.0496, lr=3.93e-6]
+
Train step of epoch 0:  35%|███▍      | 1941/5616 [24:44<48:05,  1.27it/s, gpt_loss=0.0496, lr=3.93e-6]
+
Train step of epoch 0:  35%|███▍      | 1941/5616 [24:45<48:05,  1.27it/s, gpt_loss=0.0213, lr=3.93e-6]
+
Train step of epoch 0:  35%|███▍      | 1942/5616 [24:45<45:41,  1.34it/s, gpt_loss=0.0213, lr=3.93e-6]
+
Train step of epoch 0:  35%|███▍      | 1942/5616 [24:46<45:41,  1.34it/s, gpt_loss=0.0425, lr=3.93e-6]
+
Train step of epoch 0:  35%|███▍      | 1943/5616 [24:46<44:51,  1.36it/s, gpt_loss=0.0425, lr=3.93e-6]
+
Train step of epoch 0:  35%|███▍      | 1943/5616 [24:46<44:51,  1.36it/s, gpt_loss=0.107, lr=3.92e-6] 
+
Train step of epoch 0:  35%|███▍      | 1944/5616 [24:46<45:07,  1.36it/s, gpt_loss=0.107, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1944/5616 [24:47<45:07,  1.36it/s, gpt_loss=0.0782, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1945/5616 [24:47<44:26,  1.38it/s, gpt_loss=0.0782, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1945/5616 [24:48<44:26,  1.38it/s, gpt_loss=0.0319, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1946/5616 [24:48<43:58,  1.39it/s, gpt_loss=0.0319, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1946/5616 [24:49<43:58,  1.39it/s, gpt_loss=0.00623, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1947/5616 [24:49<44:10,  1.38it/s, gpt_loss=0.00623, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1947/5616 [24:49<44:10,  1.38it/s, gpt_loss=0.0545, lr=3.92e-6] 
+
Train step of epoch 0:  35%|███▍      | 1948/5616 [24:49<44:48,  1.36it/s, gpt_loss=0.0545, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1948/5616 [24:50<44:48,  1.36it/s, gpt_loss=0.0265, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1949/5616 [24:50<44:07,  1.38it/s, gpt_loss=0.0265, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1949/5616 [24:51<44:07,  1.38it/s, gpt_loss=0.0554, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1950/5616 [24:51<46:53,  1.30it/s, gpt_loss=0.0554, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1950/5616 [24:52<46:53,  1.30it/s, gpt_loss=0.0165, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1951/5616 [24:52<46:39,  1.31it/s, gpt_loss=0.0165, lr=3.92e-6]
+
Train step of epoch 0:  35%|███▍      | 1951/5616 [24:53<46:39,  1.31it/s, gpt_loss=0.0223, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1952/5616 [24:53<48:03,  1.27it/s, gpt_loss=0.0223, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1952/5616 [24:53<48:03,  1.27it/s, gpt_loss=0.0896, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1953/5616 [24:53<47:37,  1.28it/s, gpt_loss=0.0896, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1953/5616 [24:54<47:37,  1.28it/s, gpt_loss=0.0711, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1954/5616 [24:54<47:21,  1.29it/s, gpt_loss=0.0711, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1954/5616 [24:55<47:21,  1.29it/s, gpt_loss=0.0147, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1955/5616 [24:55<45:37,  1.34it/s, gpt_loss=0.0147, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1955/5616 [24:56<45:37,  1.34it/s, gpt_loss=0.0553, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1956/5616 [24:56<47:06,  1.29it/s, gpt_loss=0.0553, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1956/5616 [24:56<47:06,  1.29it/s, gpt_loss=0.00468, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1957/5616 [24:56<45:53,  1.33it/s, gpt_loss=0.00468, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1957/5616 [24:57<45:53,  1.33it/s, gpt_loss=0.109, lr=3.91e-6]  
+
Train step of epoch 0:  35%|███▍      | 1958/5616 [24:57<46:20,  1.32it/s, gpt_loss=0.109, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1958/5616 [24:58<46:20,  1.32it/s, gpt_loss=0.0346, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1959/5616 [24:58<45:13,  1.35it/s, gpt_loss=0.0346, lr=3.91e-6]
+
Train step of epoch 0:  35%|███▍      | 1959/5616 [24:59<45:13,  1.35it/s, gpt_loss=0.0199, lr=3.9e-6] 
+
Train step of epoch 0:  35%|███▍      | 1960/5616 [24:59<48:16,  1.26it/s, gpt_loss=0.0199, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▍      | 1960/5616 [24:59<48:16,  1.26it/s, gpt_loss=0.0554, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▍      | 1961/5616 [24:59<47:50,  1.27it/s, gpt_loss=0.0554, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▍      | 1961/5616 [25:00<47:50,  1.27it/s, gpt_loss=0.059, lr=3.9e-6] 
+
Train step of epoch 0:  35%|███▍      | 1962/5616 [25:00<46:03,  1.32it/s, gpt_loss=0.059, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▍      | 1962/5616 [25:01<46:03,  1.32it/s, gpt_loss=0.0282, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▍      | 1963/5616 [25:01<46:02,  1.32it/s, gpt_loss=0.0282, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▍      | 1963/5616 [25:02<46:02,  1.32it/s, gpt_loss=0.0838, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▍      | 1964/5616 [25:02<46:06,  1.32it/s, gpt_loss=0.0838, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▍      | 1964/5616 [25:02<46:06,  1.32it/s, gpt_loss=0.125, lr=3.9e-6] 
+
Train step of epoch 0:  35%|███▍      | 1965/5616 [25:02<45:37,  1.33it/s, gpt_loss=0.125, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▍      | 1965/5616 [25:03<45:37,  1.33it/s, gpt_loss=0.209, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▌      | 1966/5616 [25:03<44:43,  1.36it/s, gpt_loss=0.209, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▌      | 1966/5616 [25:04<44:43,  1.36it/s, gpt_loss=0.0134, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▌      | 1967/5616 [25:04<43:34,  1.40it/s, gpt_loss=0.0134, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▌      | 1967/5616 [25:05<43:34,  1.40it/s, gpt_loss=0.0228, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▌      | 1968/5616 [25:05<44:54,  1.35it/s, gpt_loss=0.0228, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▌      | 1968/5616 [25:05<44:54,  1.35it/s, gpt_loss=0.0315, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▌      | 1969/5616 [25:05<43:59,  1.38it/s, gpt_loss=0.0315, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▌      | 1969/5616 [25:06<43:59,  1.38it/s, gpt_loss=0.0401, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▌      | 1970/5616 [25:06<45:15,  1.34it/s, gpt_loss=0.0401, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▌      | 1970/5616 [25:07<45:15,  1.34it/s, gpt_loss=0.0637, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▌      | 1971/5616 [25:07<44:45,  1.36it/s, gpt_loss=0.0637, lr=3.9e-6]
+
Train step of epoch 0:  35%|███▌      | 1971/5616 [25:07<44:45,  1.36it/s, gpt_loss=0.0411, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1972/5616 [25:07<45:19,  1.34it/s, gpt_loss=0.0411, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1972/5616 [25:08<45:19,  1.34it/s, gpt_loss=0.0569, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1973/5616 [25:08<45:42,  1.33it/s, gpt_loss=0.0569, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1973/5616 [25:09<45:42,  1.33it/s, gpt_loss=0.0199, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1974/5616 [25:09<46:59,  1.29it/s, gpt_loss=0.0199, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1974/5616 [25:10<46:59,  1.29it/s, gpt_loss=0.0516, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1975/5616 [25:10<45:27,  1.33it/s, gpt_loss=0.0516, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1975/5616 [25:11<45:27,  1.33it/s, gpt_loss=0.0164, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1976/5616 [25:11<45:50,  1.32it/s, gpt_loss=0.0164, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1976/5616 [25:11<45:50,  1.32it/s, gpt_loss=0.04, lr=3.89e-6]  
+
Train step of epoch 0:  35%|███▌      | 1977/5616 [25:11<46:47,  1.30it/s, gpt_loss=0.04, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1977/5616 [25:12<46:47,  1.30it/s, gpt_loss=0.0176, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1978/5616 [25:12<47:24,  1.28it/s, gpt_loss=0.0176, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1978/5616 [25:13<47:24,  1.28it/s, gpt_loss=0.11, lr=3.89e-6]  
+
Train step of epoch 0:  35%|███▌      | 1979/5616 [25:13<45:25,  1.33it/s, gpt_loss=0.11, lr=3.89e-6]
+
Train step of epoch 0:  35%|███▌      | 1979/5616 [25:14<45:25,  1.33it/s, gpt_loss=0.039, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1980/5616 [25:14<48:22,  1.25it/s, gpt_loss=0.039, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1980/5616 [25:14<48:22,  1.25it/s, gpt_loss=0.0448, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1981/5616 [25:14<46:28,  1.30it/s, gpt_loss=0.0448, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1981/5616 [25:15<46:28,  1.30it/s, gpt_loss=0.00822, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1982/5616 [25:15<44:33,  1.36it/s, gpt_loss=0.00822, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1982/5616 [25:16<44:33,  1.36it/s, gpt_loss=0.0123, lr=3.88e-6] 
+
Train step of epoch 0:  35%|███▌      | 1983/5616 [25:16<47:51,  1.27it/s, gpt_loss=0.0123, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1983/5616 [25:17<47:51,  1.27it/s, gpt_loss=0.0484, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1984/5616 [25:17<48:16,  1.25it/s, gpt_loss=0.0484, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1984/5616 [25:18<48:16,  1.25it/s, gpt_loss=0.0673, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1985/5616 [25:18<48:52,  1.24it/s, gpt_loss=0.0673, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1985/5616 [25:18<48:52,  1.24it/s, gpt_loss=0.0648, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1986/5616 [25:18<47:45,  1.27it/s, gpt_loss=0.0648, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1986/5616 [25:19<47:45,  1.27it/s, gpt_loss=0.00913, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1987/5616 [25:19<47:23,  1.28it/s, gpt_loss=0.00913, lr=3.88e-6]
+
Train step of epoch 0:  35%|███▌      | 1987/5616 [25:20<47:23,  1.28it/s, gpt_loss=0.0361, lr=3.87e-6] 
+
Train step of epoch 0:  35%|███▌      | 1988/5616 [25:20<46:14,  1.31it/s, gpt_loss=0.0361, lr=3.87e-6]
+
Train step of epoch 0:  35%|███▌      | 1988/5616 [25:21<46:14,  1.31it/s, gpt_loss=0.0403, lr=3.87e-6]
+
Train step of epoch 0:  35%|███▌      | 1989/5616 [25:21<45:09,  1.34it/s, gpt_loss=0.0403, lr=3.87e-6]
+
Train step of epoch 0:  35%|███▌      | 1989/5616 [25:21<45:09,  1.34it/s, gpt_loss=0.0184, lr=3.87e-6]
+
Train step of epoch 0:  35%|███▌      | 1990/5616 [25:21<44:53,  1.35it/s, gpt_loss=0.0184, lr=3.87e-6]
+
Train step of epoch 0:  35%|███▌      | 1990/5616 [25:22<44:53,  1.35it/s, gpt_loss=0.0205, lr=3.87e-6]
+
Train step of epoch 0:  35%|███▌      | 1991/5616 [25:22<46:47,  1.29it/s, gpt_loss=0.0205, lr=3.87e-6]
+
Train step of epoch 0:  35%|███▌      | 1991/5616 [25:23<46:47,  1.29it/s, gpt_loss=0.0812, lr=3.87e-6]
+
Train step of epoch 0:  35%|███▌      | 1992/5616 [25:23<46:56,  1.29it/s, gpt_loss=0.0812, lr=3.87e-6]
+
Train step of epoch 0:  35%|███▌      | 1992/5616 [25:24<46:56,  1.29it/s, gpt_loss=0.108, lr=3.87e-6] 
+
Train step of epoch 0:  35%|███▌      | 1993/5616 [25:24<46:19,  1.30it/s, gpt_loss=0.108, lr=3.87e-6]
+
Train step of epoch 0:  35%|███▌      | 1993/5616 [25:25<46:19,  1.30it/s, gpt_loss=0.0233, lr=3.87e-6]
+
Train step of epoch 0:  36%|███▌      | 1994/5616 [25:25<50:58,  1.18it/s, gpt_loss=0.0233, lr=3.87e-6]
+
Train step of epoch 0:  36%|███▌      | 1994/5616 [25:25<50:58,  1.18it/s, gpt_loss=0.011, lr=3.87e-6] 
+
Train step of epoch 0:  36%|███▌      | 1995/5616 [25:25<48:53,  1.23it/s, gpt_loss=0.011, lr=3.87e-6]
+
Train step of epoch 0:  36%|███▌      | 1995/5616 [25:26<48:53,  1.23it/s, gpt_loss=0.0375, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 1996/5616 [25:26<48:19,  1.25it/s, gpt_loss=0.0375, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 1996/5616 [25:27<48:19,  1.25it/s, gpt_loss=0.0426, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 1997/5616 [25:27<47:04,  1.28it/s, gpt_loss=0.0426, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 1997/5616 [25:28<47:04,  1.28it/s, gpt_loss=0.0789, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 1998/5616 [25:28<52:21,  1.15it/s, gpt_loss=0.0789, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 1998/5616 [25:29<52:21,  1.15it/s, gpt_loss=0.0417, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 1999/5616 [25:29<50:15,  1.20it/s, gpt_loss=0.0417, lr=3.86e-6][2025-06-30 07:03:38,675] [INFO] [logging.py:128:log_dist] [Rank 0] step=500, skipped=0, lr=[3.859974015726769e-06, 3.859974015726769e-06], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 07:03:38,676] [INFO] [timer.py:264:stop] epoch=0/micro_step=2000/global_step=500, RunningAvgSamplesPerSec=42.408925141922865, CurrSamplesPerSec=38.236029477067184, MemAllocated=15.82GB, MaxMemAllocated=51.79GB
+
+
Train step of epoch 0:  36%|███▌      | 1999/5616 [25:30<50:15,  1.20it/s, gpt_loss=0.0196, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2000/5616 [25:30<50:04,  1.20it/s, gpt_loss=0.0196, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2000/5616 [25:30<50:04,  1.20it/s, gpt_loss=0.00825, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2001/5616 [25:30<47:37,  1.27it/s, gpt_loss=0.00825, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2001/5616 [25:31<47:37,  1.27it/s, gpt_loss=0.0113, lr=3.86e-6] 
+
Train step of epoch 0:  36%|███▌      | 2002/5616 [25:31<48:16,  1.25it/s, gpt_loss=0.0113, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2002/5616 [25:32<48:16,  1.25it/s, gpt_loss=0.0263, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2003/5616 [25:32<45:46,  1.32it/s, gpt_loss=0.0263, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2003/5616 [25:33<45:46,  1.32it/s, gpt_loss=0.0586, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2004/5616 [25:33<47:22,  1.27it/s, gpt_loss=0.0586, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2004/5616 [25:33<47:22,  1.27it/s, gpt_loss=0.0475, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2005/5616 [25:33<47:47,  1.26it/s, gpt_loss=0.0475, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2005/5616 [25:34<47:47,  1.26it/s, gpt_loss=0.00659, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2006/5616 [25:34<45:19,  1.33it/s, gpt_loss=0.00659, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2006/5616 [25:35<45:19,  1.33it/s, gpt_loss=0.107, lr=3.86e-6]  
+
Train step of epoch 0:  36%|███▌      | 2007/5616 [25:35<43:19,  1.39it/s, gpt_loss=0.107, lr=3.86e-6]
+
Train step of epoch 0:  36%|███▌      | 2007/5616 [25:36<43:19,  1.39it/s, gpt_loss=0.0281, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2008/5616 [25:36<53:25,  1.13it/s, gpt_loss=0.0281, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2008/5616 [25:37<53:25,  1.13it/s, gpt_loss=0.0633, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2009/5616 [25:37<49:12,  1.22it/s, gpt_loss=0.0633, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2009/5616 [25:38<49:12,  1.22it/s, gpt_loss=0.0537, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2010/5616 [25:38<54:24,  1.10it/s, gpt_loss=0.0537, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2010/5616 [25:38<54:24,  1.10it/s, gpt_loss=0.0388, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2011/5616 [25:38<49:34,  1.21it/s, gpt_loss=0.0388, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2011/5616 [25:39<49:34,  1.21it/s, gpt_loss=0.0232, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2012/5616 [25:39<49:17,  1.22it/s, gpt_loss=0.0232, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2012/5616 [25:40<49:17,  1.22it/s, gpt_loss=0.0108, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2013/5616 [25:40<47:34,  1.26it/s, gpt_loss=0.0108, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2013/5616 [25:41<47:34,  1.26it/s, gpt_loss=0.025, lr=3.85e-6] 
+
Train step of epoch 0:  36%|███▌      | 2014/5616 [25:41<45:25,  1.32it/s, gpt_loss=0.025, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2014/5616 [25:42<45:25,  1.32it/s, gpt_loss=0.0842, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2015/5616 [25:42<52:00,  1.15it/s, gpt_loss=0.0842, lr=3.85e-6]
+
Train step of epoch 0:  36%|███▌      | 2015/5616 [25:43<52:00,  1.15it/s, gpt_loss=0.031, lr=3.84e-6] 
+
Train step of epoch 0:  36%|███▌      | 2016/5616 [25:43<51:06,  1.17it/s, gpt_loss=0.031, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2016/5616 [25:43<51:06,  1.17it/s, gpt_loss=0.0168, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2017/5616 [25:43<48:50,  1.23it/s, gpt_loss=0.0168, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2017/5616 [25:44<48:50,  1.23it/s, gpt_loss=0.0651, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2018/5616 [25:44<47:54,  1.25it/s, gpt_loss=0.0651, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2018/5616 [25:45<47:54,  1.25it/s, gpt_loss=0.0293, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2019/5616 [25:45<47:21,  1.27it/s, gpt_loss=0.0293, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2019/5616 [25:46<47:21,  1.27it/s, gpt_loss=0.0272, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2020/5616 [25:46<46:54,  1.28it/s, gpt_loss=0.0272, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2020/5616 [25:46<46:54,  1.28it/s, gpt_loss=0.0412, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2021/5616 [25:46<46:01,  1.30it/s, gpt_loss=0.0412, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2021/5616 [25:47<46:01,  1.30it/s, gpt_loss=0.00959, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2022/5616 [25:47<46:08,  1.30it/s, gpt_loss=0.00959, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2022/5616 [25:48<46:08,  1.30it/s, gpt_loss=0.0918, lr=3.84e-6] 
+
Train step of epoch 0:  36%|███▌      | 2023/5616 [25:48<47:43,  1.25it/s, gpt_loss=0.0918, lr=3.84e-6]
+
Train step of epoch 0:  36%|███▌      | 2023/5616 [25:49<47:43,  1.25it/s, gpt_loss=0.062, lr=3.83e-6] 
+
Train step of epoch 0:  36%|███▌      | 2024/5616 [25:49<46:46,  1.28it/s, gpt_loss=0.062, lr=3.83e-6]
+
Train step of epoch 0:  36%|███▌      | 2024/5616 [25:49<46:46,  1.28it/s, gpt_loss=0.0474, lr=3.83e-6]
+
Train step of epoch 0:  36%|███▌      | 2025/5616 [25:49<45:04,  1.33it/s, gpt_loss=0.0474, lr=3.83e-6]
+
Train step of epoch 0:  36%|███▌      | 2025/5616 [25:50<45:04,  1.33it/s, gpt_loss=0.0184, lr=3.83e-6]
+
Train step of epoch 0:  36%|███▌      | 2026/5616 [25:50<48:31,  1.23it/s, gpt_loss=0.0184, lr=3.83e-6]
+
Train step of epoch 0:  36%|███▌      | 2026/5616 [25:51<48:31,  1.23it/s, gpt_loss=0.0705, lr=3.83e-6]
+
Train step of epoch 0:  36%|███▌      | 2027/5616 [25:51<45:34,  1.31it/s, gpt_loss=0.0705, lr=3.83e-6]
+
Train step of epoch 0:  36%|███▌      | 2027/5616 [25:52<45:34,  1.31it/s, gpt_loss=0.0219, lr=3.83e-6]
+
Train step of epoch 0:  36%|███▌      | 2028/5616 [25:52<48:57,  1.22it/s, gpt_loss=0.0219, lr=3.83e-6]
+
Train step of epoch 0:  36%|███▌      | 2028/5616 [25:53<48:57,  1.22it/s, gpt_loss=0.14, lr=3.83e-6]  
+
Train step of epoch 0:  36%|███▌      | 2029/5616 [25:53<46:53,  1.27it/s, gpt_loss=0.14, lr=3.83e-6]
+
Train step of epoch 0:  36%|███▌      | 2029/5616 [25:53<46:53,  1.27it/s, gpt_loss=0.0396, lr=3.83e-6]
+
Train step of epoch 0:  36%|███▌      | 2030/5616 [25:53<44:35,  1.34it/s, gpt_loss=0.0396, lr=3.83e-6]
+
Train step of epoch 0:  36%|███▌      | 2030/5616 [25:54<44:35,  1.34it/s, gpt_loss=0.11, lr=3.83e-6]  
+
Train step of epoch 0:  36%|███▌      | 2031/5616 [25:54<43:42,  1.37it/s, gpt_loss=0.11, lr=3.83e-6]
+
Train step of epoch 0:  36%|███▌      | 2031/5616 [25:55<43:42,  1.37it/s, gpt_loss=0.0317, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▌      | 2032/5616 [25:55<45:02,  1.33it/s, gpt_loss=0.0317, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▌      | 2032/5616 [25:56<45:02,  1.33it/s, gpt_loss=0.0385, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▌      | 2033/5616 [25:56<47:26,  1.26it/s, gpt_loss=0.0385, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▌      | 2033/5616 [25:57<47:26,  1.26it/s, gpt_loss=0.0757, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▌      | 2034/5616 [25:57<48:06,  1.24it/s, gpt_loss=0.0757, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▌      | 2034/5616 [25:57<48:06,  1.24it/s, gpt_loss=0.15, lr=3.82e-6]  
+
Train step of epoch 0:  36%|███▌      | 2035/5616 [25:57<48:34,  1.23it/s, gpt_loss=0.15, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▌      | 2035/5616 [25:58<48:34,  1.23it/s, gpt_loss=0.125, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▋      | 2036/5616 [25:58<47:30,  1.26it/s, gpt_loss=0.125, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▋      | 2036/5616 [25:59<47:30,  1.26it/s, gpt_loss=0.167, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▋      | 2037/5616 [25:59<47:14,  1.26it/s, gpt_loss=0.167, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▋      | 2037/5616 [26:00<47:14,  1.26it/s, gpt_loss=0.0626, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▋      | 2038/5616 [26:00<44:30,  1.34it/s, gpt_loss=0.0626, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▋      | 2038/5616 [26:00<44:30,  1.34it/s, gpt_loss=0.0578, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▋      | 2039/5616 [26:00<43:03,  1.38it/s, gpt_loss=0.0578, lr=3.82e-6]
+
Train step of epoch 0:  36%|███▋      | 2039/5616 [26:01<43:03,  1.38it/s, gpt_loss=0.116, lr=3.81e-6] 
+
Train step of epoch 0:  36%|███▋      | 2040/5616 [26:01<43:01,  1.39it/s, gpt_loss=0.116, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2040/5616 [26:02<43:01,  1.39it/s, gpt_loss=0.0539, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2041/5616 [26:02<42:44,  1.39it/s, gpt_loss=0.0539, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2041/5616 [26:02<42:44,  1.39it/s, gpt_loss=0.0341, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2042/5616 [26:02<42:29,  1.40it/s, gpt_loss=0.0341, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2042/5616 [26:03<42:29,  1.40it/s, gpt_loss=0.0206, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2043/5616 [26:03<42:07,  1.41it/s, gpt_loss=0.0206, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2043/5616 [26:04<42:07,  1.41it/s, gpt_loss=0.0233, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2044/5616 [26:04<42:23,  1.40it/s, gpt_loss=0.0233, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2044/5616 [26:05<42:23,  1.40it/s, gpt_loss=0.0403, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2045/5616 [26:05<42:15,  1.41it/s, gpt_loss=0.0403, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2045/5616 [26:05<42:15,  1.41it/s, gpt_loss=0.0201, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2046/5616 [26:05<43:23,  1.37it/s, gpt_loss=0.0201, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2046/5616 [26:06<43:23,  1.37it/s, gpt_loss=0.015, lr=3.81e-6] 
+
Train step of epoch 0:  36%|███▋      | 2047/5616 [26:06<42:19,  1.41it/s, gpt_loss=0.015, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2047/5616 [26:07<42:19,  1.41it/s, gpt_loss=0.0471, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2048/5616 [26:07<47:10,  1.26it/s, gpt_loss=0.0471, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2048/5616 [26:08<47:10,  1.26it/s, gpt_loss=0.0102, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2049/5616 [26:08<44:59,  1.32it/s, gpt_loss=0.0102, lr=3.81e-6]
+
Train step of epoch 0:  36%|███▋      | 2049/5616 [26:08<44:59,  1.32it/s, gpt_loss=0.0418, lr=3.81e-6]
+
Train step of epoch 0:  37%|███▋      | 2050/5616 [26:08<47:12,  1.26it/s, gpt_loss=0.0418, lr=3.81e-6]
+
Train step of epoch 0:  37%|███▋      | 2050/5616 [26:09<47:12,  1.26it/s, gpt_loss=0.0187, lr=3.81e-6]
+
Train step of epoch 0:  37%|███▋      | 2051/5616 [26:09<44:51,  1.32it/s, gpt_loss=0.0187, lr=3.81e-6]
+
Train step of epoch 0:  37%|███▋      | 2051/5616 [26:10<44:51,  1.32it/s, gpt_loss=0.0424, lr=3.8e-6] 
+
Train step of epoch 0:  37%|███▋      | 2052/5616 [26:10<45:17,  1.31it/s, gpt_loss=0.0424, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2052/5616 [26:11<45:17,  1.31it/s, gpt_loss=0.0564, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2053/5616 [26:11<44:16,  1.34it/s, gpt_loss=0.0564, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2053/5616 [26:11<44:16,  1.34it/s, gpt_loss=0.0959, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2054/5616 [26:11<43:49,  1.35it/s, gpt_loss=0.0959, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2054/5616 [26:12<43:49,  1.35it/s, gpt_loss=0.0145, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2055/5616 [26:12<45:22,  1.31it/s, gpt_loss=0.0145, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2055/5616 [26:13<45:22,  1.31it/s, gpt_loss=0.0321, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2056/5616 [26:13<45:12,  1.31it/s, gpt_loss=0.0321, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2056/5616 [26:14<45:12,  1.31it/s, gpt_loss=0.121, lr=3.8e-6] 
+
Train step of epoch 0:  37%|███▋      | 2057/5616 [26:14<43:10,  1.37it/s, gpt_loss=0.121, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2057/5616 [26:15<43:10,  1.37it/s, gpt_loss=0.066, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2058/5616 [26:15<46:29,  1.28it/s, gpt_loss=0.066, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2058/5616 [26:15<46:29,  1.28it/s, gpt_loss=0.0112, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2059/5616 [26:15<45:21,  1.31it/s, gpt_loss=0.0112, lr=3.8e-6]
+
Train step of epoch 0:  37%|███▋      | 2059/5616 [26:16<45:21,  1.31it/s, gpt_loss=0.00882, lr=3.79e-6]
+
Train step of epoch 0:  37%|███▋      | 2060/5616 [26:16<45:21,  1.31it/s, gpt_loss=0.00882, lr=3.79e-6]
+
Train step of epoch 0:  37%|███▋      | 2060/5616 [26:17<45:21,  1.31it/s, gpt_loss=0.0625, lr=3.79e-6] 
+
Train step of epoch 0:  37%|███▋      | 2061/5616 [26:17<44:12,  1.34it/s, gpt_loss=0.0625, lr=3.79e-6]
+
Train step of epoch 0:  37%|███▋      | 2061/5616 [26:17<44:12,  1.34it/s, gpt_loss=0.105, lr=3.79e-6] 
+
Train step of epoch 0:  37%|███▋      | 2062/5616 [26:17<44:36,  1.33it/s, gpt_loss=0.105, lr=3.79e-6]
+
Train step of epoch 0:  37%|███▋      | 2062/5616 [26:18<44:36,  1.33it/s, gpt_loss=0.00151, lr=3.79e-6]
+
Train step of epoch 0:  37%|███▋      | 2063/5616 [26:18<42:32,  1.39it/s, gpt_loss=0.00151, lr=3.79e-6]
+
Train step of epoch 0:  37%|███▋      | 2063/5616 [26:19<42:32,  1.39it/s, gpt_loss=0.0443, lr=3.79e-6] 
+
Train step of epoch 0:  37%|███▋      | 2064/5616 [26:19<46:03,  1.29it/s, gpt_loss=0.0443, lr=3.79e-6]
+
Train step of epoch 0:  37%|███▋      | 2064/5616 [26:20<46:03,  1.29it/s, gpt_loss=0.0566, lr=3.79e-6]
+
Train step of epoch 0:  37%|███▋      | 2065/5616 [26:20<51:51,  1.14it/s, gpt_loss=0.0566, lr=3.79e-6]
+
Train step of epoch 0:  37%|███▋      | 2065/5616 [26:21<51:51,  1.14it/s, gpt_loss=0.0548, lr=3.79e-6]
+
Train step of epoch 0:  37%|███▋      | 2066/5616 [26:21<49:17,  1.20it/s, gpt_loss=0.0548, lr=3.79e-6]
+
Train step of epoch 0:  37%|███▋      | 2066/5616 [26:22<49:17,  1.20it/s, gpt_loss=0.0409, lr=3.79e-6]
+
Train step of epoch 0:  37%|███▋      | 2067/5616 [26:22<47:42,  1.24it/s, gpt_loss=0.0409, lr=3.79e-6]
+
Train step of epoch 0:  37%|███▋      | 2067/5616 [26:22<47:42,  1.24it/s, gpt_loss=0.118, lr=3.78e-6] 
+
Train step of epoch 0:  37%|███▋      | 2068/5616 [26:22<47:24,  1.25it/s, gpt_loss=0.118, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2068/5616 [26:23<47:24,  1.25it/s, gpt_loss=0.0348, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2069/5616 [26:23<45:57,  1.29it/s, gpt_loss=0.0348, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2069/5616 [26:24<45:57,  1.29it/s, gpt_loss=0.0224, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2070/5616 [26:24<43:58,  1.34it/s, gpt_loss=0.0224, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2070/5616 [26:25<43:58,  1.34it/s, gpt_loss=0.0367, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2071/5616 [26:25<44:07,  1.34it/s, gpt_loss=0.0367, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2071/5616 [26:25<44:07,  1.34it/s, gpt_loss=0.0423, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2072/5616 [26:25<44:36,  1.32it/s, gpt_loss=0.0423, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2072/5616 [26:26<44:36,  1.32it/s, gpt_loss=0.000558, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2073/5616 [26:26<43:38,  1.35it/s, gpt_loss=0.000558, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2073/5616 [26:27<43:38,  1.35it/s, gpt_loss=0.105, lr=3.78e-6]   
+
Train step of epoch 0:  37%|███▋      | 2074/5616 [26:27<42:37,  1.38it/s, gpt_loss=0.105, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2074/5616 [26:27<42:37,  1.38it/s, gpt_loss=0.00233, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2075/5616 [26:27<41:09,  1.43it/s, gpt_loss=0.00233, lr=3.78e-6]
+
Train step of epoch 0:  37%|███▋      | 2075/5616 [26:28<41:09,  1.43it/s, gpt_loss=0.035, lr=3.77e-6]  
+
Train step of epoch 0:  37%|███▋      | 2076/5616 [26:28<43:06,  1.37it/s, gpt_loss=0.035, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2076/5616 [26:29<43:06,  1.37it/s, gpt_loss=0.00224, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2077/5616 [26:29<42:31,  1.39it/s, gpt_loss=0.00224, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2077/5616 [26:30<42:31,  1.39it/s, gpt_loss=0.0983, lr=3.77e-6] 
+
Train step of epoch 0:  37%|███▋      | 2078/5616 [26:30<41:53,  1.41it/s, gpt_loss=0.0983, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2078/5616 [26:30<41:53,  1.41it/s, gpt_loss=0.0296, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2079/5616 [26:30<43:03,  1.37it/s, gpt_loss=0.0296, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2079/5616 [26:31<43:03,  1.37it/s, gpt_loss=0.0122, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2080/5616 [26:31<43:14,  1.36it/s, gpt_loss=0.0122, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2080/5616 [26:32<43:14,  1.36it/s, gpt_loss=0.0206, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2081/5616 [26:32<44:58,  1.31it/s, gpt_loss=0.0206, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2081/5616 [26:33<44:58,  1.31it/s, gpt_loss=0.0939, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2082/5616 [26:33<43:51,  1.34it/s, gpt_loss=0.0939, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2082/5616 [26:34<43:51,  1.34it/s, gpt_loss=0.0727, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2083/5616 [26:34<48:41,  1.21it/s, gpt_loss=0.0727, lr=3.77e-6]
+
Train step of epoch 0:  37%|███▋      | 2083/5616 [26:34<48:41,  1.21it/s, gpt_loss=0.0484, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2084/5616 [26:34<48:55,  1.20it/s, gpt_loss=0.0484, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2084/5616 [26:35<48:55,  1.20it/s, gpt_loss=0.0347, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2085/5616 [26:35<47:53,  1.23it/s, gpt_loss=0.0347, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2085/5616 [26:36<47:53,  1.23it/s, gpt_loss=0.0191, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2086/5616 [26:36<45:51,  1.28it/s, gpt_loss=0.0191, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2086/5616 [26:37<45:51,  1.28it/s, gpt_loss=0.00668, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2087/5616 [26:37<44:49,  1.31it/s, gpt_loss=0.00668, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2087/5616 [26:37<44:49,  1.31it/s, gpt_loss=0.0229, lr=3.76e-6] 
+
Train step of epoch 0:  37%|███▋      | 2088/5616 [26:37<45:10,  1.30it/s, gpt_loss=0.0229, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2088/5616 [26:38<45:10,  1.30it/s, gpt_loss=0.0234, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2089/5616 [26:38<44:07,  1.33it/s, gpt_loss=0.0234, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2089/5616 [26:39<44:07,  1.33it/s, gpt_loss=0.0776, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2090/5616 [26:39<42:40,  1.38it/s, gpt_loss=0.0776, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2090/5616 [26:39<42:40,  1.38it/s, gpt_loss=0.0604, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2091/5616 [26:39<41:54,  1.40it/s, gpt_loss=0.0604, lr=3.76e-6]
+
Train step of epoch 0:  37%|███▋      | 2091/5616 [26:40<41:54,  1.40it/s, gpt_loss=0.0727, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2092/5616 [26:40<45:53,  1.28it/s, gpt_loss=0.0727, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2092/5616 [26:41<45:53,  1.28it/s, gpt_loss=0.0391, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2093/5616 [26:41<44:39,  1.31it/s, gpt_loss=0.0391, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2093/5616 [26:42<44:39,  1.31it/s, gpt_loss=0.00816, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2094/5616 [26:42<43:06,  1.36it/s, gpt_loss=0.00816, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2094/5616 [26:43<43:06,  1.36it/s, gpt_loss=0.0349, lr=3.75e-6] 
+
Train step of epoch 0:  37%|███▋      | 2095/5616 [26:43<45:16,  1.30it/s, gpt_loss=0.0349, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2095/5616 [26:43<45:16,  1.30it/s, gpt_loss=0.00116, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2096/5616 [26:43<43:53,  1.34it/s, gpt_loss=0.00116, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2096/5616 [26:44<43:53,  1.34it/s, gpt_loss=0.0189, lr=3.75e-6] 
+
Train step of epoch 0:  37%|███▋      | 2097/5616 [26:44<43:41,  1.34it/s, gpt_loss=0.0189, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2097/5616 [26:45<43:41,  1.34it/s, gpt_loss=0.13, lr=3.75e-6]  
+
Train step of epoch 0:  37%|███▋      | 2098/5616 [26:45<43:07,  1.36it/s, gpt_loss=0.13, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2098/5616 [26:45<43:07,  1.36it/s, gpt_loss=0.0523, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2099/5616 [26:45<42:08,  1.39it/s, gpt_loss=0.0523, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2099/5616 [26:46<42:08,  1.39it/s, gpt_loss=0.00368, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2100/5616 [26:46<43:55,  1.33it/s, gpt_loss=0.00368, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2100/5616 [26:47<43:55,  1.33it/s, gpt_loss=0.0585, lr=3.75e-6] 
+
Train step of epoch 0:  37%|███▋      | 2101/5616 [26:47<43:06,  1.36it/s, gpt_loss=0.0585, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2101/5616 [26:48<43:06,  1.36it/s, gpt_loss=0.0722, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2102/5616 [26:48<45:17,  1.29it/s, gpt_loss=0.0722, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2102/5616 [26:49<45:17,  1.29it/s, gpt_loss=0.0108, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2103/5616 [26:49<44:27,  1.32it/s, gpt_loss=0.0108, lr=3.75e-6]
+
Train step of epoch 0:  37%|███▋      | 2103/5616 [26:49<44:27,  1.32it/s, gpt_loss=0.0342, lr=3.74e-6]
+
Train step of epoch 0:  37%|███▋      | 2104/5616 [26:49<45:23,  1.29it/s, gpt_loss=0.0342, lr=3.74e-6]
+
Train step of epoch 0:  37%|███▋      | 2104/5616 [26:50<45:23,  1.29it/s, gpt_loss=0.0187, lr=3.74e-6]
+
Train step of epoch 0:  37%|███▋      | 2105/5616 [26:50<44:06,  1.33it/s, gpt_loss=0.0187, lr=3.74e-6]
+
Train step of epoch 0:  37%|███▋      | 2105/5616 [26:51<44:06,  1.33it/s, gpt_loss=0.0333, lr=3.74e-6]
+
Train step of epoch 0:  38%|███▊      | 2106/5616 [26:51<42:46,  1.37it/s, gpt_loss=0.0333, lr=3.74e-6]
+
Train step of epoch 0:  38%|███▊      | 2106/5616 [26:52<42:46,  1.37it/s, gpt_loss=0.0357, lr=3.74e-6]
+
Train step of epoch 0:  38%|███▊      | 2107/5616 [26:52<43:00,  1.36it/s, gpt_loss=0.0357, lr=3.74e-6]
+
Train step of epoch 0:  38%|███▊      | 2107/5616 [26:52<43:00,  1.36it/s, gpt_loss=0.0465, lr=3.74e-6]
+
Train step of epoch 0:  38%|███▊      | 2108/5616 [26:52<43:19,  1.35it/s, gpt_loss=0.0465, lr=3.74e-6]
+
Train step of epoch 0:  38%|███▊      | 2108/5616 [26:53<43:19,  1.35it/s, gpt_loss=0.0319, lr=3.74e-6]
+
Train step of epoch 0:  38%|███▊      | 2109/5616 [26:53<46:59,  1.24it/s, gpt_loss=0.0319, lr=3.74e-6]
+
Train step of epoch 0:  38%|███▊      | 2109/5616 [26:54<46:59,  1.24it/s, gpt_loss=0.0465, lr=3.74e-6]
+
Train step of epoch 0:  38%|███▊      | 2110/5616 [26:54<45:11,  1.29it/s, gpt_loss=0.0465, lr=3.74e-6]
+
Train step of epoch 0:  38%|███▊      | 2110/5616 [26:55<45:11,  1.29it/s, gpt_loss=0.0256, lr=3.74e-6]
+
Train step of epoch 0:  38%|███▊      | 2111/5616 [26:55<44:39,  1.31it/s, gpt_loss=0.0256, lr=3.74e-6]
+
Train step of epoch 0:  38%|███▊      | 2111/5616 [26:56<44:39,  1.31it/s, gpt_loss=0.0418, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2112/5616 [26:56<46:36,  1.25it/s, gpt_loss=0.0418, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2112/5616 [26:56<46:36,  1.25it/s, gpt_loss=0.0124, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2113/5616 [26:56<45:21,  1.29it/s, gpt_loss=0.0124, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2113/5616 [26:57<45:21,  1.29it/s, gpt_loss=0.0151, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2114/5616 [26:57<43:47,  1.33it/s, gpt_loss=0.0151, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2114/5616 [26:58<43:47,  1.33it/s, gpt_loss=0.0576, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2115/5616 [26:58<41:58,  1.39it/s, gpt_loss=0.0576, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2115/5616 [26:59<41:58,  1.39it/s, gpt_loss=0.164, lr=3.73e-6] 
+
Train step of epoch 0:  38%|███▊      | 2116/5616 [26:59<45:21,  1.29it/s, gpt_loss=0.164, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2116/5616 [26:59<45:21,  1.29it/s, gpt_loss=0.0947, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2117/5616 [26:59<43:45,  1.33it/s, gpt_loss=0.0947, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2117/5616 [27:00<43:45,  1.33it/s, gpt_loss=0.00401, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2118/5616 [27:00<43:47,  1.33it/s, gpt_loss=0.00401, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2118/5616 [27:01<43:47,  1.33it/s, gpt_loss=0.0525, lr=3.73e-6] 
+
Train step of epoch 0:  38%|███▊      | 2119/5616 [27:01<42:35,  1.37it/s, gpt_loss=0.0525, lr=3.73e-6]
+
Train step of epoch 0:  38%|███▊      | 2119/5616 [27:01<42:35,  1.37it/s, gpt_loss=0.0562, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2120/5616 [27:01<42:31,  1.37it/s, gpt_loss=0.0562, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2120/5616 [27:02<42:31,  1.37it/s, gpt_loss=0.0182, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2121/5616 [27:02<43:19,  1.34it/s, gpt_loss=0.0182, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2121/5616 [27:03<43:19,  1.34it/s, gpt_loss=0.0191, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2122/5616 [27:03<41:29,  1.40it/s, gpt_loss=0.0191, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2122/5616 [27:04<41:29,  1.40it/s, gpt_loss=0.00741, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2123/5616 [27:04<46:50,  1.24it/s, gpt_loss=0.00741, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2123/5616 [27:05<46:50,  1.24it/s, gpt_loss=0.0464, lr=3.72e-6] 
+
Train step of epoch 0:  38%|███▊      | 2124/5616 [27:05<46:49,  1.24it/s, gpt_loss=0.0464, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2124/5616 [27:05<46:49,  1.24it/s, gpt_loss=0.102, lr=3.72e-6] 
+
Train step of epoch 0:  38%|███▊      | 2125/5616 [27:05<45:27,  1.28it/s, gpt_loss=0.102, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2125/5616 [27:06<45:27,  1.28it/s, gpt_loss=0.0319, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2126/5616 [27:06<45:54,  1.27it/s, gpt_loss=0.0319, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2126/5616 [27:07<45:54,  1.27it/s, gpt_loss=0.0779, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2127/5616 [27:07<45:40,  1.27it/s, gpt_loss=0.0779, lr=3.72e-6]
+
Train step of epoch 0:  38%|███▊      | 2127/5616 [27:08<45:40,  1.27it/s, gpt_loss=0.0566, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2128/5616 [27:08<51:53,  1.12it/s, gpt_loss=0.0566, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2128/5616 [27:09<51:53,  1.12it/s, gpt_loss=0.0402, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2129/5616 [27:09<49:06,  1.18it/s, gpt_loss=0.0402, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2129/5616 [27:10<49:06,  1.18it/s, gpt_loss=0.091, lr=3.71e-6] 
+
Train step of epoch 0:  38%|███▊      | 2130/5616 [27:10<48:07,  1.21it/s, gpt_loss=0.091, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2130/5616 [27:11<48:07,  1.21it/s, gpt_loss=0.0974, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2131/5616 [27:11<52:50,  1.10it/s, gpt_loss=0.0974, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2131/5616 [27:12<52:50,  1.10it/s, gpt_loss=0.107, lr=3.71e-6] 
+
Train step of epoch 0:  38%|███▊      | 2132/5616 [27:12<55:43,  1.04it/s, gpt_loss=0.107, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2132/5616 [27:12<55:43,  1.04it/s, gpt_loss=0.032, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2133/5616 [27:12<50:55,  1.14it/s, gpt_loss=0.032, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2133/5616 [27:13<50:55,  1.14it/s, gpt_loss=0.0241, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2134/5616 [27:13<51:11,  1.13it/s, gpt_loss=0.0241, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2134/5616 [27:14<51:11,  1.13it/s, gpt_loss=0.0136, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2135/5616 [27:14<48:32,  1.20it/s, gpt_loss=0.0136, lr=3.71e-6]
+
Train step of epoch 0:  38%|███▊      | 2135/5616 [27:15<48:32,  1.20it/s, gpt_loss=0.00909, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2136/5616 [27:15<48:51,  1.19it/s, gpt_loss=0.00909, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2136/5616 [27:16<48:51,  1.19it/s, gpt_loss=0.029, lr=3.7e-6]  
+
Train step of epoch 0:  38%|███▊      | 2137/5616 [27:16<46:19,  1.25it/s, gpt_loss=0.029, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2137/5616 [27:16<46:19,  1.25it/s, gpt_loss=0.114, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2138/5616 [27:16<44:56,  1.29it/s, gpt_loss=0.114, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2138/5616 [27:17<44:56,  1.29it/s, gpt_loss=0.0118, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2139/5616 [27:17<43:35,  1.33it/s, gpt_loss=0.0118, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2139/5616 [27:18<43:35,  1.33it/s, gpt_loss=0.0665, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2140/5616 [27:18<44:04,  1.31it/s, gpt_loss=0.0665, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2140/5616 [27:19<44:04,  1.31it/s, gpt_loss=0.0395, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2141/5616 [27:19<42:46,  1.35it/s, gpt_loss=0.0395, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2141/5616 [27:19<42:46,  1.35it/s, gpt_loss=0.034, lr=3.7e-6] 
+
Train step of epoch 0:  38%|███▊      | 2142/5616 [27:19<41:10,  1.41it/s, gpt_loss=0.034, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2142/5616 [27:20<41:10,  1.41it/s, gpt_loss=0.0335, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2143/5616 [27:20<42:49,  1.35it/s, gpt_loss=0.0335, lr=3.7e-6]
+
Train step of epoch 0:  38%|███▊      | 2143/5616 [27:21<42:49,  1.35it/s, gpt_loss=0.00929, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2144/5616 [27:21<44:36,  1.30it/s, gpt_loss=0.00929, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2144/5616 [27:21<44:36,  1.30it/s, gpt_loss=0.0108, lr=3.69e-6] 
+
Train step of epoch 0:  38%|███▊      | 2145/5616 [27:21<42:44,  1.35it/s, gpt_loss=0.0108, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2145/5616 [27:22<42:44,  1.35it/s, gpt_loss=0.0762, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2146/5616 [27:22<41:30,  1.39it/s, gpt_loss=0.0762, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2146/5616 [27:23<41:30,  1.39it/s, gpt_loss=0.0216, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2147/5616 [27:23<40:34,  1.42it/s, gpt_loss=0.0216, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2147/5616 [27:24<40:34,  1.42it/s, gpt_loss=0.0423, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2148/5616 [27:24<44:10,  1.31it/s, gpt_loss=0.0423, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2148/5616 [27:24<44:10,  1.31it/s, gpt_loss=0.0538, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2149/5616 [27:24<43:30,  1.33it/s, gpt_loss=0.0538, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2149/5616 [27:25<43:30,  1.33it/s, gpt_loss=0.0109, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2150/5616 [27:25<43:57,  1.31it/s, gpt_loss=0.0109, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2150/5616 [27:26<43:57,  1.31it/s, gpt_loss=0.0343, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2151/5616 [27:26<42:42,  1.35it/s, gpt_loss=0.0343, lr=3.69e-6]
+
Train step of epoch 0:  38%|███▊      | 2151/5616 [27:27<42:42,  1.35it/s, gpt_loss=0.101, lr=3.68e-6] 
+
Train step of epoch 0:  38%|███▊      | 2152/5616 [27:27<42:57,  1.34it/s, gpt_loss=0.101, lr=3.68e-6]
+
Train step of epoch 0:  38%|███▊      | 2152/5616 [27:27<42:57,  1.34it/s, gpt_loss=0.00915, lr=3.68e-6]
+
Train step of epoch 0:  38%|███▊      | 2153/5616 [27:27<40:53,  1.41it/s, gpt_loss=0.00915, lr=3.68e-6]
+
Train step of epoch 0:  38%|███▊      | 2153/5616 [27:28<40:53,  1.41it/s, gpt_loss=0.0278, lr=3.68e-6] 
+
Train step of epoch 0:  38%|███▊      | 2154/5616 [27:28<40:39,  1.42it/s, gpt_loss=0.0278, lr=3.68e-6]
+
Train step of epoch 0:  38%|███▊      | 2154/5616 [27:29<40:39,  1.42it/s, gpt_loss=0.00689, lr=3.68e-6]
+
Train step of epoch 0:  38%|███▊      | 2155/5616 [27:29<40:06,  1.44it/s, gpt_loss=0.00689, lr=3.68e-6]
+
Train step of epoch 0:  38%|███▊      | 2155/5616 [27:29<40:06,  1.44it/s, gpt_loss=0.0332, lr=3.68e-6] 
+
Train step of epoch 0:  38%|███▊      | 2156/5616 [27:29<41:33,  1.39it/s, gpt_loss=0.0332, lr=3.68e-6]
+
Train step of epoch 0:  38%|███▊      | 2156/5616 [27:30<41:33,  1.39it/s, gpt_loss=0.0655, lr=3.68e-6]
+
Train step of epoch 0:  38%|███▊      | 2157/5616 [27:30<40:00,  1.44it/s, gpt_loss=0.0655, lr=3.68e-6]
+
Train step of epoch 0:  38%|███▊      | 2157/5616 [27:31<40:00,  1.44it/s, gpt_loss=0.0221, lr=3.68e-6]
+
Train step of epoch 0:  38%|███▊      | 2158/5616 [27:31<40:55,  1.41it/s, gpt_loss=0.0221, lr=3.68e-6]
+
Train step of epoch 0:  38%|███▊      | 2158/5616 [27:32<40:55,  1.41it/s, gpt_loss=0.00269, lr=3.68e-6]
+
Train step of epoch 0:  38%|███▊      | 2159/5616 [27:32<40:39,  1.42it/s, gpt_loss=0.00269, lr=3.68e-6]
+
Train step of epoch 0:  38%|███▊      | 2159/5616 [27:32<40:39,  1.42it/s, gpt_loss=0.0105, lr=3.67e-6] 
+
Train step of epoch 0:  38%|███▊      | 2160/5616 [27:32<43:05,  1.34it/s, gpt_loss=0.0105, lr=3.67e-6]
+
Train step of epoch 0:  38%|███▊      | 2160/5616 [27:33<43:05,  1.34it/s, gpt_loss=0.00865, lr=3.67e-6]
+
Train step of epoch 0:  38%|███▊      | 2161/5616 [27:33<43:02,  1.34it/s, gpt_loss=0.00865, lr=3.67e-6]
+
Train step of epoch 0:  38%|███▊      | 2161/5616 [27:34<43:02,  1.34it/s, gpt_loss=0.0904, lr=3.67e-6] 
+
Train step of epoch 0:  38%|███▊      | 2162/5616 [27:34<43:20,  1.33it/s, gpt_loss=0.0904, lr=3.67e-6]
+
Train step of epoch 0:  38%|███▊      | 2162/5616 [27:35<43:20,  1.33it/s, gpt_loss=0.0482, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2163/5616 [27:35<44:42,  1.29it/s, gpt_loss=0.0482, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2163/5616 [27:35<44:42,  1.29it/s, gpt_loss=0.0198, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2164/5616 [27:35<43:23,  1.33it/s, gpt_loss=0.0198, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2164/5616 [27:36<43:23,  1.33it/s, gpt_loss=0.0171, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2165/5616 [27:36<45:08,  1.27it/s, gpt_loss=0.0171, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2165/5616 [27:37<45:08,  1.27it/s, gpt_loss=0.0115, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2166/5616 [27:37<43:10,  1.33it/s, gpt_loss=0.0115, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2166/5616 [27:38<43:10,  1.33it/s, gpt_loss=0.128, lr=3.67e-6] 
+
Train step of epoch 0:  39%|███▊      | 2167/5616 [27:38<42:38,  1.35it/s, gpt_loss=0.128, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2167/5616 [27:39<42:38,  1.35it/s, gpt_loss=0.123, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2168/5616 [27:39<46:02,  1.25it/s, gpt_loss=0.123, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2168/5616 [27:39<46:02,  1.25it/s, gpt_loss=0.168, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2169/5616 [27:39<43:22,  1.32it/s, gpt_loss=0.168, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2169/5616 [27:40<43:22,  1.32it/s, gpt_loss=0.0547, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2170/5616 [27:40<41:31,  1.38it/s, gpt_loss=0.0547, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2170/5616 [27:41<41:31,  1.38it/s, gpt_loss=0.0225, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2171/5616 [27:41<41:57,  1.37it/s, gpt_loss=0.0225, lr=3.67e-6]
+
Train step of epoch 0:  39%|███▊      | 2171/5616 [27:41<41:57,  1.37it/s, gpt_loss=0.0947, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▊      | 2172/5616 [27:41<42:47,  1.34it/s, gpt_loss=0.0947, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▊      | 2172/5616 [27:42<42:47,  1.34it/s, gpt_loss=0.043, lr=3.66e-6] 
+
Train step of epoch 0:  39%|███▊      | 2173/5616 [27:42<41:10,  1.39it/s, gpt_loss=0.043, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▊      | 2173/5616 [27:43<41:10,  1.39it/s, gpt_loss=0.008, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▊      | 2174/5616 [27:43<40:16,  1.42it/s, gpt_loss=0.008, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▊      | 2174/5616 [27:43<40:16,  1.42it/s, gpt_loss=0.0674, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▊      | 2175/5616 [27:43<40:12,  1.43it/s, gpt_loss=0.0674, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▊      | 2175/5616 [27:44<40:12,  1.43it/s, gpt_loss=0.0114, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▊      | 2176/5616 [27:44<41:48,  1.37it/s, gpt_loss=0.0114, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▊      | 2176/5616 [27:45<41:48,  1.37it/s, gpt_loss=0.132, lr=3.66e-6] 
+
Train step of epoch 0:  39%|███▉      | 2177/5616 [27:45<42:34,  1.35it/s, gpt_loss=0.132, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▉      | 2177/5616 [27:46<42:34,  1.35it/s, gpt_loss=0.0588, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▉      | 2178/5616 [27:46<41:19,  1.39it/s, gpt_loss=0.0588, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▉      | 2178/5616 [27:46<41:19,  1.39it/s, gpt_loss=0.0863, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▉      | 2179/5616 [27:46<40:40,  1.41it/s, gpt_loss=0.0863, lr=3.66e-6]
+
Train step of epoch 0:  39%|███▉      | 2179/5616 [27:47<40:40,  1.41it/s, gpt_loss=0.0149, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2180/5616 [27:47<40:35,  1.41it/s, gpt_loss=0.0149, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2180/5616 [27:48<40:35,  1.41it/s, gpt_loss=0.0699, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2181/5616 [27:48<39:29,  1.45it/s, gpt_loss=0.0699, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2181/5616 [27:48<39:29,  1.45it/s, gpt_loss=0.0267, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2182/5616 [27:48<40:34,  1.41it/s, gpt_loss=0.0267, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2182/5616 [27:49<40:34,  1.41it/s, gpt_loss=0.0264, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2183/5616 [27:49<40:49,  1.40it/s, gpt_loss=0.0264, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2183/5616 [27:50<40:49,  1.40it/s, gpt_loss=0.0169, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2184/5616 [27:50<44:21,  1.29it/s, gpt_loss=0.0169, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2184/5616 [27:51<44:21,  1.29it/s, gpt_loss=0.0443, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2185/5616 [27:51<43:10,  1.32it/s, gpt_loss=0.0443, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2185/5616 [27:51<43:10,  1.32it/s, gpt_loss=0.0477, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2186/5616 [27:51<41:16,  1.39it/s, gpt_loss=0.0477, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2186/5616 [27:52<41:16,  1.39it/s, gpt_loss=0.0175, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2187/5616 [27:52<41:45,  1.37it/s, gpt_loss=0.0175, lr=3.65e-6]
+
Train step of epoch 0:  39%|███▉      | 2187/5616 [27:53<41:45,  1.37it/s, gpt_loss=0.15, lr=3.64e-6]  
+
Train step of epoch 0:  39%|███▉      | 2188/5616 [27:53<41:59,  1.36it/s, gpt_loss=0.15, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2188/5616 [27:54<41:59,  1.36it/s, gpt_loss=0.0145, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2189/5616 [27:54<44:12,  1.29it/s, gpt_loss=0.0145, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2189/5616 [27:55<44:12,  1.29it/s, gpt_loss=0.0412, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2190/5616 [27:55<44:34,  1.28it/s, gpt_loss=0.0412, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2190/5616 [27:55<44:34,  1.28it/s, gpt_loss=0.0734, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2191/5616 [27:55<44:08,  1.29it/s, gpt_loss=0.0734, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2191/5616 [27:56<44:08,  1.29it/s, gpt_loss=0.0272, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2192/5616 [27:56<44:17,  1.29it/s, gpt_loss=0.0272, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2192/5616 [27:57<44:17,  1.29it/s, gpt_loss=0.0106, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2193/5616 [27:57<42:27,  1.34it/s, gpt_loss=0.0106, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2193/5616 [27:58<42:27,  1.34it/s, gpt_loss=0.0416, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2194/5616 [27:58<41:25,  1.38it/s, gpt_loss=0.0416, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2194/5616 [27:58<41:25,  1.38it/s, gpt_loss=0.00733, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2195/5616 [27:58<40:06,  1.42it/s, gpt_loss=0.00733, lr=3.64e-6]
+
Train step of epoch 0:  39%|███▉      | 2195/5616 [27:59<40:06,  1.42it/s, gpt_loss=0.0149, lr=3.63e-6] 
+
Train step of epoch 0:  39%|███▉      | 2196/5616 [27:59<40:02,  1.42it/s, gpt_loss=0.0149, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2196/5616 [28:00<40:02,  1.42it/s, gpt_loss=0.0627, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2197/5616 [28:00<39:57,  1.43it/s, gpt_loss=0.0627, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2197/5616 [28:00<39:57,  1.43it/s, gpt_loss=0.00362, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2198/5616 [28:00<41:26,  1.37it/s, gpt_loss=0.00362, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2198/5616 [28:01<41:26,  1.37it/s, gpt_loss=0.0626, lr=3.63e-6] 
+
Train step of epoch 0:  39%|███▉      | 2199/5616 [28:01<39:33,  1.44it/s, gpt_loss=0.0626, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2199/5616 [28:02<39:33,  1.44it/s, gpt_loss=0.0103, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2200/5616 [28:02<40:36,  1.40it/s, gpt_loss=0.0103, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2200/5616 [28:02<40:36,  1.40it/s, gpt_loss=0.0275, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2201/5616 [28:02<40:20,  1.41it/s, gpt_loss=0.0275, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2201/5616 [28:03<40:20,  1.41it/s, gpt_loss=0.0346, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2202/5616 [28:03<41:04,  1.39it/s, gpt_loss=0.0346, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2202/5616 [28:04<41:04,  1.39it/s, gpt_loss=0.0214, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2203/5616 [28:04<40:42,  1.40it/s, gpt_loss=0.0214, lr=3.63e-6]
+
Train step of epoch 0:  39%|███▉      | 2203/5616 [28:05<40:42,  1.40it/s, gpt_loss=0.052, lr=3.62e-6] 
+
Train step of epoch 0:  39%|███▉      | 2204/5616 [28:05<41:43,  1.36it/s, gpt_loss=0.052, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2204/5616 [28:05<41:43,  1.36it/s, gpt_loss=0.0179, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2205/5616 [28:05<40:00,  1.42it/s, gpt_loss=0.0179, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2205/5616 [28:06<40:00,  1.42it/s, gpt_loss=0.011, lr=3.62e-6] 
+
Train step of epoch 0:  39%|███▉      | 2206/5616 [28:06<39:21,  1.44it/s, gpt_loss=0.011, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2206/5616 [28:07<39:21,  1.44it/s, gpt_loss=0.0226, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2207/5616 [28:07<39:04,  1.45it/s, gpt_loss=0.0226, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2207/5616 [28:07<39:04,  1.45it/s, gpt_loss=0.0154, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2208/5616 [28:07<39:39,  1.43it/s, gpt_loss=0.0154, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2208/5616 [28:08<39:39,  1.43it/s, gpt_loss=0.0269, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2209/5616 [28:08<39:51,  1.42it/s, gpt_loss=0.0269, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2209/5616 [28:09<39:51,  1.42it/s, gpt_loss=0.0403, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2210/5616 [28:09<41:45,  1.36it/s, gpt_loss=0.0403, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2210/5616 [28:10<41:45,  1.36it/s, gpt_loss=0.0125, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2211/5616 [28:10<41:07,  1.38it/s, gpt_loss=0.0125, lr=3.62e-6]
+
Train step of epoch 0:  39%|███▉      | 2211/5616 [28:10<41:07,  1.38it/s, gpt_loss=0.0737, lr=3.61e-6]
+
Train step of epoch 0:  39%|███▉      | 2212/5616 [28:10<41:28,  1.37it/s, gpt_loss=0.0737, lr=3.61e-6]
+
Train step of epoch 0:  39%|███▉      | 2212/5616 [28:11<41:28,  1.37it/s, gpt_loss=0.0387, lr=3.61e-6]
+
Train step of epoch 0:  39%|███▉      | 2213/5616 [28:11<42:08,  1.35it/s, gpt_loss=0.0387, lr=3.61e-6]
+
Train step of epoch 0:  39%|███▉      | 2213/5616 [28:12<42:08,  1.35it/s, gpt_loss=0.0619, lr=3.61e-6]
+
Train step of epoch 0:  39%|███▉      | 2214/5616 [28:12<40:32,  1.40it/s, gpt_loss=0.0619, lr=3.61e-6]
+
Train step of epoch 0:  39%|███▉      | 2214/5616 [28:12<40:32,  1.40it/s, gpt_loss=0.163, lr=3.61e-6] 
+
Train step of epoch 0:  39%|███▉      | 2215/5616 [28:12<40:40,  1.39it/s, gpt_loss=0.163, lr=3.61e-6]
+
Train step of epoch 0:  39%|███▉      | 2215/5616 [28:13<40:40,  1.39it/s, gpt_loss=0.0368, lr=3.61e-6]
+
Train step of epoch 0:  39%|███▉      | 2216/5616 [28:13<41:12,  1.38it/s, gpt_loss=0.0368, lr=3.61e-6]
+
Train step of epoch 0:  39%|███▉      | 2216/5616 [28:14<41:12,  1.38it/s, gpt_loss=0.0348, lr=3.61e-6]
+
Train step of epoch 0:  39%|███▉      | 2217/5616 [28:14<40:23,  1.40it/s, gpt_loss=0.0348, lr=3.61e-6]
+
Train step of epoch 0:  39%|███▉      | 2217/5616 [28:15<40:23,  1.40it/s, gpt_loss=0.0336, lr=3.61e-6]
+
Train step of epoch 0:  39%|███▉      | 2218/5616 [28:15<40:44,  1.39it/s, gpt_loss=0.0336, lr=3.61e-6]
+
Train step of epoch 0:  39%|███▉      | 2218/5616 [28:15<40:44,  1.39it/s, gpt_loss=0.0128, lr=3.61e-6]
+
Train step of epoch 0:  40%|███▉      | 2219/5616 [28:15<41:15,  1.37it/s, gpt_loss=0.0128, lr=3.61e-6]
+
Train step of epoch 0:  40%|███▉      | 2219/5616 [28:16<41:15,  1.37it/s, gpt_loss=0.0326, lr=3.6e-6] 
+
Train step of epoch 0:  40%|███▉      | 2220/5616 [28:16<43:14,  1.31it/s, gpt_loss=0.0326, lr=3.6e-6]
+
Train step of epoch 0:  40%|███▉      | 2220/5616 [28:17<43:14,  1.31it/s, gpt_loss=0.00401, lr=3.6e-6]
+
Train step of epoch 0:  40%|███▉      | 2221/5616 [28:17<41:42,  1.36it/s, gpt_loss=0.00401, lr=3.6e-6]
+
Train step of epoch 0:  40%|███▉      | 2221/5616 [28:18<41:42,  1.36it/s, gpt_loss=0.0404, lr=3.6e-6] 
+
Train step of epoch 0:  40%|███▉      | 2222/5616 [28:18<40:42,  1.39it/s, gpt_loss=0.0404, lr=3.6e-6]
+
Train step of epoch 0:  40%|███▉      | 2222/5616 [28:18<40:42,  1.39it/s, gpt_loss=0.00822, lr=3.6e-6]
+
Train step of epoch 0:  40%|███▉      | 2223/5616 [28:18<41:15,  1.37it/s, gpt_loss=0.00822, lr=3.6e-6]
+
Train step of epoch 0:  40%|███▉      | 2223/5616 [28:19<41:15,  1.37it/s, gpt_loss=0.145, lr=3.6e-6]  
+
Train step of epoch 0:  40%|███▉      | 2224/5616 [28:19<45:40,  1.24it/s, gpt_loss=0.145, lr=3.6e-6]
+
Train step of epoch 0:  40%|███▉      | 2224/5616 [28:20<45:40,  1.24it/s, gpt_loss=0.0287, lr=3.6e-6]
+
Train step of epoch 0:  40%|███▉      | 2225/5616 [28:20<43:49,  1.29it/s, gpt_loss=0.0287, lr=3.6e-6]
+
Train step of epoch 0:  40%|███▉      | 2225/5616 [28:21<43:49,  1.29it/s, gpt_loss=0.0409, lr=3.6e-6]
+
Train step of epoch 0:  40%|███▉      | 2226/5616 [28:21<42:52,  1.32it/s, gpt_loss=0.0409, lr=3.6e-6]
+
Train step of epoch 0:  40%|███▉      | 2226/5616 [28:21<42:52,  1.32it/s, gpt_loss=0.0292, lr=3.6e-6]
+
Train step of epoch 0:  40%|███▉      | 2227/5616 [28:21<41:54,  1.35it/s, gpt_loss=0.0292, lr=3.6e-6]
+
Train step of epoch 0:  40%|███▉      | 2227/5616 [28:22<41:54,  1.35it/s, gpt_loss=0.034, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2228/5616 [28:22<42:34,  1.33it/s, gpt_loss=0.034, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2228/5616 [28:23<42:34,  1.33it/s, gpt_loss=0.0876, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2229/5616 [28:23<42:18,  1.33it/s, gpt_loss=0.0876, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2229/5616 [28:24<42:18,  1.33it/s, gpt_loss=0.0107, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2230/5616 [28:24<42:33,  1.33it/s, gpt_loss=0.0107, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2230/5616 [28:24<42:33,  1.33it/s, gpt_loss=0.0727, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2231/5616 [28:24<40:21,  1.40it/s, gpt_loss=0.0727, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2231/5616 [28:25<40:21,  1.40it/s, gpt_loss=0.0422, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2232/5616 [28:25<41:31,  1.36it/s, gpt_loss=0.0422, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2232/5616 [28:26<41:31,  1.36it/s, gpt_loss=0.0216, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2233/5616 [28:26<42:02,  1.34it/s, gpt_loss=0.0216, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2233/5616 [28:27<42:02,  1.34it/s, gpt_loss=0.0183, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2234/5616 [28:27<42:08,  1.34it/s, gpt_loss=0.0183, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2234/5616 [28:27<42:08,  1.34it/s, gpt_loss=0.113, lr=3.59e-6] 
+
Train step of epoch 0:  40%|███▉      | 2235/5616 [28:27<43:17,  1.30it/s, gpt_loss=0.113, lr=3.59e-6]
+
Train step of epoch 0:  40%|███▉      | 2235/5616 [28:29<43:17,  1.30it/s, gpt_loss=0.0919, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2236/5616 [28:29<49:28,  1.14it/s, gpt_loss=0.0919, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2236/5616 [28:29<49:28,  1.14it/s, gpt_loss=0.00734, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2237/5616 [28:29<45:56,  1.23it/s, gpt_loss=0.00734, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2237/5616 [28:30<45:56,  1.23it/s, gpt_loss=0.0116, lr=3.58e-6] 
+
Train step of epoch 0:  40%|███▉      | 2238/5616 [28:30<44:24,  1.27it/s, gpt_loss=0.0116, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2238/5616 [28:31<44:24,  1.27it/s, gpt_loss=0.0134, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2239/5616 [28:31<43:02,  1.31it/s, gpt_loss=0.0134, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2239/5616 [28:31<43:02,  1.31it/s, gpt_loss=0.00649, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2240/5616 [28:31<42:35,  1.32it/s, gpt_loss=0.00649, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2240/5616 [28:32<42:35,  1.32it/s, gpt_loss=0.00256, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2241/5616 [28:32<46:19,  1.21it/s, gpt_loss=0.00256, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2241/5616 [28:33<46:19,  1.21it/s, gpt_loss=0.0206, lr=3.58e-6] 
+
Train step of epoch 0:  40%|███▉      | 2242/5616 [28:33<43:38,  1.29it/s, gpt_loss=0.0206, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2242/5616 [28:34<43:38,  1.29it/s, gpt_loss=0.0186, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2243/5616 [28:34<41:46,  1.35it/s, gpt_loss=0.0186, lr=3.58e-6]
+
Train step of epoch 0:  40%|███▉      | 2243/5616 [28:35<41:46,  1.35it/s, gpt_loss=0.0965, lr=3.57e-6]
+
Train step of epoch 0:  40%|███▉      | 2244/5616 [28:35<43:57,  1.28it/s, gpt_loss=0.0965, lr=3.57e-6]
+
Train step of epoch 0:  40%|███▉      | 2244/5616 [28:35<43:57,  1.28it/s, gpt_loss=0.117, lr=3.57e-6] 
+
Train step of epoch 0:  40%|███▉      | 2245/5616 [28:35<42:27,  1.32it/s, gpt_loss=0.117, lr=3.57e-6]
+
Train step of epoch 0:  40%|███▉      | 2245/5616 [28:36<42:27,  1.32it/s, gpt_loss=0.0317, lr=3.57e-6]
+
Train step of epoch 0:  40%|███▉      | 2246/5616 [28:36<41:18,  1.36it/s, gpt_loss=0.0317, lr=3.57e-6]
+
Train step of epoch 0:  40%|███▉      | 2246/5616 [28:37<41:18,  1.36it/s, gpt_loss=0.055, lr=3.57e-6] 
+
Train step of epoch 0:  40%|████      | 2247/5616 [28:37<48:45,  1.15it/s, gpt_loss=0.055, lr=3.57e-6]
+
Train step of epoch 0:  40%|████      | 2247/5616 [28:38<48:45,  1.15it/s, gpt_loss=0.0284, lr=3.57e-6]
+
Train step of epoch 0:  40%|████      | 2248/5616 [28:38<49:43,  1.13it/s, gpt_loss=0.0284, lr=3.57e-6]
+
Train step of epoch 0:  40%|████      | 2248/5616 [28:39<49:43,  1.13it/s, gpt_loss=0.0228, lr=3.57e-6]
+
Train step of epoch 0:  40%|████      | 2249/5616 [28:39<46:40,  1.20it/s, gpt_loss=0.0228, lr=3.57e-6]
+
Train step of epoch 0:  40%|████      | 2249/5616 [28:40<46:40,  1.20it/s, gpt_loss=0.129, lr=3.57e-6] 
+
Train step of epoch 0:  40%|████      | 2250/5616 [28:40<44:26,  1.26it/s, gpt_loss=0.129, lr=3.57e-6]
+
Train step of epoch 0:  40%|████      | 2250/5616 [28:40<44:26,  1.26it/s, gpt_loss=0.0431, lr=3.57e-6]
+
Train step of epoch 0:  40%|████      | 2251/5616 [28:40<42:40,  1.31it/s, gpt_loss=0.0431, lr=3.57e-6]
+
Train step of epoch 0:  40%|████      | 2251/5616 [28:41<42:40,  1.31it/s, gpt_loss=0.0417, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2252/5616 [28:41<42:47,  1.31it/s, gpt_loss=0.0417, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2252/5616 [28:42<42:47,  1.31it/s, gpt_loss=0.0147, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2253/5616 [28:42<41:49,  1.34it/s, gpt_loss=0.0147, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2253/5616 [28:43<41:49,  1.34it/s, gpt_loss=0.0431, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2254/5616 [28:43<50:06,  1.12it/s, gpt_loss=0.0431, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2254/5616 [28:44<50:06,  1.12it/s, gpt_loss=0.0437, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2255/5616 [28:44<46:39,  1.20it/s, gpt_loss=0.0437, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2255/5616 [28:44<46:39,  1.20it/s, gpt_loss=0.0404, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2256/5616 [28:44<45:40,  1.23it/s, gpt_loss=0.0404, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2256/5616 [28:45<45:40,  1.23it/s, gpt_loss=0.0285, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2257/5616 [28:45<45:18,  1.24it/s, gpt_loss=0.0285, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2257/5616 [28:46<45:18,  1.24it/s, gpt_loss=0.0621, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2258/5616 [28:46<47:43,  1.17it/s, gpt_loss=0.0621, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2258/5616 [28:47<47:43,  1.17it/s, gpt_loss=0.0477, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2259/5616 [28:47<47:16,  1.18it/s, gpt_loss=0.0477, lr=3.56e-6]
+
Train step of epoch 0:  40%|████      | 2259/5616 [28:48<47:16,  1.18it/s, gpt_loss=0.0615, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2260/5616 [28:48<46:42,  1.20it/s, gpt_loss=0.0615, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2260/5616 [28:49<46:42,  1.20it/s, gpt_loss=0.0775, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2261/5616 [28:49<44:29,  1.26it/s, gpt_loss=0.0775, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2261/5616 [28:49<44:29,  1.26it/s, gpt_loss=0.0114, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2262/5616 [28:49<42:48,  1.31it/s, gpt_loss=0.0114, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2262/5616 [28:50<42:48,  1.31it/s, gpt_loss=0.0559, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2263/5616 [28:50<43:30,  1.28it/s, gpt_loss=0.0559, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2263/5616 [28:51<43:30,  1.28it/s, gpt_loss=0.00969, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2264/5616 [28:51<43:36,  1.28it/s, gpt_loss=0.00969, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2264/5616 [28:52<43:36,  1.28it/s, gpt_loss=0.0484, lr=3.55e-6] 
+
Train step of epoch 0:  40%|████      | 2265/5616 [28:52<43:12,  1.29it/s, gpt_loss=0.0484, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2265/5616 [28:52<43:12,  1.29it/s, gpt_loss=0.0988, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2266/5616 [28:52<42:48,  1.30it/s, gpt_loss=0.0988, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2266/5616 [28:53<42:48,  1.30it/s, gpt_loss=0.0394, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2267/5616 [28:53<44:05,  1.27it/s, gpt_loss=0.0394, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2267/5616 [28:54<44:05,  1.27it/s, gpt_loss=0.0335, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2268/5616 [28:54<44:01,  1.27it/s, gpt_loss=0.0335, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2268/5616 [28:55<44:01,  1.27it/s, gpt_loss=0.0437, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2269/5616 [28:55<41:44,  1.34it/s, gpt_loss=0.0437, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2269/5616 [28:55<41:44,  1.34it/s, gpt_loss=0.0434, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2270/5616 [28:55<39:58,  1.40it/s, gpt_loss=0.0434, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2270/5616 [28:56<39:58,  1.40it/s, gpt_loss=0.0938, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2271/5616 [28:56<40:01,  1.39it/s, gpt_loss=0.0938, lr=3.55e-6]
+
Train step of epoch 0:  40%|████      | 2271/5616 [28:57<40:01,  1.39it/s, gpt_loss=0.047, lr=3.54e-6] 
+
Train step of epoch 0:  40%|████      | 2272/5616 [28:57<41:41,  1.34it/s, gpt_loss=0.047, lr=3.54e-6]
+
Train step of epoch 0:  40%|████      | 2272/5616 [28:58<41:41,  1.34it/s, gpt_loss=0.00845, lr=3.54e-6]
+
Train step of epoch 0:  40%|████      | 2273/5616 [28:58<42:59,  1.30it/s, gpt_loss=0.00845, lr=3.54e-6]
+
Train step of epoch 0:  40%|████      | 2273/5616 [28:58<42:59,  1.30it/s, gpt_loss=0.0109, lr=3.54e-6] 
+
Train step of epoch 0:  40%|████      | 2274/5616 [28:58<42:27,  1.31it/s, gpt_loss=0.0109, lr=3.54e-6]
+
Train step of epoch 0:  40%|████      | 2274/5616 [28:59<42:27,  1.31it/s, gpt_loss=0.016, lr=3.54e-6] 
+
Train step of epoch 0:  41%|████      | 2275/5616 [28:59<40:42,  1.37it/s, gpt_loss=0.016, lr=3.54e-6]
+
Train step of epoch 0:  41%|████      | 2275/5616 [29:00<40:42,  1.37it/s, gpt_loss=0.0714, lr=3.54e-6]
+
Train step of epoch 0:  41%|████      | 2276/5616 [29:00<40:43,  1.37it/s, gpt_loss=0.0714, lr=3.54e-6]
+
Train step of epoch 0:  41%|████      | 2276/5616 [29:01<40:43,  1.37it/s, gpt_loss=0.0106, lr=3.54e-6]
+
Train step of epoch 0:  41%|████      | 2277/5616 [29:01<41:50,  1.33it/s, gpt_loss=0.0106, lr=3.54e-6]
+
Train step of epoch 0:  41%|████      | 2277/5616 [29:01<41:50,  1.33it/s, gpt_loss=0.00999, lr=3.54e-6]
+
Train step of epoch 0:  41%|████      | 2278/5616 [29:01<40:27,  1.38it/s, gpt_loss=0.00999, lr=3.54e-6]
+
Train step of epoch 0:  41%|████      | 2278/5616 [29:02<40:27,  1.38it/s, gpt_loss=0.033, lr=3.54e-6]  
+
Train step of epoch 0:  41%|████      | 2279/5616 [29:02<40:21,  1.38it/s, gpt_loss=0.033, lr=3.54e-6]
+
Train step of epoch 0:  41%|████      | 2279/5616 [29:03<40:21,  1.38it/s, gpt_loss=0.0537, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2280/5616 [29:03<43:47,  1.27it/s, gpt_loss=0.0537, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2280/5616 [29:04<43:47,  1.27it/s, gpt_loss=0.0739, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2281/5616 [29:04<44:46,  1.24it/s, gpt_loss=0.0739, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2281/5616 [29:05<44:46,  1.24it/s, gpt_loss=0.0212, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2282/5616 [29:05<45:00,  1.23it/s, gpt_loss=0.0212, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2282/5616 [29:05<45:00,  1.23it/s, gpt_loss=0.0347, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2283/5616 [29:05<44:41,  1.24it/s, gpt_loss=0.0347, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2283/5616 [29:06<44:41,  1.24it/s, gpt_loss=0.0479, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2284/5616 [29:06<43:23,  1.28it/s, gpt_loss=0.0479, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2284/5616 [29:07<43:23,  1.28it/s, gpt_loss=0.02, lr=3.53e-6]  
+
Train step of epoch 0:  41%|████      | 2285/5616 [29:07<41:06,  1.35it/s, gpt_loss=0.02, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2285/5616 [29:07<41:06,  1.35it/s, gpt_loss=0.0177, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2286/5616 [29:07<41:31,  1.34it/s, gpt_loss=0.0177, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2286/5616 [29:08<41:31,  1.34it/s, gpt_loss=0.0689, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2287/5616 [29:08<40:20,  1.38it/s, gpt_loss=0.0689, lr=3.53e-6]
+
Train step of epoch 0:  41%|████      | 2287/5616 [29:09<40:20,  1.38it/s, gpt_loss=0.0735, lr=3.52e-6]
+
Train step of epoch 0:  41%|████      | 2288/5616 [29:09<41:36,  1.33it/s, gpt_loss=0.0735, lr=3.52e-6]
+
Train step of epoch 0:  41%|████      | 2288/5616 [29:10<41:36,  1.33it/s, gpt_loss=0.112, lr=3.52e-6] 
+
Train step of epoch 0:  41%|████      | 2289/5616 [29:10<41:33,  1.33it/s, gpt_loss=0.112, lr=3.52e-6]
+
Train step of epoch 0:  41%|████      | 2289/5616 [29:11<41:33,  1.33it/s, gpt_loss=0.0024, lr=3.52e-6]
+
Train step of epoch 0:  41%|████      | 2290/5616 [29:11<43:06,  1.29it/s, gpt_loss=0.0024, lr=3.52e-6]
+
Train step of epoch 0:  41%|████      | 2290/5616 [29:11<43:06,  1.29it/s, gpt_loss=0.127, lr=3.52e-6] 
+
Train step of epoch 0:  41%|████      | 2291/5616 [29:11<41:30,  1.34it/s, gpt_loss=0.127, lr=3.52e-6]
+
Train step of epoch 0:  41%|████      | 2291/5616 [29:12<41:30,  1.34it/s, gpt_loss=0.108, lr=3.52e-6]
+
Train step of epoch 0:  41%|████      | 2292/5616 [29:12<41:00,  1.35it/s, gpt_loss=0.108, lr=3.52e-6]
+
Train step of epoch 0:  41%|████      | 2292/5616 [29:13<41:00,  1.35it/s, gpt_loss=0.00399, lr=3.52e-6]
+
Train step of epoch 0:  41%|████      | 2293/5616 [29:13<40:37,  1.36it/s, gpt_loss=0.00399, lr=3.52e-6]
+
Train step of epoch 0:  41%|████      | 2293/5616 [29:13<40:37,  1.36it/s, gpt_loss=0.0393, lr=3.52e-6] 
+
Train step of epoch 0:  41%|████      | 2294/5616 [29:13<39:28,  1.40it/s, gpt_loss=0.0393, lr=3.52e-6]
+
Train step of epoch 0:  41%|████      | 2294/5616 [29:14<39:28,  1.40it/s, gpt_loss=0.0387, lr=3.52e-6]
+
Train step of epoch 0:  41%|████      | 2295/5616 [29:14<39:13,  1.41it/s, gpt_loss=0.0387, lr=3.52e-6]
+
Train step of epoch 0:  41%|████      | 2295/5616 [29:15<39:13,  1.41it/s, gpt_loss=0.0396, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2296/5616 [29:15<44:35,  1.24it/s, gpt_loss=0.0396, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2296/5616 [29:16<44:35,  1.24it/s, gpt_loss=0.0286, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2297/5616 [29:16<43:58,  1.26it/s, gpt_loss=0.0286, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2297/5616 [29:17<43:58,  1.26it/s, gpt_loss=0.0016, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2298/5616 [29:17<43:12,  1.28it/s, gpt_loss=0.0016, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2298/5616 [29:17<43:12,  1.28it/s, gpt_loss=0.0132, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2299/5616 [29:17<41:39,  1.33it/s, gpt_loss=0.0132, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2299/5616 [29:18<41:39,  1.33it/s, gpt_loss=0.0357, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2300/5616 [29:18<41:06,  1.34it/s, gpt_loss=0.0357, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2300/5616 [29:19<41:06,  1.34it/s, gpt_loss=0.0699, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2301/5616 [29:19<41:53,  1.32it/s, gpt_loss=0.0699, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2301/5616 [29:20<41:53,  1.32it/s, gpt_loss=0.099, lr=3.51e-6] 
+
Train step of epoch 0:  41%|████      | 2302/5616 [29:20<41:46,  1.32it/s, gpt_loss=0.099, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2302/5616 [29:20<41:46,  1.32it/s, gpt_loss=0.0376, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2303/5616 [29:20<40:50,  1.35it/s, gpt_loss=0.0376, lr=3.51e-6]
+
Train step of epoch 0:  41%|████      | 2303/5616 [29:21<40:50,  1.35it/s, gpt_loss=0.0585, lr=3.5e-6] 
+
Train step of epoch 0:  41%|████      | 2304/5616 [29:21<41:57,  1.32it/s, gpt_loss=0.0585, lr=3.5e-6]
+
Train step of epoch 0:  41%|████      | 2304/5616 [29:22<41:57,  1.32it/s, gpt_loss=0.146, lr=3.5e-6] 
+
Train step of epoch 0:  41%|████      | 2305/5616 [29:22<40:53,  1.35it/s, gpt_loss=0.146, lr=3.5e-6]
+
Train step of epoch 0:  41%|████      | 2305/5616 [29:22<40:53,  1.35it/s, gpt_loss=0.109, lr=3.5e-6]
+
Train step of epoch 0:  41%|████      | 2306/5616 [29:22<39:46,  1.39it/s, gpt_loss=0.109, lr=3.5e-6]
+
Train step of epoch 0:  41%|████      | 2306/5616 [29:23<39:46,  1.39it/s, gpt_loss=0.0738, lr=3.5e-6]
+
Train step of epoch 0:  41%|████      | 2307/5616 [29:23<38:12,  1.44it/s, gpt_loss=0.0738, lr=3.5e-6]
+
Train step of epoch 0:  41%|████      | 2307/5616 [29:24<38:12,  1.44it/s, gpt_loss=0.0833, lr=3.5e-6]
+
Train step of epoch 0:  41%|████      | 2308/5616 [29:24<39:08,  1.41it/s, gpt_loss=0.0833, lr=3.5e-6]
+
Train step of epoch 0:  41%|████      | 2308/5616 [29:24<39:08,  1.41it/s, gpt_loss=0.0148, lr=3.5e-6]
+
Train step of epoch 0:  41%|████      | 2309/5616 [29:24<38:43,  1.42it/s, gpt_loss=0.0148, lr=3.5e-6]
+
Train step of epoch 0:  41%|████      | 2309/5616 [29:25<38:43,  1.42it/s, gpt_loss=0.0123, lr=3.5e-6]
+
Train step of epoch 0:  41%|████      | 2310/5616 [29:25<38:38,  1.43it/s, gpt_loss=0.0123, lr=3.5e-6]
+
Train step of epoch 0:  41%|████      | 2310/5616 [29:26<38:38,  1.43it/s, gpt_loss=0.078, lr=3.5e-6] 
+
Train step of epoch 0:  41%|████      | 2311/5616 [29:26<40:22,  1.36it/s, gpt_loss=0.078, lr=3.5e-6]
+
Train step of epoch 0:  41%|████      | 2311/5616 [29:27<40:22,  1.36it/s, gpt_loss=0.00567, lr=3.49e-6]
+
Train step of epoch 0:  41%|████      | 2312/5616 [29:27<40:05,  1.37it/s, gpt_loss=0.00567, lr=3.49e-6]
+
Train step of epoch 0:  41%|████      | 2312/5616 [29:27<40:05,  1.37it/s, gpt_loss=0.0976, lr=3.49e-6] 
+
Train step of epoch 0:  41%|████      | 2313/5616 [29:27<41:14,  1.34it/s, gpt_loss=0.0976, lr=3.49e-6]
+
Train step of epoch 0:  41%|████      | 2313/5616 [29:28<41:14,  1.34it/s, gpt_loss=0.0694, lr=3.49e-6]
+
Train step of epoch 0:  41%|████      | 2314/5616 [29:28<40:26,  1.36it/s, gpt_loss=0.0694, lr=3.49e-6]
+
Train step of epoch 0:  41%|████      | 2314/5616 [29:29<40:26,  1.36it/s, gpt_loss=0.0681, lr=3.49e-6]
+
Train step of epoch 0:  41%|████      | 2315/5616 [29:29<43:52,  1.25it/s, gpt_loss=0.0681, lr=3.49e-6]
+
Train step of epoch 0:  41%|████      | 2315/5616 [29:30<43:52,  1.25it/s, gpt_loss=0.0458, lr=3.49e-6]
+
Train step of epoch 0:  41%|████      | 2316/5616 [29:30<46:16,  1.19it/s, gpt_loss=0.0458, lr=3.49e-6]
+
Train step of epoch 0:  41%|████      | 2316/5616 [29:31<46:16,  1.19it/s, gpt_loss=0.0387, lr=3.49e-6]
+
Train step of epoch 0:  41%|████▏     | 2317/5616 [29:31<44:57,  1.22it/s, gpt_loss=0.0387, lr=3.49e-6]
+
Train step of epoch 0:  41%|████▏     | 2317/5616 [29:32<44:57,  1.22it/s, gpt_loss=0.114, lr=3.49e-6] 
+
Train step of epoch 0:  41%|████▏     | 2318/5616 [29:32<43:44,  1.26it/s, gpt_loss=0.114, lr=3.49e-6]
+
Train step of epoch 0:  41%|████▏     | 2318/5616 [29:32<43:44,  1.26it/s, gpt_loss=0.0477, lr=3.49e-6]
+
Train step of epoch 0:  41%|████▏     | 2319/5616 [29:32<41:49,  1.31it/s, gpt_loss=0.0477, lr=3.49e-6]
+
Train step of epoch 0:  41%|████▏     | 2319/5616 [29:33<41:49,  1.31it/s, gpt_loss=0.092, lr=3.48e-6] 
+
Train step of epoch 0:  41%|████▏     | 2320/5616 [29:33<43:05,  1.27it/s, gpt_loss=0.092, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2320/5616 [29:34<43:05,  1.27it/s, gpt_loss=0.122, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2321/5616 [29:34<40:56,  1.34it/s, gpt_loss=0.122, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2321/5616 [29:34<40:56,  1.34it/s, gpt_loss=0.0202, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2322/5616 [29:34<40:11,  1.37it/s, gpt_loss=0.0202, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2322/5616 [29:35<40:11,  1.37it/s, gpt_loss=0.0113, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2323/5616 [29:35<39:36,  1.39it/s, gpt_loss=0.0113, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2323/5616 [29:36<39:36,  1.39it/s, gpt_loss=0.0888, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2324/5616 [29:36<44:33,  1.23it/s, gpt_loss=0.0888, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2324/5616 [29:37<44:33,  1.23it/s, gpt_loss=0.0767, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2325/5616 [29:37<42:41,  1.28it/s, gpt_loss=0.0767, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2325/5616 [29:38<42:41,  1.28it/s, gpt_loss=0.0448, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2326/5616 [29:38<41:02,  1.34it/s, gpt_loss=0.0448, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2326/5616 [29:38<41:02,  1.34it/s, gpt_loss=0.0346, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2327/5616 [29:38<39:52,  1.37it/s, gpt_loss=0.0346, lr=3.48e-6]
+
Train step of epoch 0:  41%|████▏     | 2327/5616 [29:39<39:52,  1.37it/s, gpt_loss=0.0545, lr=3.47e-6]
+
Train step of epoch 0:  41%|████▏     | 2328/5616 [29:39<41:10,  1.33it/s, gpt_loss=0.0545, lr=3.47e-6]
+
Train step of epoch 0:  41%|████▏     | 2328/5616 [29:40<41:10,  1.33it/s, gpt_loss=0.0165, lr=3.47e-6]
+
Train step of epoch 0:  41%|████▏     | 2329/5616 [29:40<40:45,  1.34it/s, gpt_loss=0.0165, lr=3.47e-6]
+
Train step of epoch 0:  41%|████▏     | 2329/5616 [29:41<40:45,  1.34it/s, gpt_loss=0.00895, lr=3.47e-6]
+
Train step of epoch 0:  41%|████▏     | 2330/5616 [29:41<43:21,  1.26it/s, gpt_loss=0.00895, lr=3.47e-6]
+
Train step of epoch 0:  41%|████▏     | 2330/5616 [29:41<43:21,  1.26it/s, gpt_loss=0.0194, lr=3.47e-6] 
+
Train step of epoch 0:  42%|████▏     | 2331/5616 [29:41<41:06,  1.33it/s, gpt_loss=0.0194, lr=3.47e-6]
+
Train step of epoch 0:  42%|████▏     | 2331/5616 [29:42<41:06,  1.33it/s, gpt_loss=0.0661, lr=3.47e-6]
+
Train step of epoch 0:  42%|████▏     | 2332/5616 [29:42<41:26,  1.32it/s, gpt_loss=0.0661, lr=3.47e-6]
+
Train step of epoch 0:  42%|████▏     | 2332/5616 [29:43<41:26,  1.32it/s, gpt_loss=0.127, lr=3.47e-6] 
+
Train step of epoch 0:  42%|████▏     | 2333/5616 [29:43<42:26,  1.29it/s, gpt_loss=0.127, lr=3.47e-6]
+
Train step of epoch 0:  42%|████▏     | 2333/5616 [29:44<42:26,  1.29it/s, gpt_loss=0.0512, lr=3.47e-6]
+
Train step of epoch 0:  42%|████▏     | 2334/5616 [29:44<43:06,  1.27it/s, gpt_loss=0.0512, lr=3.47e-6]
+
Train step of epoch 0:  42%|████▏     | 2334/5616 [29:44<43:06,  1.27it/s, gpt_loss=0.0352, lr=3.47e-6]
+
Train step of epoch 0:  42%|████▏     | 2335/5616 [29:44<41:31,  1.32it/s, gpt_loss=0.0352, lr=3.47e-6]
+
Train step of epoch 0:  42%|████▏     | 2335/5616 [29:45<41:31,  1.32it/s, gpt_loss=0.0263, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2336/5616 [29:45<41:44,  1.31it/s, gpt_loss=0.0263, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2336/5616 [29:46<41:44,  1.31it/s, gpt_loss=0.0437, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2337/5616 [29:46<39:34,  1.38it/s, gpt_loss=0.0437, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2337/5616 [29:46<39:34,  1.38it/s, gpt_loss=0.0256, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2338/5616 [29:46<38:33,  1.42it/s, gpt_loss=0.0256, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2338/5616 [29:47<38:33,  1.42it/s, gpt_loss=0.0415, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2339/5616 [29:47<39:28,  1.38it/s, gpt_loss=0.0415, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2339/5616 [29:48<39:28,  1.38it/s, gpt_loss=0.2, lr=3.46e-6]   
+
Train step of epoch 0:  42%|████▏     | 2340/5616 [29:48<40:07,  1.36it/s, gpt_loss=0.2, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2340/5616 [29:49<40:07,  1.36it/s, gpt_loss=0.0124, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2341/5616 [29:49<39:29,  1.38it/s, gpt_loss=0.0124, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2341/5616 [29:49<39:29,  1.38it/s, gpt_loss=0.0219, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2342/5616 [29:49<39:22,  1.39it/s, gpt_loss=0.0219, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2342/5616 [29:50<39:22,  1.39it/s, gpt_loss=0.00819, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2343/5616 [29:50<38:54,  1.40it/s, gpt_loss=0.00819, lr=3.46e-6]
+
Train step of epoch 0:  42%|████▏     | 2343/5616 [29:51<38:54,  1.40it/s, gpt_loss=0.129, lr=3.45e-6]  
+
Train step of epoch 0:  42%|████▏     | 2344/5616 [29:51<39:07,  1.39it/s, gpt_loss=0.129, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2344/5616 [29:52<39:07,  1.39it/s, gpt_loss=0.0397, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2345/5616 [29:52<38:56,  1.40it/s, gpt_loss=0.0397, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2345/5616 [29:52<38:56,  1.40it/s, gpt_loss=0.0757, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2346/5616 [29:52<40:22,  1.35it/s, gpt_loss=0.0757, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2346/5616 [29:53<40:22,  1.35it/s, gpt_loss=0.0377, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2347/5616 [29:53<40:04,  1.36it/s, gpt_loss=0.0377, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2347/5616 [29:54<40:04,  1.36it/s, gpt_loss=0.0513, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2348/5616 [29:54<42:52,  1.27it/s, gpt_loss=0.0513, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2348/5616 [29:55<42:52,  1.27it/s, gpt_loss=0.0374, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2349/5616 [29:55<47:02,  1.16it/s, gpt_loss=0.0374, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2349/5616 [29:56<47:02,  1.16it/s, gpt_loss=0.0143, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2350/5616 [29:56<45:35,  1.19it/s, gpt_loss=0.0143, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2350/5616 [29:57<45:35,  1.19it/s, gpt_loss=0.0687, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2351/5616 [29:57<43:47,  1.24it/s, gpt_loss=0.0687, lr=3.45e-6]
+
Train step of epoch 0:  42%|████▏     | 2351/5616 [29:57<43:47,  1.24it/s, gpt_loss=0.0163, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2352/5616 [29:57<43:51,  1.24it/s, gpt_loss=0.0163, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2352/5616 [29:58<43:51,  1.24it/s, gpt_loss=0.0637, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2353/5616 [29:58<41:26,  1.31it/s, gpt_loss=0.0637, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2353/5616 [29:59<41:26,  1.31it/s, gpt_loss=0.0462, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2354/5616 [29:59<45:13,  1.20it/s, gpt_loss=0.0462, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2354/5616 [30:00<45:13,  1.20it/s, gpt_loss=0.0115, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2355/5616 [30:00<46:01,  1.18it/s, gpt_loss=0.0115, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2355/5616 [30:01<46:01,  1.18it/s, gpt_loss=0.0238, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2356/5616 [30:01<45:24,  1.20it/s, gpt_loss=0.0238, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2356/5616 [30:01<45:24,  1.20it/s, gpt_loss=0.128, lr=3.44e-6] 
+
Train step of epoch 0:  42%|████▏     | 2357/5616 [30:01<43:01,  1.26it/s, gpt_loss=0.128, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2357/5616 [30:02<43:01,  1.26it/s, gpt_loss=0.029, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2358/5616 [30:02<41:22,  1.31it/s, gpt_loss=0.029, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2358/5616 [30:03<41:22,  1.31it/s, gpt_loss=0.0369, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2359/5616 [30:03<40:07,  1.35it/s, gpt_loss=0.0369, lr=3.44e-6]
+
Train step of epoch 0:  42%|████▏     | 2359/5616 [30:04<40:07,  1.35it/s, gpt_loss=0.0234, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2360/5616 [30:04<40:46,  1.33it/s, gpt_loss=0.0234, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2360/5616 [30:04<40:46,  1.33it/s, gpt_loss=0.0245, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2361/5616 [30:04<40:27,  1.34it/s, gpt_loss=0.0245, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2361/5616 [30:05<40:27,  1.34it/s, gpt_loss=0.019, lr=3.43e-6] 
+
Train step of epoch 0:  42%|████▏     | 2362/5616 [30:05<39:24,  1.38it/s, gpt_loss=0.019, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2362/5616 [30:06<39:24,  1.38it/s, gpt_loss=0.00483, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2363/5616 [30:06<39:08,  1.38it/s, gpt_loss=0.00483, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2363/5616 [30:06<39:08,  1.38it/s, gpt_loss=0.0342, lr=3.43e-6] 
+
Train step of epoch 0:  42%|████▏     | 2364/5616 [30:06<40:31,  1.34it/s, gpt_loss=0.0342, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2364/5616 [30:07<40:31,  1.34it/s, gpt_loss=0.0556, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2365/5616 [30:07<40:24,  1.34it/s, gpt_loss=0.0556, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2365/5616 [30:08<40:24,  1.34it/s, gpt_loss=0.0883, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2366/5616 [30:08<39:37,  1.37it/s, gpt_loss=0.0883, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2366/5616 [30:09<39:37,  1.37it/s, gpt_loss=0.0544, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2367/5616 [30:09<39:29,  1.37it/s, gpt_loss=0.0544, lr=3.43e-6]
+
Train step of epoch 0:  42%|████▏     | 2367/5616 [30:09<39:29,  1.37it/s, gpt_loss=0.0666, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2368/5616 [30:09<41:01,  1.32it/s, gpt_loss=0.0666, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2368/5616 [30:10<41:01,  1.32it/s, gpt_loss=0.0433, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2369/5616 [30:10<40:08,  1.35it/s, gpt_loss=0.0433, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2369/5616 [30:11<40:08,  1.35it/s, gpt_loss=0.0218, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2370/5616 [30:11<38:18,  1.41it/s, gpt_loss=0.0218, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2370/5616 [30:11<38:18,  1.41it/s, gpt_loss=0.0515, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2371/5616 [30:11<38:08,  1.42it/s, gpt_loss=0.0515, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2371/5616 [30:12<38:08,  1.42it/s, gpt_loss=0.0246, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2372/5616 [30:12<39:47,  1.36it/s, gpt_loss=0.0246, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2372/5616 [30:13<39:47,  1.36it/s, gpt_loss=0.0474, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2373/5616 [30:13<38:47,  1.39it/s, gpt_loss=0.0474, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2373/5616 [30:14<38:47,  1.39it/s, gpt_loss=0.0166, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2374/5616 [30:14<39:44,  1.36it/s, gpt_loss=0.0166, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2374/5616 [30:15<39:44,  1.36it/s, gpt_loss=0.0218, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2375/5616 [30:15<41:04,  1.32it/s, gpt_loss=0.0218, lr=3.42e-6]
+
Train step of epoch 0:  42%|████▏     | 2375/5616 [30:16<41:04,  1.32it/s, gpt_loss=0.0182, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2376/5616 [30:16<49:35,  1.09it/s, gpt_loss=0.0182, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2376/5616 [30:17<49:35,  1.09it/s, gpt_loss=0.0354, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2377/5616 [30:17<46:43,  1.16it/s, gpt_loss=0.0354, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2377/5616 [30:17<46:43,  1.16it/s, gpt_loss=0.0852, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2378/5616 [30:17<43:47,  1.23it/s, gpt_loss=0.0852, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2378/5616 [30:18<43:47,  1.23it/s, gpt_loss=0.153, lr=3.41e-6] 
+
Train step of epoch 0:  42%|████▏     | 2379/5616 [30:18<41:13,  1.31it/s, gpt_loss=0.153, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2379/5616 [30:19<41:13,  1.31it/s, gpt_loss=0.0454, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2380/5616 [30:19<40:29,  1.33it/s, gpt_loss=0.0454, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2380/5616 [30:19<40:29,  1.33it/s, gpt_loss=0.0145, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2381/5616 [30:19<39:28,  1.37it/s, gpt_loss=0.0145, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2381/5616 [30:20<39:28,  1.37it/s, gpt_loss=0.0119, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2382/5616 [30:20<39:31,  1.36it/s, gpt_loss=0.0119, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2382/5616 [30:21<39:31,  1.36it/s, gpt_loss=0.0501, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2383/5616 [30:21<42:04,  1.28it/s, gpt_loss=0.0501, lr=3.41e-6]
+
Train step of epoch 0:  42%|████▏     | 2383/5616 [30:22<42:04,  1.28it/s, gpt_loss=0.0466, lr=3.4e-6] 
+
Train step of epoch 0:  42%|████▏     | 2384/5616 [30:22<41:54,  1.29it/s, gpt_loss=0.0466, lr=3.4e-6]
+
Train step of epoch 0:  42%|████▏     | 2384/5616 [30:23<41:54,  1.29it/s, gpt_loss=0.0324, lr=3.4e-6]
+
Train step of epoch 0:  42%|████▏     | 2385/5616 [30:23<41:47,  1.29it/s, gpt_loss=0.0324, lr=3.4e-6]
+
Train step of epoch 0:  42%|████▏     | 2385/5616 [30:23<41:47,  1.29it/s, gpt_loss=0.168, lr=3.4e-6] 
+
Train step of epoch 0:  42%|████▏     | 2386/5616 [30:23<40:16,  1.34it/s, gpt_loss=0.168, lr=3.4e-6]
+
Train step of epoch 0:  42%|████▏     | 2386/5616 [30:24<40:16,  1.34it/s, gpt_loss=0.0971, lr=3.4e-6]
+
Train step of epoch 0:  43%|████▎     | 2387/5616 [30:24<40:33,  1.33it/s, gpt_loss=0.0971, lr=3.4e-6]
+
Train step of epoch 0:  43%|████▎     | 2387/5616 [30:25<40:33,  1.33it/s, gpt_loss=0.15, lr=3.4e-6]  
+
Train step of epoch 0:  43%|████▎     | 2388/5616 [30:25<41:47,  1.29it/s, gpt_loss=0.15, lr=3.4e-6]
+
Train step of epoch 0:  43%|████▎     | 2388/5616 [30:25<41:47,  1.29it/s, gpt_loss=0.099, lr=3.4e-6]
+
Train step of epoch 0:  43%|████▎     | 2389/5616 [30:25<40:20,  1.33it/s, gpt_loss=0.099, lr=3.4e-6]
+
Train step of epoch 0:  43%|████▎     | 2389/5616 [30:26<40:20,  1.33it/s, gpt_loss=0.13, lr=3.4e-6] 
+
Train step of epoch 0:  43%|████▎     | 2390/5616 [30:26<41:46,  1.29it/s, gpt_loss=0.13, lr=3.4e-6]
+
Train step of epoch 0:  43%|████▎     | 2390/5616 [30:27<41:46,  1.29it/s, gpt_loss=0.0588, lr=3.4e-6]
+
Train step of epoch 0:  43%|████▎     | 2391/5616 [30:27<40:16,  1.33it/s, gpt_loss=0.0588, lr=3.4e-6]
+
Train step of epoch 0:  43%|████▎     | 2391/5616 [30:28<40:16,  1.33it/s, gpt_loss=0.0562, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2392/5616 [30:28<41:03,  1.31it/s, gpt_loss=0.0562, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2392/5616 [30:28<41:03,  1.31it/s, gpt_loss=0.0739, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2393/5616 [30:28<39:49,  1.35it/s, gpt_loss=0.0739, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2393/5616 [30:29<39:49,  1.35it/s, gpt_loss=0.156, lr=3.39e-6] 
+
Train step of epoch 0:  43%|████▎     | 2394/5616 [30:29<38:36,  1.39it/s, gpt_loss=0.156, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2394/5616 [30:30<38:36,  1.39it/s, gpt_loss=0.0289, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2395/5616 [30:30<37:14,  1.44it/s, gpt_loss=0.0289, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2395/5616 [30:31<37:14,  1.44it/s, gpt_loss=0.0599, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2396/5616 [30:31<37:39,  1.42it/s, gpt_loss=0.0599, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2396/5616 [30:31<37:39,  1.42it/s, gpt_loss=0.0236, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2397/5616 [30:31<37:35,  1.43it/s, gpt_loss=0.0236, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2397/5616 [30:32<37:35,  1.43it/s, gpt_loss=0.0982, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2398/5616 [30:32<38:53,  1.38it/s, gpt_loss=0.0982, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2398/5616 [30:33<38:53,  1.38it/s, gpt_loss=0.0108, lr=3.39e-6]
+
Train step of epoch 0:  43%|████▎     | 2399/5616 [30:33<38:56,  1.38it/s, gpt_loss=0.0108, lr=3.39e-6][2025-06-30 07:08:42,490] [INFO] [logging.py:128:log_dist] [Rank 0] step=600, skipped=0, lr=[3.382765600058964e-06, 3.382765600058964e-06], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 07:08:42,491] [INFO] [timer.py:264:stop] epoch=0/micro_step=2400/global_step=600, RunningAvgSamplesPerSec=42.448476859989654, CurrSamplesPerSec=44.12196463648819, MemAllocated=15.88GB, MaxMemAllocated=51.79GB
+
+
Train step of epoch 0:  43%|████▎     | 2399/5616 [30:33<38:56,  1.38it/s, gpt_loss=0.131, lr=3.38e-6] 
+
Train step of epoch 0:  43%|████▎     | 2400/5616 [30:33<39:00,  1.37it/s, gpt_loss=0.131, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2400/5616 [30:34<39:00,  1.37it/s, gpt_loss=0.0161, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2401/5616 [30:34<40:31,  1.32it/s, gpt_loss=0.0161, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2401/5616 [30:35<40:31,  1.32it/s, gpt_loss=0.0248, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2402/5616 [30:35<39:42,  1.35it/s, gpt_loss=0.0248, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2402/5616 [30:36<39:42,  1.35it/s, gpt_loss=0.0452, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2403/5616 [30:36<38:46,  1.38it/s, gpt_loss=0.0452, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2403/5616 [30:36<38:46,  1.38it/s, gpt_loss=0.0316, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2404/5616 [30:36<40:28,  1.32it/s, gpt_loss=0.0316, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2404/5616 [30:37<40:28,  1.32it/s, gpt_loss=0.0464, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2405/5616 [30:37<39:28,  1.36it/s, gpt_loss=0.0464, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2405/5616 [30:38<39:28,  1.36it/s, gpt_loss=0.0229, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2406/5616 [30:38<40:27,  1.32it/s, gpt_loss=0.0229, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2406/5616 [30:39<40:27,  1.32it/s, gpt_loss=0.00724, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2407/5616 [30:39<39:26,  1.36it/s, gpt_loss=0.00724, lr=3.38e-6]
+
Train step of epoch 0:  43%|████▎     | 2407/5616 [30:39<39:26,  1.36it/s, gpt_loss=0.132, lr=3.37e-6]  
+
Train step of epoch 0:  43%|████▎     | 2408/5616 [30:39<39:36,  1.35it/s, gpt_loss=0.132, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2408/5616 [30:40<39:36,  1.35it/s, gpt_loss=0.176, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2409/5616 [30:40<39:51,  1.34it/s, gpt_loss=0.176, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2409/5616 [30:41<39:51,  1.34it/s, gpt_loss=0.0592, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2410/5616 [30:41<40:39,  1.31it/s, gpt_loss=0.0592, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2410/5616 [30:42<40:39,  1.31it/s, gpt_loss=0.0418, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2411/5616 [30:42<41:18,  1.29it/s, gpt_loss=0.0418, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2411/5616 [30:43<41:18,  1.29it/s, gpt_loss=0.00458, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2412/5616 [30:43<40:51,  1.31it/s, gpt_loss=0.00458, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2412/5616 [30:43<40:51,  1.31it/s, gpt_loss=0.193, lr=3.37e-6]  
+
Train step of epoch 0:  43%|████▎     | 2413/5616 [30:43<39:56,  1.34it/s, gpt_loss=0.193, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2413/5616 [30:44<39:56,  1.34it/s, gpt_loss=0.102, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2414/5616 [30:44<40:51,  1.31it/s, gpt_loss=0.102, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2414/5616 [30:45<40:51,  1.31it/s, gpt_loss=0.0339, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2415/5616 [30:45<39:10,  1.36it/s, gpt_loss=0.0339, lr=3.37e-6]
+
Train step of epoch 0:  43%|████▎     | 2415/5616 [30:45<39:10,  1.36it/s, gpt_loss=0.0516, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2416/5616 [30:45<39:38,  1.35it/s, gpt_loss=0.0516, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2416/5616 [30:46<39:38,  1.35it/s, gpt_loss=0.0509, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2417/5616 [30:46<40:59,  1.30it/s, gpt_loss=0.0509, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2417/5616 [30:47<40:59,  1.30it/s, gpt_loss=0.00744, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2418/5616 [30:47<39:19,  1.36it/s, gpt_loss=0.00744, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2418/5616 [30:48<39:19,  1.36it/s, gpt_loss=0.2, lr=3.36e-6]    
+
Train step of epoch 0:  43%|████▎     | 2419/5616 [30:48<49:04,  1.09it/s, gpt_loss=0.2, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2419/5616 [30:49<49:04,  1.09it/s, gpt_loss=0.142, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2420/5616 [30:49<47:42,  1.12it/s, gpt_loss=0.142, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2420/5616 [30:50<47:42,  1.12it/s, gpt_loss=0.0542, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2421/5616 [30:50<46:03,  1.16it/s, gpt_loss=0.0542, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2421/5616 [30:51<46:03,  1.16it/s, gpt_loss=0.0109, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2422/5616 [30:51<43:45,  1.22it/s, gpt_loss=0.0109, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2422/5616 [30:51<43:45,  1.22it/s, gpt_loss=0.0243, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2423/5616 [30:51<42:31,  1.25it/s, gpt_loss=0.0243, lr=3.36e-6]
+
Train step of epoch 0:  43%|████▎     | 2423/5616 [30:52<42:31,  1.25it/s, gpt_loss=0.0999, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2424/5616 [30:52<41:55,  1.27it/s, gpt_loss=0.0999, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2424/5616 [30:53<41:55,  1.27it/s, gpt_loss=0.0195, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2425/5616 [30:53<40:20,  1.32it/s, gpt_loss=0.0195, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2425/5616 [30:54<40:20,  1.32it/s, gpt_loss=0.0209, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2426/5616 [30:54<39:06,  1.36it/s, gpt_loss=0.0209, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2426/5616 [30:54<39:06,  1.36it/s, gpt_loss=0.0708, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2427/5616 [30:54<37:45,  1.41it/s, gpt_loss=0.0708, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2427/5616 [30:55<37:45,  1.41it/s, gpt_loss=0.00583, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2428/5616 [30:55<38:18,  1.39it/s, gpt_loss=0.00583, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2428/5616 [30:56<38:18,  1.39it/s, gpt_loss=0.0161, lr=3.35e-6] 
+
Train step of epoch 0:  43%|████▎     | 2429/5616 [30:56<37:59,  1.40it/s, gpt_loss=0.0161, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2429/5616 [30:56<37:59,  1.40it/s, gpt_loss=0.0407, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2430/5616 [30:56<37:56,  1.40it/s, gpt_loss=0.0407, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2430/5616 [30:57<37:56,  1.40it/s, gpt_loss=0.102, lr=3.35e-6] 
+
Train step of epoch 0:  43%|████▎     | 2431/5616 [30:57<39:51,  1.33it/s, gpt_loss=0.102, lr=3.35e-6]
+
Train step of epoch 0:  43%|████▎     | 2431/5616 [30:58<39:51,  1.33it/s, gpt_loss=0.0178, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2432/5616 [30:58<40:24,  1.31it/s, gpt_loss=0.0178, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2432/5616 [30:59<40:24,  1.31it/s, gpt_loss=0.0292, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2433/5616 [30:59<41:10,  1.29it/s, gpt_loss=0.0292, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2433/5616 [31:00<41:10,  1.29it/s, gpt_loss=0.127, lr=3.34e-6] 
+
Train step of epoch 0:  43%|████▎     | 2434/5616 [31:00<40:30,  1.31it/s, gpt_loss=0.127, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2434/5616 [31:00<40:30,  1.31it/s, gpt_loss=0.0171, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2435/5616 [31:00<38:31,  1.38it/s, gpt_loss=0.0171, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2435/5616 [31:01<38:31,  1.38it/s, gpt_loss=0.0428, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2436/5616 [31:01<39:24,  1.34it/s, gpt_loss=0.0428, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2436/5616 [31:02<39:24,  1.34it/s, gpt_loss=0.0342, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2437/5616 [31:02<38:36,  1.37it/s, gpt_loss=0.0342, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2437/5616 [31:02<38:36,  1.37it/s, gpt_loss=0.0912, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2438/5616 [31:02<37:23,  1.42it/s, gpt_loss=0.0912, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2438/5616 [31:03<37:23,  1.42it/s, gpt_loss=0.147, lr=3.34e-6] 
+
Train step of epoch 0:  43%|████▎     | 2439/5616 [31:03<37:24,  1.42it/s, gpt_loss=0.147, lr=3.34e-6]
+
Train step of epoch 0:  43%|████▎     | 2439/5616 [31:04<37:24,  1.42it/s, gpt_loss=0.0674, lr=3.33e-6]
+
Train step of epoch 0:  43%|████▎     | 2440/5616 [31:04<38:29,  1.38it/s, gpt_loss=0.0674, lr=3.33e-6]
+
Train step of epoch 0:  43%|████▎     | 2440/5616 [31:05<38:29,  1.38it/s, gpt_loss=0.0433, lr=3.33e-6]
+
Train step of epoch 0:  43%|████▎     | 2441/5616 [31:05<43:51,  1.21it/s, gpt_loss=0.0433, lr=3.33e-6]
+
Train step of epoch 0:  43%|████▎     | 2441/5616 [31:06<43:51,  1.21it/s, gpt_loss=0.0748, lr=3.33e-6]
+
Train step of epoch 0:  43%|████▎     | 2442/5616 [31:06<42:32,  1.24it/s, gpt_loss=0.0748, lr=3.33e-6]
+
Train step of epoch 0:  43%|████▎     | 2442/5616 [31:06<42:32,  1.24it/s, gpt_loss=0.0558, lr=3.33e-6]
+
Train step of epoch 0:  44%|████▎     | 2443/5616 [31:06<41:35,  1.27it/s, gpt_loss=0.0558, lr=3.33e-6]
+
Train step of epoch 0:  44%|████▎     | 2443/5616 [31:07<41:35,  1.27it/s, gpt_loss=0.0732, lr=3.33e-6]
+
Train step of epoch 0:  44%|████▎     | 2444/5616 [31:07<40:59,  1.29it/s, gpt_loss=0.0732, lr=3.33e-6]
+
Train step of epoch 0:  44%|████▎     | 2444/5616 [31:08<40:59,  1.29it/s, gpt_loss=0.0412, lr=3.33e-6]
+
Train step of epoch 0:  44%|████▎     | 2445/5616 [31:08<43:33,  1.21it/s, gpt_loss=0.0412, lr=3.33e-6]
+
Train step of epoch 0:  44%|████▎     | 2445/5616 [31:09<43:33,  1.21it/s, gpt_loss=0.028, lr=3.33e-6] 
+
Train step of epoch 0:  44%|████▎     | 2446/5616 [31:09<43:09,  1.22it/s, gpt_loss=0.028, lr=3.33e-6]
+
Train step of epoch 0:  44%|████▎     | 2446/5616 [31:10<43:09,  1.22it/s, gpt_loss=0.0388, lr=3.33e-6]
+
Train step of epoch 0:  44%|████▎     | 2447/5616 [31:10<41:52,  1.26it/s, gpt_loss=0.0388, lr=3.33e-6]
+
Train step of epoch 0:  44%|████▎     | 2447/5616 [31:10<41:52,  1.26it/s, gpt_loss=0.0255, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2448/5616 [31:10<43:13,  1.22it/s, gpt_loss=0.0255, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2448/5616 [31:11<43:13,  1.22it/s, gpt_loss=0.0293, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2449/5616 [31:11<45:20,  1.16it/s, gpt_loss=0.0293, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2449/5616 [31:12<45:20,  1.16it/s, gpt_loss=0.14, lr=3.32e-6]  
+
Train step of epoch 0:  44%|████▎     | 2450/5616 [31:12<42:21,  1.25it/s, gpt_loss=0.14, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2450/5616 [31:13<42:21,  1.25it/s, gpt_loss=0.0706, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2451/5616 [31:13<40:08,  1.31it/s, gpt_loss=0.0706, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2451/5616 [31:13<40:08,  1.31it/s, gpt_loss=0.0104, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2452/5616 [31:13<39:37,  1.33it/s, gpt_loss=0.0104, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2452/5616 [31:14<39:37,  1.33it/s, gpt_loss=0.0276, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2453/5616 [31:14<37:42,  1.40it/s, gpt_loss=0.0276, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2453/5616 [31:15<37:42,  1.40it/s, gpt_loss=0.0614, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2454/5616 [31:15<37:07,  1.42it/s, gpt_loss=0.0614, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2454/5616 [31:16<37:07,  1.42it/s, gpt_loss=0.0131, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2455/5616 [31:16<37:52,  1.39it/s, gpt_loss=0.0131, lr=3.32e-6]
+
Train step of epoch 0:  44%|████▎     | 2455/5616 [31:16<37:52,  1.39it/s, gpt_loss=0.0207, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▎     | 2456/5616 [31:16<39:37,  1.33it/s, gpt_loss=0.0207, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▎     | 2456/5616 [31:17<39:37,  1.33it/s, gpt_loss=0.0188, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2457/5616 [31:17<38:56,  1.35it/s, gpt_loss=0.0188, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2457/5616 [31:18<38:56,  1.35it/s, gpt_loss=0.0158, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2458/5616 [31:18<38:09,  1.38it/s, gpt_loss=0.0158, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2458/5616 [31:18<38:09,  1.38it/s, gpt_loss=0.0264, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2459/5616 [31:18<37:55,  1.39it/s, gpt_loss=0.0264, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2459/5616 [31:19<37:55,  1.39it/s, gpt_loss=0.0384, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2460/5616 [31:19<39:35,  1.33it/s, gpt_loss=0.0384, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2460/5616 [31:20<39:35,  1.33it/s, gpt_loss=0.0153, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2461/5616 [31:20<38:32,  1.36it/s, gpt_loss=0.0153, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2461/5616 [31:21<38:32,  1.36it/s, gpt_loss=0.0103, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2462/5616 [31:21<43:19,  1.21it/s, gpt_loss=0.0103, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2462/5616 [31:22<43:19,  1.21it/s, gpt_loss=0.0452, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2463/5616 [31:22<41:08,  1.28it/s, gpt_loss=0.0452, lr=3.31e-6]
+
Train step of epoch 0:  44%|████▍     | 2463/5616 [31:22<41:08,  1.28it/s, gpt_loss=0.0163, lr=3.3e-6] 
+
Train step of epoch 0:  44%|████▍     | 2464/5616 [31:22<40:44,  1.29it/s, gpt_loss=0.0163, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2464/5616 [31:23<40:44,  1.29it/s, gpt_loss=0.0483, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2465/5616 [31:23<41:09,  1.28it/s, gpt_loss=0.0483, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2465/5616 [31:24<41:09,  1.28it/s, gpt_loss=0.00534, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2466/5616 [31:24<40:29,  1.30it/s, gpt_loss=0.00534, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2466/5616 [31:25<40:29,  1.30it/s, gpt_loss=0.0581, lr=3.3e-6] 
+
Train step of epoch 0:  44%|████▍     | 2467/5616 [31:25<43:35,  1.20it/s, gpt_loss=0.0581, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2467/5616 [31:26<43:35,  1.20it/s, gpt_loss=0.0609, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2468/5616 [31:26<42:37,  1.23it/s, gpt_loss=0.0609, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2468/5616 [31:26<42:37,  1.23it/s, gpt_loss=0.0416, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2469/5616 [31:26<40:55,  1.28it/s, gpt_loss=0.0416, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2469/5616 [31:27<40:55,  1.28it/s, gpt_loss=0.0135, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2470/5616 [31:27<39:48,  1.32it/s, gpt_loss=0.0135, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2470/5616 [31:28<39:48,  1.32it/s, gpt_loss=0.0504, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2471/5616 [31:28<41:22,  1.27it/s, gpt_loss=0.0504, lr=3.3e-6]
+
Train step of epoch 0:  44%|████▍     | 2471/5616 [31:29<41:22,  1.27it/s, gpt_loss=0.0448, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2472/5616 [31:29<41:13,  1.27it/s, gpt_loss=0.0448, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2472/5616 [31:29<41:13,  1.27it/s, gpt_loss=0.0104, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2473/5616 [31:29<39:16,  1.33it/s, gpt_loss=0.0104, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2473/5616 [31:30<39:16,  1.33it/s, gpt_loss=0.0591, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2474/5616 [31:30<39:06,  1.34it/s, gpt_loss=0.0591, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2474/5616 [31:31<39:06,  1.34it/s, gpt_loss=0.0171, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2475/5616 [31:31<38:57,  1.34it/s, gpt_loss=0.0171, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2475/5616 [31:32<38:57,  1.34it/s, gpt_loss=0.00515, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2476/5616 [31:32<40:22,  1.30it/s, gpt_loss=0.00515, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2476/5616 [31:32<40:22,  1.30it/s, gpt_loss=0.074, lr=3.29e-6]  
+
Train step of epoch 0:  44%|████▍     | 2477/5616 [31:32<39:29,  1.32it/s, gpt_loss=0.074, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2477/5616 [31:33<39:29,  1.32it/s, gpt_loss=0.0307, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2478/5616 [31:33<40:30,  1.29it/s, gpt_loss=0.0307, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2478/5616 [31:34<40:30,  1.29it/s, gpt_loss=0.129, lr=3.29e-6] 
+
Train step of epoch 0:  44%|████▍     | 2479/5616 [31:34<39:46,  1.31it/s, gpt_loss=0.129, lr=3.29e-6]
+
Train step of epoch 0:  44%|████▍     | 2479/5616 [31:35<39:46,  1.31it/s, gpt_loss=0.0147, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2480/5616 [31:35<39:44,  1.32it/s, gpt_loss=0.0147, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2480/5616 [31:35<39:44,  1.32it/s, gpt_loss=0.0128, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2481/5616 [31:35<37:57,  1.38it/s, gpt_loss=0.0128, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2481/5616 [31:36<37:57,  1.38it/s, gpt_loss=0.0871, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2482/5616 [31:36<38:09,  1.37it/s, gpt_loss=0.0871, lr=3.28e-6]
+
Train step of epoch 0:  44%|████���     | 2482/5616 [31:37<38:09,  1.37it/s, gpt_loss=0.0399, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2483/5616 [31:37<37:43,  1.38it/s, gpt_loss=0.0399, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2483/5616 [31:38<37:43,  1.38it/s, gpt_loss=0.117, lr=3.28e-6] 
+
Train step of epoch 0:  44%|████▍     | 2484/5616 [31:38<38:13,  1.37it/s, gpt_loss=0.117, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2484/5616 [31:38<38:13,  1.37it/s, gpt_loss=0.0491, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2485/5616 [31:38<38:50,  1.34it/s, gpt_loss=0.0491, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2485/5616 [31:39<38:50,  1.34it/s, gpt_loss=0.0299, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2486/5616 [31:39<41:39,  1.25it/s, gpt_loss=0.0299, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2486/5616 [31:40<41:39,  1.25it/s, gpt_loss=0.00419, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2487/5616 [31:40<41:41,  1.25it/s, gpt_loss=0.00419, lr=3.28e-6]
+
Train step of epoch 0:  44%|████▍     | 2487/5616 [31:41<41:41,  1.25it/s, gpt_loss=0.0293, lr=3.27e-6] 
+
Train step of epoch 0:  44%|████▍     | 2488/5616 [31:41<42:39,  1.22it/s, gpt_loss=0.0293, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2488/5616 [31:42<42:39,  1.22it/s, gpt_loss=0.0083, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2489/5616 [31:42<40:38,  1.28it/s, gpt_loss=0.0083, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2489/5616 [31:42<40:38,  1.28it/s, gpt_loss=0.0366, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2490/5616 [31:42<40:18,  1.29it/s, gpt_loss=0.0366, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2490/5616 [31:43<40:18,  1.29it/s, gpt_loss=0.0645, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2491/5616 [31:43<41:02,  1.27it/s, gpt_loss=0.0645, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2491/5616 [31:44<41:02,  1.27it/s, gpt_loss=0.00744, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2492/5616 [31:44<41:24,  1.26it/s, gpt_loss=0.00744, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2492/5616 [31:45<41:24,  1.26it/s, gpt_loss=0.00917, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2493/5616 [31:45<40:40,  1.28it/s, gpt_loss=0.00917, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2493/5616 [31:46<40:40,  1.28it/s, gpt_loss=0.00587, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2494/5616 [31:46<42:54,  1.21it/s, gpt_loss=0.00587, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2494/5616 [31:47<42:54,  1.21it/s, gpt_loss=0.0214, lr=3.27e-6] 
+
Train step of epoch 0:  44%|████▍     | 2495/5616 [31:47<41:45,  1.25it/s, gpt_loss=0.0214, lr=3.27e-6]
+
Train step of epoch 0:  44%|████▍     | 2495/5616 [31:48<41:45,  1.25it/s, gpt_loss=0.0517, lr=3.26e-6]
+
Train step of epoch 0:  44%|████▍     | 2496/5616 [31:48<45:11,  1.15it/s, gpt_loss=0.0517, lr=3.26e-6]
+
Train step of epoch 0:  44%|████▍     | 2496/5616 [31:48<45:11,  1.15it/s, gpt_loss=0.0307, lr=3.26e-6]
+
Train step of epoch 0:  44%|████▍     | 2497/5616 [31:48<42:55,  1.21it/s, gpt_loss=0.0307, lr=3.26e-6]
+
Train step of epoch 0:  44%|████▍     | 2497/5616 [31:49<42:55,  1.21it/s, gpt_loss=0.00419, lr=3.26e-6]
+
Train step of epoch 0:  44%|████▍     | 2498/5616 [31:49<41:02,  1.27it/s, gpt_loss=0.00419, lr=3.26e-6]
+
Train step of epoch 0:  44%|████▍     | 2498/5616 [31:50<41:02,  1.27it/s, gpt_loss=0.0193, lr=3.26e-6] 
+
Train step of epoch 0:  44%|████▍     | 2499/5616 [31:50<47:04,  1.10it/s, gpt_loss=0.0193, lr=3.26e-6]
+
Train step of epoch 0:  44%|████▍     | 2499/5616 [31:51<47:04,  1.10it/s, gpt_loss=0.00591, lr=3.26e-6]
+
Train step of epoch 0:  45%|████▍     | 2500/5616 [31:51<45:06,  1.15it/s, gpt_loss=0.00591, lr=3.26e-6]
+
Train step of epoch 0:  45%|████▍     | 2500/5616 [31:52<45:06,  1.15it/s, gpt_loss=0.103, lr=3.26e-6]  
+
Train step of epoch 0:  45%|████▍     | 2501/5616 [31:52<43:43,  1.19it/s, gpt_loss=0.103, lr=3.26e-6]
+
Train step of epoch 0:  45%|████▍     | 2501/5616 [31:52<43:43,  1.19it/s, gpt_loss=0.00313, lr=3.26e-6]
+
Train step of epoch 0:  45%|████▍     | 2502/5616 [31:52<41:20,  1.26it/s, gpt_loss=0.00313, lr=3.26e-6]
+
Train step of epoch 0:  45%|████▍     | 2502/5616 [31:53<41:20,  1.26it/s, gpt_loss=0.205, lr=3.26e-6]  
+
Train step of epoch 0:  45%|████▍     | 2503/5616 [31:53<42:13,  1.23it/s, gpt_loss=0.205, lr=3.26e-6]
+
Train step of epoch 0:  45%|████▍     | 2503/5616 [31:54<42:13,  1.23it/s, gpt_loss=0.288, lr=3.25e-6]
+
Train step of epoch 0:  45%|���███▍     | 2504/5616 [31:54<41:38,  1.25it/s, gpt_loss=0.288, lr=3.25e-6]
+
Train step of epoch 0:  45%|████▍     | 2504/5616 [31:55<41:38,  1.25it/s, gpt_loss=0.0792, lr=3.25e-6]
+
Train step of epoch 0:  45%|████▍     | 2505/5616 [31:55<40:01,  1.30it/s, gpt_loss=0.0792, lr=3.25e-6]
+
Train step of epoch 0:  45%|████▍     | 2505/5616 [31:56<40:01,  1.30it/s, gpt_loss=0.0687, lr=3.25e-6]
+
Train step of epoch 0:  45%|████▍     | 2506/5616 [31:56<40:25,  1.28it/s, gpt_loss=0.0687, lr=3.25e-6]
+
Train step of epoch 0:  45%|████▍     | 2506/5616 [31:56<40:25,  1.28it/s, gpt_loss=0.00325, lr=3.25e-6]
+
Train step of epoch 0:  45%|████▍     | 2507/5616 [31:56<38:42,  1.34it/s, gpt_loss=0.00325, lr=3.25e-6]
+
Train step of epoch 0:  45%|████▍     | 2507/5616 [31:57<38:42,  1.34it/s, gpt_loss=0.0378, lr=3.25e-6] 
+
Train step of epoch 0:  45%|████▍     | 2508/5616 [31:57<40:55,  1.27it/s, gpt_loss=0.0378, lr=3.25e-6]
+
Train step of epoch 0:  45%|████▍     | 2508/5616 [31:58<40:55,  1.27it/s, gpt_loss=0.157, lr=3.25e-6] 
+
Train step of epoch 0:  45%|████▍     | 2509/5616 [31:58<42:06,  1.23it/s, gpt_loss=0.157, lr=3.25e-6]
+
Train step of epoch 0:  45%|████▍     | 2509/5616 [31:59<42:06,  1.23it/s, gpt_loss=0.0255, lr=3.25e-6]
+
Train step of epoch 0:  45%|████▍     | 2510/5616 [31:59<39:53,  1.30it/s, gpt_loss=0.0255, lr=3.25e-6]
+
Train step of epoch 0:  45%|████▍     | 2510/5616 [31:59<39:53,  1.30it/s, gpt_loss=0.0292, lr=3.25e-6]
+
Train step of epoch 0:  45%|████▍     | 2511/5616 [31:59<39:07,  1.32it/s, gpt_loss=0.0292, lr=3.25e-6]
+
Train step of epoch 0:  45%|████▍     | 2511/5616 [32:00<39:07,  1.32it/s, gpt_loss=0.0288, lr=3.24e-6]
+
Train step of epoch 0:  45%|████▍     | 2512/5616 [32:00<39:46,  1.30it/s, gpt_loss=0.0288, lr=3.24e-6]
+
Train step of epoch 0:  45%|████▍     | 2512/5616 [32:01<39:46,  1.30it/s, gpt_loss=0.134, lr=3.24e-6] 
+
Train step of epoch 0:  45%|████▍     | 2513/5616 [32:01<38:36,  1.34it/s, gpt_loss=0.134, lr=3.24e-6]
+
Train step of epoch 0:  45%|████▍     | 2513/5616 [32:02<38:36,  1.34it/s, gpt_loss=0.0149, lr=3.24e-6]
+
Train step of epoch 0:  45%|████▍     | 2514/5616 [32:02<37:40,  1.37it/s, gpt_loss=0.0149, lr=3.24e-6]
+
Train step of epoch 0:  45%|████▍     | 2514/5616 [32:02<37:40,  1.37it/s, gpt_loss=0.00547, lr=3.24e-6]
+
Train step of epoch 0:  45%|████▍     | 2515/5616 [32:02<36:14,  1.43it/s, gpt_loss=0.00547, lr=3.24e-6]
+
Train step of epoch 0:  45%|████▍     | 2515/5616 [32:03<36:14,  1.43it/s, gpt_loss=0.0281, lr=3.24e-6] 
+
Train step of epoch 0:  45%|████▍     | 2516/5616 [32:03<37:11,  1.39it/s, gpt_loss=0.0281, lr=3.24e-6]
+
Train step of epoch 0:  45%|████▍     | 2516/5616 [32:04<37:11,  1.39it/s, gpt_loss=0.059, lr=3.24e-6] 
+
Train step of epoch 0:  45%|████▍     | 2517/5616 [32:04<37:46,  1.37it/s, gpt_loss=0.059, lr=3.24e-6]
+
Train step of epoch 0:  45%|████▍     | 2517/5616 [32:05<37:46,  1.37it/s, gpt_loss=0.0272, lr=3.24e-6]
+
Train step of epoch 0:  45%|████▍     | 2518/5616 [32:05<39:17,  1.31it/s, gpt_loss=0.0272, lr=3.24e-6]
+
Train step of epoch 0:  45%|████▍     | 2518/5616 [32:05<39:17,  1.31it/s, gpt_loss=0.0725, lr=3.24e-6]
+
Train step of epoch 0:  45%|████▍     | 2519/5616 [32:05<39:39,  1.30it/s, gpt_loss=0.0725, lr=3.24e-6]
+
Train step of epoch 0:  45%|████▍     | 2519/5616 [32:06<39:39,  1.30it/s, gpt_loss=0.0804, lr=3.23e-6]
+
Train step of epoch 0:  45%|████▍     | 2520/5616 [32:06<39:33,  1.30it/s, gpt_loss=0.0804, lr=3.23e-6]
+
Train step of epoch 0:  45%|████▍     | 2520/5616 [32:07<39:33,  1.30it/s, gpt_loss=0.012, lr=3.23e-6] 
+
Train step of epoch 0:  45%|████▍     | 2521/5616 [32:07<39:19,  1.31it/s, gpt_loss=0.012, lr=3.23e-6]
+
Train step of epoch 0:  45%|████▍     | 2521/5616 [32:08<39:19,  1.31it/s, gpt_loss=0.00334, lr=3.23e-6]
+
Train step of epoch 0:  45%|████▍     | 2522/5616 [32:08<38:40,  1.33it/s, gpt_loss=0.00334, lr=3.23e-6]
+
Train step of epoch 0:  45%|████▍     | 2522/5616 [32:08<38:40,  1.33it/s, gpt_loss=0.017, lr=3.23e-6]  
+
Train step of epoch 0:  45%|████▍     | 2523/5616 [32:08<40:02,  1.29it/s, gpt_loss=0.017, lr=3.23e-6]
+
Train step of epoch 0:  45%|████▍     | 2523/5616 [32:09<40:02,  1.29it/s, gpt_loss=0.0703, lr=3.23e-6]
+
Train step of epoch 0:  45%|████▍     | 2524/5616 [32:09<40:15,  1.28it/s, gpt_loss=0.0703, lr=3.23e-6]
+
Train step of epoch 0:  45%|████▍     | 2524/5616 [32:10<40:15,  1.28it/s, gpt_loss=0.0679, lr=3.23e-6]
+
Train step of epoch 0:  45%|████▍     | 2525/5616 [32:10<39:46,  1.30it/s, gpt_loss=0.0679, lr=3.23e-6]
+
Train step of epoch 0:  45%|���███▍     | 2525/5616 [32:11<39:46,  1.30it/s, gpt_loss=0.0413, lr=3.23e-6]
+
Train step of epoch 0:  45%|████▍     | 2526/5616 [32:11<39:54,  1.29it/s, gpt_loss=0.0413, lr=3.23e-6]
+
Train step of epoch 0:  45%|████▍     | 2526/5616 [32:11<39:54,  1.29it/s, gpt_loss=0.102, lr=3.23e-6] 
+
Train step of epoch 0:  45%|████▍     | 2527/5616 [32:11<39:21,  1.31it/s, gpt_loss=0.102, lr=3.23e-6]
+
Train step of epoch 0:  45%|████▍     | 2527/5616 [32:12<39:21,  1.31it/s, gpt_loss=0.00859, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2528/5616 [32:12<39:17,  1.31it/s, gpt_loss=0.00859, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2528/5616 [32:13<39:17,  1.31it/s, gpt_loss=0.069, lr=3.22e-6]  
+
Train step of epoch 0:  45%|████▌     | 2529/5616 [32:13<39:33,  1.30it/s, gpt_loss=0.069, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2529/5616 [32:14<39:33,  1.30it/s, gpt_loss=0.0571, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2530/5616 [32:14<37:22,  1.38it/s, gpt_loss=0.0571, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2530/5616 [32:14<37:22,  1.38it/s, gpt_loss=0.0244, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2531/5616 [32:14<35:52,  1.43it/s, gpt_loss=0.0244, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2531/5616 [32:15<35:52,  1.43it/s, gpt_loss=0.0164, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2532/5616 [32:15<37:27,  1.37it/s, gpt_loss=0.0164, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2532/5616 [32:16<37:27,  1.37it/s, gpt_loss=0.0323, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2533/5616 [32:16<37:12,  1.38it/s, gpt_loss=0.0323, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2533/5616 [32:17<37:12,  1.38it/s, gpt_loss=0.0205, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2534/5616 [32:17<37:27,  1.37it/s, gpt_loss=0.0205, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2534/5616 [32:17<37:27,  1.37it/s, gpt_loss=0.131, lr=3.22e-6] 
+
Train step of epoch 0:  45%|████▌     | 2535/5616 [32:17<37:08,  1.38it/s, gpt_loss=0.131, lr=3.22e-6]
+
Train step of epoch 0:  45%|████▌     | 2535/5616 [32:18<37:08,  1.38it/s, gpt_loss=0.0109, lr=3.21e-6]
+
Train step of epoch 0:  45%|████▌     | 2536/5616 [32:18<38:30,  1.33it/s, gpt_loss=0.0109, lr=3.21e-6]
+
Train step of epoch 0:  45%|████▌     | 2536/5616 [32:19<38:30,  1.33it/s, gpt_loss=0.00799, lr=3.21e-6]
+
Train step of epoch 0:  45%|████▌     | 2537/5616 [32:19<41:12,  1.25it/s, gpt_loss=0.00799, lr=3.21e-6]
+
Train step of epoch 0:  45%|████▌     | 2537/5616 [32:20<41:12,  1.25it/s, gpt_loss=0.0178, lr=3.21e-6] 
+
Train step of epoch 0:  45%|████▌     | 2538/5616 [32:20<39:21,  1.30it/s, gpt_loss=0.0178, lr=3.21e-6]
+
Train step of epoch 0:  45%|████▌     | 2538/5616 [32:20<39:21,  1.30it/s, gpt_loss=0.00761, lr=3.21e-6]
+
Train step of epoch 0:  45%|████▌     | 2539/5616 [32:20<38:58,  1.32it/s, gpt_loss=0.00761, lr=3.21e-6]
+
Train step of epoch 0:  45%|████▌     | 2539/5616 [32:21<38:58,  1.32it/s, gpt_loss=0.103, lr=3.21e-6]  
+
Train step of epoch 0:  45%|████▌     | 2540/5616 [32:21<39:52,  1.29it/s, gpt_loss=0.103, lr=3.21e-6]
+
Train step of epoch 0:  45%|████▌     | 2540/5616 [32:22<39:52,  1.29it/s, gpt_loss=0.0181, lr=3.21e-6]
+
Train step of epoch 0:  45%|████▌     | 2541/5616 [32:22<39:05,  1.31it/s, gpt_loss=0.0181, lr=3.21e-6]
+
Train step of epoch 0:  45%|████▌     | 2541/5616 [32:23<39:05,  1.31it/s, gpt_loss=0.106, lr=3.21e-6] 
+
Train step of epoch 0:  45%|████▌     | 2542/5616 [32:23<37:53,  1.35it/s, gpt_loss=0.106, lr=3.21e-6]
+
Train step of epoch 0:  45%|████▌     | 2542/5616 [32:23<37:53,  1.35it/s, gpt_loss=0.0234, lr=3.21e-6]
+
Train step of epoch 0:  45%|████▌     | 2543/5616 [32:23<36:50,  1.39it/s, gpt_loss=0.0234, lr=3.21e-6]
+
Train step of epoch 0:  45%|████▌     | 2543/5616 [32:24<36:50,  1.39it/s, gpt_loss=0.0558, lr=3.2e-6] 
+
Train step of epoch 0:  45%|████▌     | 2544/5616 [32:24<37:19,  1.37it/s, gpt_loss=0.0558, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2544/5616 [32:25<37:19,  1.37it/s, gpt_loss=0.0081, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2545/5616 [32:25<39:32,  1.29it/s, gpt_loss=0.0081, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2545/5616 [32:26<39:32,  1.29it/s, gpt_loss=0.0324, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2546/5616 [32:26<39:54,  1.28it/s, gpt_loss=0.0324, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2546/5616 [32:26<39:54,  1.28it/s, gpt_loss=0.0113, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2547/5616 [32:26<38:28,  1.33it/s, gpt_loss=0.0113, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2547/5616 [32:27<38:28,  1.33it/s, gpt_loss=0.025, lr=3.2e-6] 
+
Train step of epoch 0:  45%|████▌     | 2548/5616 [32:27<41:08,  1.24it/s, gpt_loss=0.025, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2548/5616 [32:28<41:08,  1.24it/s, gpt_loss=0.0216, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2549/5616 [32:28<40:40,  1.26it/s, gpt_loss=0.0216, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2549/5616 [32:29<40:40,  1.26it/s, gpt_loss=0.0144, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2550/5616 [32:29<40:42,  1.26it/s, gpt_loss=0.0144, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2550/5616 [32:30<40:42,  1.26it/s, gpt_loss=0.0139, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2551/5616 [32:30<40:21,  1.27it/s, gpt_loss=0.0139, lr=3.2e-6]
+
Train step of epoch 0:  45%|████▌     | 2551/5616 [32:30<40:21,  1.27it/s, gpt_loss=0.105, lr=3.19e-6]
+
Train step of epoch 0:  45%|████▌     | 2552/5616 [32:30<39:27,  1.29it/s, gpt_loss=0.105, lr=3.19e-6]
+
Train step of epoch 0:  45%|████▌     | 2552/5616 [32:31<39:27,  1.29it/s, gpt_loss=0.0171, lr=3.19e-6]
+
Train step of epoch 0:  45%|████▌     | 2553/5616 [32:31<38:54,  1.31it/s, gpt_loss=0.0171, lr=3.19e-6]
+
Train step of epoch 0:  45%|████▌     | 2553/5616 [32:32<38:54,  1.31it/s, gpt_loss=0.012, lr=3.19e-6] 
+
Train step of epoch 0:  45%|████▌     | 2554/5616 [32:32<38:33,  1.32it/s, gpt_loss=0.012, lr=3.19e-6]
+
Train step of epoch 0:  45%|████▌     | 2554/5616 [32:33<38:33,  1.32it/s, gpt_loss=0.0127, lr=3.19e-6]
+
Train step of epoch 0:  45%|████▌     | 2555/5616 [32:33<37:23,  1.36it/s, gpt_loss=0.0127, lr=3.19e-6]
+
Train step of epoch 0:  45%|████▌     | 2555/5616 [32:33<37:23,  1.36it/s, gpt_loss=0.0365, lr=3.19e-6]
+
Train step of epoch 0:  46%|████▌     | 2556/5616 [32:33<40:01,  1.27it/s, gpt_loss=0.0365, lr=3.19e-6]
+
Train step of epoch 0:  46%|████▌     | 2556/5616 [32:34<40:01,  1.27it/s, gpt_loss=0.0132, lr=3.19e-6]
+
Train step of epoch 0:  46%|████▌     | 2557/5616 [32:34<40:18,  1.27it/s, gpt_loss=0.0132, lr=3.19e-6]
+
Train step of epoch 0:  46%|████▌     | 2557/5616 [32:35<40:18,  1.27it/s, gpt_loss=0.0302, lr=3.19e-6]
+
Train step of epoch 0:  46%|████▌     | 2558/5616 [32:35<42:33,  1.20it/s, gpt_loss=0.0302, lr=3.19e-6]
+
Train step of epoch 0:  46%|████▌     | 2558/5616 [32:36<42:33,  1.20it/s, gpt_loss=0.0061, lr=3.19e-6]
+
Train step of epoch 0:  46%|████▌     | 2559/5616 [32:36<40:44,  1.25it/s, gpt_loss=0.0061, lr=3.19e-6]
+
Train step of epoch 0:  46%|████▌     | 2559/5616 [32:37<40:44,  1.25it/s, gpt_loss=0.0102, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2560/5616 [32:37<40:35,  1.25it/s, gpt_loss=0.0102, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2560/5616 [32:37<40:35,  1.25it/s, gpt_loss=0.0682, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2561/5616 [32:37<39:08,  1.30it/s, gpt_loss=0.0682, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2561/5616 [32:38<39:08,  1.30it/s, gpt_loss=0.0171, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2562/5616 [32:38<38:29,  1.32it/s, gpt_loss=0.0171, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2562/5616 [32:39<38:29,  1.32it/s, gpt_loss=0.131, lr=3.18e-6] 
+
Train step of epoch 0:  46%|████▌     | 2563/5616 [32:39<37:40,  1.35it/s, gpt_loss=0.131, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2563/5616 [32:40<37:40,  1.35it/s, gpt_loss=0.0218, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2564/5616 [32:40<37:47,  1.35it/s, gpt_loss=0.0218, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2564/5616 [32:40<37:47,  1.35it/s, gpt_loss=0.0977, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2565/5616 [32:40<36:32,  1.39it/s, gpt_loss=0.0977, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2565/5616 [32:41<36:32,  1.39it/s, gpt_loss=0.0167, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2566/5616 [32:41<39:43,  1.28it/s, gpt_loss=0.0167, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2566/5616 [32:42<39:43,  1.28it/s, gpt_loss=0.00104, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2567/5616 [32:42<38:12,  1.33it/s, gpt_loss=0.00104, lr=3.18e-6]
+
Train step of epoch 0:  46%|████▌     | 2567/5616 [32:43<38:12,  1.33it/s, gpt_loss=0.0314, lr=3.17e-6] 
+
Train step of epoch 0:  46%|████▌     | 2568/5616 [32:43<38:30,  1.32it/s, gpt_loss=0.0314, lr=3.17e-6]
+
Train step of epoch 0:  46%|████▌     | 2568/5616 [32:43<38:30,  1.32it/s, gpt_loss=0.0488, lr=3.17e-6]
+
Train step of epoch 0:  46%|████▌     | 2569/5616 [32:43<36:51,  1.38it/s, gpt_loss=0.0488, lr=3.17e-6]
+
Train step of epoch 0:  46%|████▌     | 2569/5616 [32:44<36:51,  1.38it/s, gpt_loss=0.00523, lr=3.17e-6]
+
Train step of epoch 0:  46%|████▌     | 2570/5616 [32:44<37:15,  1.36it/s, gpt_loss=0.00523, lr=3.17e-6]
+
Train step of epoch 0:  46%|████▌     | 2570/5616 [32:45<37:15,  1.36it/s, gpt_loss=0.0472, lr=3.17e-6] 
+
Train step of epoch 0:  46%|████▌     | 2571/5616 [32:45<35:44,  1.42it/s, gpt_loss=0.0472, lr=3.17e-6]
+
Train step of epoch 0:  46%|████▌     | 2571/5616 [32:46<35:44,  1.42it/s, gpt_loss=0.109, lr=3.17e-6] 
+
Train step of epoch 0:  46%|████▌     | 2572/5616 [32:46<39:53,  1.27it/s, gpt_loss=0.109, lr=3.17e-6]
+
Train step of epoch 0:  46%|████▌     | 2572/5616 [32:46<39:53,  1.27it/s, gpt_loss=0.00183, lr=3.17e-6]
+
Train step of epoch 0:  46%|████▌     | 2573/5616 [32:46<38:25,  1.32it/s, gpt_loss=0.00183, lr=3.17e-6]
+
Train step of epoch 0:  46%|████▌     | 2573/5616 [32:47<38:25,  1.32it/s, gpt_loss=0.155, lr=3.17e-6]  
+
Train step of epoch 0:  46%|████▌     | 2574/5616 [32:47<36:31,  1.39it/s, gpt_loss=0.155, lr=3.17e-6]
+
Train step of epoch 0:  46%|████▌     | 2574/5616 [32:48<36:31,  1.39it/s, gpt_loss=0.165, lr=3.17e-6]
+
Train step of epoch 0:  46%|████▌     | 2575/5616 [32:48<43:49,  1.16it/s, gpt_loss=0.165, lr=3.17e-6]
+
Train step of epoch 0:  46%|████▌     | 2575/5616 [32:49<43:49,  1.16it/s, gpt_loss=0.0527, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2576/5616 [32:49<43:43,  1.16it/s, gpt_loss=0.0527, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2576/5616 [32:50<43:43,  1.16it/s, gpt_loss=0.0193, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2577/5616 [32:50<40:54,  1.24it/s, gpt_loss=0.0193, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2577/5616 [32:50<40:54,  1.24it/s, gpt_loss=0.0433, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2578/5616 [32:50<39:04,  1.30it/s, gpt_loss=0.0433, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2578/5616 [32:51<39:04,  1.30it/s, gpt_loss=0.0302, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2579/5616 [32:51<37:28,  1.35it/s, gpt_loss=0.0302, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2579/5616 [32:52<37:28,  1.35it/s, gpt_loss=0.0182, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2580/5616 [32:52<44:00,  1.15it/s, gpt_loss=0.0182, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2580/5616 [32:53<44:00,  1.15it/s, gpt_loss=0.0804, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2581/5616 [32:53<41:43,  1.21it/s, gpt_loss=0.0804, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2581/5616 [32:54<41:43,  1.21it/s, gpt_loss=0.0286, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2582/5616 [32:54<39:22,  1.28it/s, gpt_loss=0.0286, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2582/5616 [32:54<39:22,  1.28it/s, gpt_loss=0.0301, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2583/5616 [32:54<37:18,  1.35it/s, gpt_loss=0.0301, lr=3.16e-6]
+
Train step of epoch 0:  46%|████▌     | 2583/5616 [32:55<37:18,  1.35it/s, gpt_loss=0.0796, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2584/5616 [32:55<37:24,  1.35it/s, gpt_loss=0.0796, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2584/5616 [32:56<37:24,  1.35it/s, gpt_loss=0.00988, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2585/5616 [32:56<37:26,  1.35it/s, gpt_loss=0.00988, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2585/5616 [32:56<37:26,  1.35it/s, gpt_loss=0.00172, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2586/5616 [32:56<36:20,  1.39it/s, gpt_loss=0.00172, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2586/5616 [32:57<36:20,  1.39it/s, gpt_loss=0.0132, lr=3.15e-6] 
+
Train step of epoch 0:  46%|████▌     | 2587/5616 [32:57<36:29,  1.38it/s, gpt_loss=0.0132, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2587/5616 [32:58<36:29,  1.38it/s, gpt_loss=0.0271, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2588/5616 [32:58<36:52,  1.37it/s, gpt_loss=0.0271, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2588/5616 [32:59<36:52,  1.37it/s, gpt_loss=0.0037, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2589/5616 [32:59<38:14,  1.32it/s, gpt_loss=0.0037, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2589/5616 [33:00<38:14,  1.32it/s, gpt_loss=0.00636, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2590/5616 [33:00<38:29,  1.31it/s, gpt_loss=0.00636, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2590/5616 [33:00<38:29,  1.31it/s, gpt_loss=0.0192, lr=3.15e-6] 
+
Train step of epoch 0:  46%|████▌     | 2591/5616 [33:00<37:56,  1.33it/s, gpt_loss=0.0192, lr=3.15e-6]
+
Train step of epoch 0:  46%|████▌     | 2591/5616 [33:01<37:56,  1.33it/s, gpt_loss=0.00404, lr=3.14e-6]
+
Train step of epoch 0:  46%|████▌     | 2592/5616 [33:01<38:21,  1.31it/s, gpt_loss=0.00404, lr=3.14e-6]
+
Train step of epoch 0:  46%|████▌     | 2592/5616 [33:02<38:21,  1.31it/s, gpt_loss=0.00664, lr=3.14e-6]
+
Train step of epoch 0:  46%|████▌     | 2593/5616 [33:02<36:59,  1.36it/s, gpt_loss=0.00664, lr=3.14e-6]
+
Train step of epoch 0:  46%|████▌     | 2593/5616 [33:02<36:59,  1.36it/s, gpt_loss=0.0488, lr=3.14e-6] 
+
Train step of epoch 0:  46%|████▌     | 2594/5616 [33:02<37:30,  1.34it/s, gpt_loss=0.0488, lr=3.14e-6]
+
Train step of epoch 0:  46%|████▌     | 2594/5616 [33:03<37:30,  1.34it/s, gpt_loss=0.0114, lr=3.14e-6]
+
Train step of epoch 0:  46%|████▌     | 2595/5616 [33:03<36:43,  1.37it/s, gpt_loss=0.0114, lr=3.14e-6]
+
Train step of epoch 0:  46%|████▌     | 2595/5616 [33:04<36:43,  1.37it/s, gpt_loss=0.00385, lr=3.14e-6]
+
Train step of epoch 0:  46%|████▌     | 2596/5616 [33:04<38:17,  1.31it/s, gpt_loss=0.00385, lr=3.14e-6]
+
Train step of epoch 0:  46%|████▌     | 2596/5616 [33:05<38:17,  1.31it/s, gpt_loss=0.0471, lr=3.14e-6] 
+
Train step of epoch 0:  46%|████▌     | 2597/5616 [33:05<38:12,  1.32it/s, gpt_loss=0.0471, lr=3.14e-6]
+
Train step of epoch 0:  46%|████▌     | 2597/5616 [33:05<38:12,  1.32it/s, gpt_loss=0.152, lr=3.14e-6] 
+
Train step of epoch 0:  46%|████▋     | 2598/5616 [33:05<36:30,  1.38it/s, gpt_loss=0.152, lr=3.14e-6]
+
Train step of epoch 0:  46%|████▋     | 2598/5616 [33:06<36:30,  1.38it/s, gpt_loss=0.00242, lr=3.14e-6]
+
Train step of epoch 0:  46%|████▋     | 2599/5616 [33:06<35:38,  1.41it/s, gpt_loss=0.00242, lr=3.14e-6]
+
Train step of epoch 0:  46%|████▋     | 2599/5616 [33:07<35:38,  1.41it/s, gpt_loss=0.0242, lr=3.13e-6] 
+
Train step of epoch 0:  46%|████▋     | 2600/5616 [33:07<36:33,  1.37it/s, gpt_loss=0.0242, lr=3.13e-6]
+
Train step of epoch 0:  46%|████▋     | 2600/5616 [33:08<36:33,  1.37it/s, gpt_loss=0.162, lr=3.13e-6] 
+
Train step of epoch 0:  46%|████▋     | 2601/5616 [33:08<36:33,  1.37it/s, gpt_loss=0.162, lr=3.13e-6]
+
Train step of epoch 0:  46%|████▋     | 2601/5616 [33:08<36:33,  1.37it/s, gpt_loss=0.0121, lr=3.13e-6]
+
Train step of epoch 0:  46%|████▋     | 2602/5616 [33:08<36:11,  1.39it/s, gpt_loss=0.0121, lr=3.13e-6]
+
Train step of epoch 0:  46%|████▋     | 2602/5616 [33:09<36:11,  1.39it/s, gpt_loss=0.0294, lr=3.13e-6]
+
Train step of epoch 0:  46%|████▋     | 2603/5616 [33:09<35:35,  1.41it/s, gpt_loss=0.0294, lr=3.13e-6]
+
Train step of epoch 0:  46%|████▋     | 2603/5616 [33:10<35:35,  1.41it/s, gpt_loss=0.0245, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2604/5616 [33:10<36:41,  1.37it/s, gpt_loss=0.0245, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2604/5616 [33:11<36:41,  1.37it/s, gpt_loss=0.0756, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2605/5616 [33:11<38:26,  1.31it/s, gpt_loss=0.0756, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2605/5616 [33:11<38:26,  1.31it/s, gpt_loss=0.0128, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2606/5616 [33:11<37:08,  1.35it/s, gpt_loss=0.0128, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2606/5616 [33:12<37:08,  1.35it/s, gpt_loss=0.109, lr=3.12e-6] 
+
Train step of epoch 0:  46%|████▋     | 2607/5616 [33:12<37:50,  1.33it/s, gpt_loss=0.109, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2607/5616 [33:13<37:50,  1.33it/s, gpt_loss=0.0492, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2608/5616 [33:13<39:13,  1.28it/s, gpt_loss=0.0492, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2608/5616 [33:14<39:13,  1.28it/s, gpt_loss=0.0979, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2609/5616 [33:14<37:34,  1.33it/s, gpt_loss=0.0979, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2609/5616 [33:14<37:34,  1.33it/s, gpt_loss=0.018, lr=3.12e-6] 
+
Train step of epoch 0:  46%|████▋     | 2610/5616 [33:14<36:36,  1.37it/s, gpt_loss=0.018, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2610/5616 [33:15<36:36,  1.37it/s, gpt_loss=0.0108, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2611/5616 [33:15<35:39,  1.40it/s, gpt_loss=0.0108, lr=3.12e-6]
+
Train step of epoch 0:  46%|████▋     | 2611/5616 [33:16<35:39,  1.40it/s, gpt_loss=0.0294, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2612/5616 [33:16<37:05,  1.35it/s, gpt_loss=0.0294, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2612/5616 [33:16<37:05,  1.35it/s, gpt_loss=0.0379, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2613/5616 [33:16<35:29,  1.41it/s, gpt_loss=0.0379, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2613/5616 [33:17<35:29,  1.41it/s, gpt_loss=0.00605, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2614/5616 [33:17<39:49,  1.26it/s, gpt_loss=0.00605, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2614/5616 [33:18<39:49,  1.26it/s, gpt_loss=0.0642, lr=3.11e-6] 
+
Train step of epoch 0:  47%|████▋     | 2615/5616 [33:18<37:43,  1.33it/s, gpt_loss=0.0642, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2615/5616 [33:19<37:43,  1.33it/s, gpt_loss=0.00148, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2616/5616 [33:19<39:57,  1.25it/s, gpt_loss=0.00148, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2616/5616 [33:20<39:57,  1.25it/s, gpt_loss=0.0138, lr=3.11e-6] 
+
Train step of epoch 0:  47%|████▋     | 2617/5616 [33:20<38:57,  1.28it/s, gpt_loss=0.0138, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2617/5616 [33:20<38:57,  1.28it/s, gpt_loss=0.0187, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2618/5616 [33:20<39:13,  1.27it/s, gpt_loss=0.0187, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2618/5616 [33:21<39:13,  1.27it/s, gpt_loss=0.0129, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2619/5616 [33:21<38:18,  1.30it/s, gpt_loss=0.0129, lr=3.11e-6]
+
Train step of epoch 0:  47%|████▋     | 2619/5616 [33:22<38:18,  1.30it/s, gpt_loss=0.00699, lr=3.1e-6]
+
Train step of epoch 0:  47%|████▋     | 2620/5616 [33:22<38:36,  1.29it/s, gpt_loss=0.00699, lr=3.1e-6]
+
Train step of epoch 0:  47%|████▋     | 2620/5616 [33:23<38:36,  1.29it/s, gpt_loss=0.0258, lr=3.1e-6] 
+
Train step of epoch 0:  47%|████▋     | 2621/5616 [33:23<43:41,  1.14it/s, gpt_loss=0.0258, lr=3.1e-6]
+
Train step of epoch 0:  47%|████▋     | 2621/5616 [33:24<43:41,  1.14it/s, gpt_loss=0.00239, lr=3.1e-6]
+
Train step of epoch 0:  47%|████▋     | 2622/5616 [33:24<40:10,  1.24it/s, gpt_loss=0.00239, lr=3.1e-6]
+
Train step of epoch 0:  47%|████▋     | 2622/5616 [33:24<40:10,  1.24it/s, gpt_loss=0.0932, lr=3.1e-6] 
+
Train step of epoch 0:  47%|████▋     | 2623/5616 [33:24<38:36,  1.29it/s, gpt_loss=0.0932, lr=3.1e-6]
+
Train step of epoch 0:  47%|████▋     | 2623/5616 [33:25<38:36,  1.29it/s, gpt_loss=0.0364, lr=3.1e-6]
+
Train step of epoch 0:  47%|████▋     | 2624/5616 [33:25<39:46,  1.25it/s, gpt_loss=0.0364, lr=3.1e-6]
+
Train step of epoch 0:  47%|████▋     | 2624/5616 [33:26<39:46,  1.25it/s, gpt_loss=0.00961, lr=3.1e-6]
+
Train step of epoch 0:  47%|████▋     | 2625/5616 [33:26<38:49,  1.28it/s, gpt_loss=0.00961, lr=3.1e-6]
+
Train step of epoch 0:  47%|████▋     | 2625/5616 [33:27<38:49,  1.28it/s, gpt_loss=0.00308, lr=3.1e-6]
+
Train step of epoch 0:  47%|████▋     | 2626/5616 [33:27<37:20,  1.33it/s, gpt_loss=0.00308, lr=3.1e-6]
+
Train step of epoch 0:  47%|████▋     | 2626/5616 [33:27<37:20,  1.33it/s, gpt_loss=0.0175, lr=3.1e-6] 
+
Train step of epoch 0:  47%|████▋     | 2627/5616 [33:27<36:20,  1.37it/s, gpt_loss=0.0175, lr=3.1e-6]
+
Train step of epoch 0:  47%|████▋     | 2627/5616 [33:28<36:20,  1.37it/s, gpt_loss=0.044, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2628/5616 [33:28<38:40,  1.29it/s, gpt_loss=0.044, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2628/5616 [33:29<38:40,  1.29it/s, gpt_loss=0.00568, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2629/5616 [33:29<37:23,  1.33it/s, gpt_loss=0.00568, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2629/5616 [33:30<37:23,  1.33it/s, gpt_loss=0.00121, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2630/5616 [33:30<36:39,  1.36it/s, gpt_loss=0.00121, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2630/5616 [33:30<36:39,  1.36it/s, gpt_loss=0.0737, lr=3.09e-6] 
+
Train step of epoch 0:  47%|████▋     | 2631/5616 [33:30<37:57,  1.31it/s, gpt_loss=0.0737, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2631/5616 [33:31<37:57,  1.31it/s, gpt_loss=0.0177, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2632/5616 [33:31<38:43,  1.28it/s, gpt_loss=0.0177, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2632/5616 [33:32<38:43,  1.28it/s, gpt_loss=0.0196, lr=3.09e-6]
+
Train step of epoch 0:  47%|██��█▋     | 2633/5616 [33:32<37:11,  1.34it/s, gpt_loss=0.0196, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2633/5616 [33:33<37:11,  1.34it/s, gpt_loss=0.0211, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2634/5616 [33:33<36:09,  1.37it/s, gpt_loss=0.0211, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2634/5616 [33:33<36:09,  1.37it/s, gpt_loss=0.00445, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2635/5616 [33:33<36:11,  1.37it/s, gpt_loss=0.00445, lr=3.09e-6]
+
Train step of epoch 0:  47%|████▋     | 2635/5616 [33:34<36:11,  1.37it/s, gpt_loss=0.0696, lr=3.08e-6] 
+
Train step of epoch 0:  47%|████▋     | 2636/5616 [33:34<36:09,  1.37it/s, gpt_loss=0.0696, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2636/5616 [33:35<36:09,  1.37it/s, gpt_loss=0.0904, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2637/5616 [33:35<36:34,  1.36it/s, gpt_loss=0.0904, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2637/5616 [33:36<36:34,  1.36it/s, gpt_loss=0.0254, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2638/5616 [33:36<37:03,  1.34it/s, gpt_loss=0.0254, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2638/5616 [33:36<37:03,  1.34it/s, gpt_loss=0.0022, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2639/5616 [33:36<36:20,  1.37it/s, gpt_loss=0.0022, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2639/5616 [33:37<36:20,  1.37it/s, gpt_loss=0.0932, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2640/5616 [33:37<37:04,  1.34it/s, gpt_loss=0.0932, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2640/5616 [33:38<37:04,  1.34it/s, gpt_loss=0.00578, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2641/5616 [33:38<36:28,  1.36it/s, gpt_loss=0.00578, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2641/5616 [33:39<36:28,  1.36it/s, gpt_loss=0.0633, lr=3.08e-6] 
+
Train step of epoch 0:  47%|████▋     | 2642/5616 [33:39<39:14,  1.26it/s, gpt_loss=0.0633, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2642/5616 [33:39<39:14,  1.26it/s, gpt_loss=0.00467, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2643/5616 [33:39<38:03,  1.30it/s, gpt_loss=0.00467, lr=3.08e-6]
+
Train step of epoch 0:  47%|████▋     | 2643/5616 [33:40<38:03,  1.30it/s, gpt_loss=0.0633, lr=3.07e-6] 
+
Train step of epoch 0:  47%|████▋     | 2644/5616 [33:40<38:31,  1.29it/s, gpt_loss=0.0633, lr=3.07e-6]
+
Train step of epoch 0:  47%|████▋     | 2644/5616 [33:41<38:31,  1.29it/s, gpt_loss=0.012, lr=3.07e-6] 
+
Train step of epoch 0:  47%|████▋     | 2645/5616 [33:41<37:19,  1.33it/s, gpt_loss=0.012, lr=3.07e-6]
+
Train step of epoch 0:  47%|████▋     | 2645/5616 [33:42<37:19,  1.33it/s, gpt_loss=0.0377, lr=3.07e-6]
+
Train step of epoch 0:  47%|████▋     | 2646/5616 [33:42<37:11,  1.33it/s, gpt_loss=0.0377, lr=3.07e-6]
+
Train step of epoch 0:  47%|████▋     | 2646/5616 [33:42<37:11,  1.33it/s, gpt_loss=0.00714, lr=3.07e-6]
+
Train step of epoch 0:  47%|████▋     | 2647/5616 [33:42<37:03,  1.34it/s, gpt_loss=0.00714, lr=3.07e-6]
+
Train step of epoch 0:  47%|████▋     | 2647/5616 [33:43<37:03,  1.34it/s, gpt_loss=0.0238, lr=3.07e-6] 
+
Train step of epoch 0:  47%|████▋     | 2648/5616 [33:43<37:21,  1.32it/s, gpt_loss=0.0238, lr=3.07e-6]
+
Train step of epoch 0:  47%|████▋     | 2648/5616 [33:44<37:21,  1.32it/s, gpt_loss=0.00331, lr=3.07e-6]
+
Train step of epoch 0:  47%|████▋     | 2649/5616 [33:44<37:17,  1.33it/s, gpt_loss=0.00331, lr=3.07e-6]
+
Train step of epoch 0:  47%|████▋     | 2649/5616 [33:45<37:17,  1.33it/s, gpt_loss=0.00408, lr=3.07e-6]
+
Train step of epoch 0:  47%|████▋     | 2650/5616 [33:45<38:15,  1.29it/s, gpt_loss=0.00408, lr=3.07e-6]
+
Train step of epoch 0:  47%|████▋     | 2650/5616 [33:46<38:15,  1.29it/s, gpt_loss=0.0481, lr=3.07e-6] 
+
Train step of epoch 0:  47%|████▋     | 2651/5616 [33:46<38:49,  1.27it/s, gpt_loss=0.0481, lr=3.07e-6]
+
Train step of epoch 0:  47%|████▋     | 2651/5616 [33:46<38:49,  1.27it/s, gpt_loss=0.0321, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2652/5616 [33:46<38:59,  1.27it/s, gpt_loss=0.0321, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2652/5616 [33:47<38:59,  1.27it/s, gpt_loss=0.0469, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2653/5616 [33:47<37:42,  1.31it/s, gpt_loss=0.0469, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2653/5616 [33:48<37:42,  1.31it/s, gpt_loss=0.00516, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2654/5616 [33:48<37:17,  1.32it/s, gpt_loss=0.00516, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2654/5616 [33:49<37:17,  1.32it/s, gpt_loss=0.0304, lr=3.06e-6] 
+
Train step of epoch 0:  47%|████▋     | 2655/5616 [33:49<35:55,  1.37it/s, gpt_loss=0.0304, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2655/5616 [33:49<35:55,  1.37it/s, gpt_loss=0.0659, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2656/5616 [33:49<37:54,  1.30it/s, gpt_loss=0.0659, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2656/5616 [33:50<37:54,  1.30it/s, gpt_loss=0.00189, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2657/5616 [33:50<38:23,  1.28it/s, gpt_loss=0.00189, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2657/5616 [33:51<38:23,  1.28it/s, gpt_loss=0.00765, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2658/5616 [33:51<37:07,  1.33it/s, gpt_loss=0.00765, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2658/5616 [33:52<37:07,  1.33it/s, gpt_loss=0.00323, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2659/5616 [33:52<36:06,  1.37it/s, gpt_loss=0.00323, lr=3.06e-6]
+
Train step of epoch 0:  47%|████▋     | 2659/5616 [33:52<36:06,  1.37it/s, gpt_loss=0.0696, lr=3.05e-6] 
+
Train step of epoch 0:  47%|████▋     | 2660/5616 [33:52<39:07,  1.26it/s, gpt_loss=0.0696, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2660/5616 [33:53<39:07,  1.26it/s, gpt_loss=0.0368, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2661/5616 [33:53<37:02,  1.33it/s, gpt_loss=0.0368, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2661/5616 [33:54<37:02,  1.33it/s, gpt_loss=0.0144, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2662/5616 [33:54<35:31,  1.39it/s, gpt_loss=0.0144, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2662/5616 [33:55<35:31,  1.39it/s, gpt_loss=0.0615, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2663/5616 [33:55<36:35,  1.35it/s, gpt_loss=0.0615, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2663/5616 [33:55<36:35,  1.35it/s, gpt_loss=0.0707, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2664/5616 [33:55<36:17,  1.36it/s, gpt_loss=0.0707, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2664/5616 [33:56<36:17,  1.36it/s, gpt_loss=0.0303, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2665/5616 [33:56<37:38,  1.31it/s, gpt_loss=0.0303, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2665/5616 [33:57<37:38,  1.31it/s, gpt_loss=0.035, lr=3.05e-6] 
+
Train step of epoch 0:  47%|████▋     | 2666/5616 [33:57<37:14,  1.32it/s, gpt_loss=0.035, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2666/5616 [33:58<37:14,  1.32it/s, gpt_loss=0.00726, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2667/5616 [33:58<35:31,  1.38it/s, gpt_loss=0.00726, lr=3.05e-6]
+
Train step of epoch 0:  47%|████▋     | 2667/5616 [33:58<35:31,  1.38it/s, gpt_loss=0.0125, lr=3.04e-6] 
+
Train step of epoch 0:  48%|████▊     | 2668/5616 [33:58<37:14,  1.32it/s, gpt_loss=0.0125, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2668/5616 [33:59<37:14,  1.32it/s, gpt_loss=0.0143, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2669/5616 [33:59<35:38,  1.38it/s, gpt_loss=0.0143, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2669/5616 [34:00<35:38,  1.38it/s, gpt_loss=0.0226, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2670/5616 [34:00<35:12,  1.39it/s, gpt_loss=0.0226, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2670/5616 [34:00<35:12,  1.39it/s, gpt_loss=0.0131, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2671/5616 [34:00<36:13,  1.36it/s, gpt_loss=0.0131, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2671/5616 [34:01<36:13,  1.36it/s, gpt_loss=0.0871, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2672/5616 [34:01<36:17,  1.35it/s, gpt_loss=0.0871, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2672/5616 [34:02<36:17,  1.35it/s, gpt_loss=0.122, lr=3.04e-6] 
+
Train step of epoch 0:  48%|████▊     | 2673/5616 [34:02<35:26,  1.38it/s, gpt_loss=0.122, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2673/5616 [34:03<35:26,  1.38it/s, gpt_loss=0.0603, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2674/5616 [34:03<34:18,  1.43it/s, gpt_loss=0.0603, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2674/5616 [34:03<34:18,  1.43it/s, gpt_loss=0.0266, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2675/5616 [34:03<35:10,  1.39it/s, gpt_loss=0.0266, lr=3.04e-6]
+
Train step of epoch 0:  48%|████▊     | 2675/5616 [34:04<35:10,  1.39it/s, gpt_loss=0.0093, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2676/5616 [34:04<37:47,  1.30it/s, gpt_loss=0.0093, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2676/5616 [34:05<37:47,  1.30it/s, gpt_loss=0.015, lr=3.03e-6] 
+
Train step of epoch 0:  48%|████▊     | 2677/5616 [34:05<35:44,  1.37it/s, gpt_loss=0.015, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2677/5616 [34:05<35:44,  1.37it/s, gpt_loss=0.00616, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2678/5616 [34:05<34:20,  1.43it/s, gpt_loss=0.00616, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2678/5616 [34:06<34:20,  1.43it/s, gpt_loss=0.00607, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2679/5616 [34:06<34:02,  1.44it/s, gpt_loss=0.00607, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2679/5616 [34:07<34:02,  1.44it/s, gpt_loss=0.0881, lr=3.03e-6] 
+
Train step of epoch 0:  48%|████▊     | 2680/5616 [34:07<36:00,  1.36it/s, gpt_loss=0.0881, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2680/5616 [34:08<36:00,  1.36it/s, gpt_loss=0.0596, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2681/5616 [34:08<36:12,  1.35it/s, gpt_loss=0.0596, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2681/5616 [34:09<36:12,  1.35it/s, gpt_loss=0.0187, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2682/5616 [34:09<36:17,  1.35it/s, gpt_loss=0.0187, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2682/5616 [34:09<36:17,  1.35it/s, gpt_loss=0.0247, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2683/5616 [34:09<36:19,  1.35it/s, gpt_loss=0.0247, lr=3.03e-6]
+
Train step of epoch 0:  48%|████▊     | 2683/5616 [34:10<36:19,  1.35it/s, gpt_loss=0.00466, lr=3.02e-6]
+
Train step of epoch 0:  48%|████▊     | 2684/5616 [34:10<37:17,  1.31it/s, gpt_loss=0.00466, lr=3.02e-6]
+
Train step of epoch 0:  48%|████▊     | 2684/5616 [34:11<37:17,  1.31it/s, gpt_loss=0.0264, lr=3.02e-6] 
+
Train step of epoch 0:  48%|████▊     | 2685/5616 [34:11<39:05,  1.25it/s, gpt_loss=0.0264, lr=3.02e-6]
+
Train step of epoch 0:  48%|████▊     | 2685/5616 [34:12<39:05,  1.25it/s, gpt_loss=0.00533, lr=3.02e-6]
+
Train step of epoch 0:  48%|████▊     | 2686/5616 [34:12<37:26,  1.30it/s, gpt_loss=0.00533, lr=3.02e-6]
+
Train step of epoch 0:  48%|████▊     | 2686/5616 [34:12<37:26,  1.30it/s, gpt_loss=0.00817, lr=3.02e-6]
+
Train step of epoch 0:  48%|████▊     | 2687/5616 [34:12<36:24,  1.34it/s, gpt_loss=0.00817, lr=3.02e-6]
+
Train step of epoch 0:  48%|████▊     | 2687/5616 [34:14<36:24,  1.34it/s, gpt_loss=0.0215, lr=3.02e-6] 
+
Train step of epoch 0:  48%|████▊     | 2688/5616 [34:14<42:44,  1.14it/s, gpt_loss=0.0215, lr=3.02e-6]
+
Train step of epoch 0:  48%|████▊     | 2688/5616 [34:14<42:44,  1.14it/s, gpt_loss=0.076, lr=3.02e-6] 
+
Train step of epoch 0:  48%|████▊     | 2689/5616 [34:14<41:17,  1.18it/s, gpt_loss=0.076, lr=3.02e-6]
+
Train step of epoch 0:  48%|████▊     | 2689/5616 [34:15<41:17,  1.18it/s, gpt_loss=0.0143, lr=3.02e-6]
+
Train step of epoch 0:  48%|████▊     | 2690/5616 [34:15<39:08,  1.25it/s, gpt_loss=0.0143, lr=3.02e-6]
+
Train step of epoch 0:  48%|████▊     | 2690/5616 [34:16<39:08,  1.25it/s, gpt_loss=0.0072, lr=3.02e-6]
+
Train step of epoch 0:  48%|████▊     | 2691/5616 [34:16<38:53,  1.25it/s, gpt_loss=0.0072, lr=3.02e-6]
+
Train step of epoch 0:  48%|████▊     | 2691/5616 [34:17<38:53,  1.25it/s, gpt_loss=0.0174, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2692/5616 [34:17<38:05,  1.28it/s, gpt_loss=0.0174, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2692/5616 [34:18<38:05,  1.28it/s, gpt_loss=0.0617, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2693/5616 [34:18<42:19,  1.15it/s, gpt_loss=0.0617, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2693/5616 [34:18<42:19,  1.15it/s, gpt_loss=0.000993, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2694/5616 [34:18<39:32,  1.23it/s, gpt_loss=0.000993, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2694/5616 [34:19<39:32,  1.23it/s, gpt_loss=0.0852, lr=3.01e-6]  
+
Train step of epoch 0:  48%|████▊     | 2695/5616 [34:19<37:45,  1.29it/s, gpt_loss=0.0852, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2695/5616 [34:20<37:45,  1.29it/s, gpt_loss=0.0368, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2696/5616 [34:20<37:31,  1.30it/s, gpt_loss=0.0368, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2696/5616 [34:21<37:31,  1.30it/s, gpt_loss=0.105, lr=3.01e-6] 
+
Train step of epoch 0:  48%|████▊     | 2697/5616 [34:21<39:54,  1.22it/s, gpt_loss=0.105, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2697/5616 [34:22<39:54,  1.22it/s, gpt_loss=0.125, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2698/5616 [34:22<40:21,  1.21it/s, gpt_loss=0.125, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2698/5616 [34:22<40:21,  1.21it/s, gpt_loss=0.0413, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2699/5616 [34:22<38:04,  1.28it/s, gpt_loss=0.0413, lr=3.01e-6]
+
Train step of epoch 0:  48%|████▊     | 2699/5616 [34:23<38:04,  1.28it/s, gpt_loss=0.0207, lr=3e-6]   
+
Train step of epoch 0:  48%|████▊     | 2700/5616 [34:23<39:41,  1.22it/s, gpt_loss=0.0207, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2700/5616 [34:24<39:41,  1.22it/s, gpt_loss=0.0357, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2701/5616 [34:24<38:30,  1.26it/s, gpt_loss=0.0357, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2701/5616 [34:24<38:30,  1.26it/s, gpt_loss=0.0738, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2702/5616 [34:24<36:57,  1.31it/s, gpt_loss=0.0738, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2702/5616 [34:25<36:57,  1.31it/s, gpt_loss=0.0398, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2703/5616 [34:25<35:54,  1.35it/s, gpt_loss=0.0398, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2703/5616 [34:26<35:54,  1.35it/s, gpt_loss=0.0202, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2704/5616 [34:26<36:57,  1.31it/s, gpt_loss=0.0202, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2704/5616 [34:27<36:57,  1.31it/s, gpt_loss=0.0213, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2705/5616 [34:27<37:45,  1.28it/s, gpt_loss=0.0213, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2705/5616 [34:28<37:45,  1.28it/s, gpt_loss=0.0313, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2706/5616 [34:28<39:31,  1.23it/s, gpt_loss=0.0313, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2706/5616 [34:28<39:31,  1.23it/s, gpt_loss=0.0553, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2707/5616 [34:28<37:43,  1.29it/s, gpt_loss=0.0553, lr=3e-6]
+
Train step of epoch 0:  48%|████▊     | 2707/5616 [34:29<37:43,  1.29it/s, gpt_loss=0.0423, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2708/5616 [34:29<37:28,  1.29it/s, gpt_loss=0.0423, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2708/5616 [34:30<37:28,  1.29it/s, gpt_loss=0.0729, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2709/5616 [34:30<37:05,  1.31it/s, gpt_loss=0.0729, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2709/5616 [34:31<37:05,  1.31it/s, gpt_loss=0.101, lr=2.99e-6] 
+
Train step of epoch 0:  48%|████▊     | 2710/5616 [34:31<37:37,  1.29it/s, gpt_loss=0.101, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2710/5616 [34:31<37:37,  1.29it/s, gpt_loss=0.0248, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2711/5616 [34:31<37:09,  1.30it/s, gpt_loss=0.0248, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2711/5616 [34:32<37:09,  1.30it/s, gpt_loss=0.047, lr=2.99e-6] 
+
Train step of epoch 0:  48%|████▊     | 2712/5616 [34:32<36:24,  1.33it/s, gpt_loss=0.047, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2712/5616 [34:33<36:24,  1.33it/s, gpt_loss=0.0381, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2713/5616 [34:33<35:35,  1.36it/s, gpt_loss=0.0381, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2713/5616 [34:34<35:35,  1.36it/s, gpt_loss=0.0151, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2714/5616 [34:34<35:07,  1.38it/s, gpt_loss=0.0151, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2714/5616 [34:34<35:07,  1.38it/s, gpt_loss=0.0429, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2715/5616 [34:34<34:01,  1.42it/s, gpt_loss=0.0429, lr=2.99e-6]
+
Train step of epoch 0:  48%|████▊     | 2715/5616 [34:35<34:01,  1.42it/s, gpt_loss=0.0201, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2716/5616 [34:35<35:59,  1.34it/s, gpt_loss=0.0201, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2716/5616 [34:36<35:59,  1.34it/s, gpt_loss=0.0117, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2717/5616 [34:36<36:28,  1.32it/s, gpt_loss=0.0117, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2717/5616 [34:37<36:28,  1.32it/s, gpt_loss=0.0677, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2718/5616 [34:37<36:20,  1.33it/s, gpt_loss=0.0677, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2718/5616 [34:37<36:20,  1.33it/s, gpt_loss=0.0317, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2719/5616 [34:37<36:03,  1.34it/s, gpt_loss=0.0317, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2719/5616 [34:38<36:03,  1.34it/s, gpt_loss=0.0358, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2720/5616 [34:38<35:39,  1.35it/s, gpt_loss=0.0358, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2720/5616 [34:39<35:39,  1.35it/s, gpt_loss=0.0247, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2721/5616 [34:39<34:24,  1.40it/s, gpt_loss=0.0247, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2721/5616 [34:39<34:24,  1.40it/s, gpt_loss=0.0463, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2722/5616 [34:39<33:11,  1.45it/s, gpt_loss=0.0463, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2722/5616 [34:40<33:11,  1.45it/s, gpt_loss=0.0308, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2723/5616 [34:40<33:39,  1.43it/s, gpt_loss=0.0308, lr=2.98e-6]
+
Train step of epoch 0:  48%|████▊     | 2723/5616 [34:41<33:39,  1.43it/s, gpt_loss=0.0711, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2724/5616 [34:41<35:10,  1.37it/s, gpt_loss=0.0711, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2724/5616 [34:42<35:10,  1.37it/s, gpt_loss=0.0838, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2725/5616 [34:42<36:14,  1.33it/s, gpt_loss=0.0838, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2725/5616 [34:42<36:14,  1.33it/s, gpt_loss=0.0312, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2726/5616 [34:42<35:29,  1.36it/s, gpt_loss=0.0312, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2726/5616 [34:43<35:29,  1.36it/s, gpt_loss=0.00987, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2727/5616 [34:43<36:11,  1.33it/s, gpt_loss=0.00987, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2727/5616 [34:44<36:11,  1.33it/s, gpt_loss=0.0415, lr=2.97e-6] 
+
Train step of epoch 0:  49%|████▊     | 2728/5616 [34:44<36:27,  1.32it/s, gpt_loss=0.0415, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2728/5616 [34:45<36:27,  1.32it/s, gpt_loss=0.067, lr=2.97e-6] 
+
Train step of epoch 0:  49%|████▊     | 2729/5616 [34:45<36:07,  1.33it/s, gpt_loss=0.067, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2729/5616 [34:45<36:07,  1.33it/s, gpt_loss=0.0235, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2730/5616 [34:45<34:25,  1.40it/s, gpt_loss=0.0235, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2730/5616 [34:46<34:25,  1.40it/s, gpt_loss=0.0443, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2731/5616 [34:46<35:26,  1.36it/s, gpt_loss=0.0443, lr=2.97e-6]
+
Train step of epoch 0:  49%|████▊     | 2731/5616 [34:47<35:26,  1.36it/s, gpt_loss=0.0927, lr=2.96e-6]
+
Train step of epoch 0:  49%|████▊     | 2732/5616 [34:47<37:49,  1.27it/s, gpt_loss=0.0927, lr=2.96e-6]
+
Train step of epoch 0:  49%|████▊     | 2732/5616 [34:48<37:49,  1.27it/s, gpt_loss=0.0498, lr=2.96e-6]
+
Train step of epoch 0:  49%|████▊     | 2733/5616 [34:48<36:33,  1.31it/s, gpt_loss=0.0498, lr=2.96e-6]
+
Train step of epoch 0:  49%|████▊     | 2733/5616 [34:48<36:33,  1.31it/s, gpt_loss=0.0604, lr=2.96e-6]
+
Train step of epoch 0:  49%|████▊     | 2734/5616 [34:48<35:18,  1.36it/s, gpt_loss=0.0604, lr=2.96e-6]
+
Train step of epoch 0:  49%|████▊     | 2734/5616 [34:49<35:18,  1.36it/s, gpt_loss=0.00852, lr=2.96e-6]
+
Train step of epoch 0:  49%|████▊     | 2735/5616 [34:49<37:44,  1.27it/s, gpt_loss=0.00852, lr=2.96e-6]
+
Train step of epoch 0:  49%|████▊     | 2735/5616 [34:50<37:44,  1.27it/s, gpt_loss=0.0285, lr=2.95e-6] 
+
Train step of epoch 0:  49%|████▊     | 2736/5616 [34:50<38:50,  1.24it/s, gpt_loss=0.0285, lr=2.95e-6]
+
Train step of epoch 0:  49%|████▊     | 2736/5616 [34:51<38:50,  1.24it/s, gpt_loss=0.113, lr=2.95e-6] 
+
Train step of epoch 0:  49%|████▊     | 2737/5616 [34:51<38:16,  1.25it/s, gpt_loss=0.113, lr=2.95e-6]
+
Train step of epoch 0:  49%|████▊     | 2737/5616 [34:52<38:16,  1.25it/s, gpt_loss=0.0441, lr=2.95e-6]
+
Train step of epoch 0:  49%|████▉     | 2738/5616 [34:52<39:53,  1.20it/s, gpt_loss=0.0441, lr=2.95e-6]
+
Train step of epoch 0:  49%|████▉     | 2738/5616 [34:52<39:53,  1.20it/s, gpt_loss=0.014, lr=2.95e-6] 
+
Train step of epoch 0:  49%|████▉     | 2739/5616 [34:52<37:31,  1.28it/s, gpt_loss=0.014, lr=2.95e-6]
+
Train step of epoch 0:  49%|████▉     | 2739/5616 [34:53<37:31,  1.28it/s, gpt_loss=0.0068, lr=2.95e-6]
+
Train step of epoch 0:  49%|████▉     | 2740/5616 [34:53<37:07,  1.29it/s, gpt_loss=0.0068, lr=2.95e-6]
+
Train step of epoch 0:  49%|████▉     | 2740/5616 [34:54<37:07,  1.29it/s, gpt_loss=0.0319, lr=2.95e-6]
+
Train step of epoch 0:  49%|████▉     | 2741/5616 [34:54<36:54,  1.30it/s, gpt_loss=0.0319, lr=2.95e-6]
+
Train step of epoch 0:  49%|████▉     | 2741/5616 [34:55<36:54,  1.30it/s, gpt_loss=0.00948, lr=2.95e-6]
+
Train step of epoch 0:  49%|████▉     | 2742/5616 [34:55<35:38,  1.34it/s, gpt_loss=0.00948, lr=2.95e-6]
+
Train step of epoch 0:  49%|████▉     | 2742/5616 [34:55<35:38,  1.34it/s, gpt_loss=0.0663, lr=2.95e-6] 
+
Train step of epoch 0:  49%|████▉     | 2743/5616 [34:55<35:01,  1.37it/s, gpt_loss=0.0663, lr=2.95e-6]
+
Train step of epoch 0:  49%|████▉     | 2743/5616 [34:57<35:01,  1.37it/s, gpt_loss=0.0433, lr=2.94e-6]
+
Train step of epoch 0:  49%|████▉     | 2744/5616 [34:57<42:29,  1.13it/s, gpt_loss=0.0433, lr=2.94e-6]
+
Train step of epoch 0:  49%|████▉     | 2744/5616 [34:57<42:29,  1.13it/s, gpt_loss=0.175, lr=2.94e-6] 
+
Train step of epoch 0:  49%|████▉     | 2745/5616 [34:57<40:15,  1.19it/s, gpt_loss=0.175, lr=2.94e-6]
+
Train step of epoch 0:  49%|████▉     | 2745/5616 [34:58<40:15,  1.19it/s, gpt_loss=0.0285, lr=2.94e-6]
+
Train step of epoch 0:  49%|████▉     | 2746/5616 [34:58<39:52,  1.20it/s, gpt_loss=0.0285, lr=2.94e-6]
+
Train step of epoch 0:  49%|████▉     | 2746/5616 [34:59<39:52,  1.20it/s, gpt_loss=0.0332, lr=2.94e-6]
+
Train step of epoch 0:  49%|████▉     | 2747/5616 [34:59<37:22,  1.28it/s, gpt_loss=0.0332, lr=2.94e-6]
+
Train step of epoch 0:  49%|████▉     | 2747/5616 [35:00<37:22,  1.28it/s, gpt_loss=0.0194, lr=2.94e-6]
+
Train step of epoch 0:  49%|████▉     | 2748/5616 [35:00<37:15,  1.28it/s, gpt_loss=0.0194, lr=2.94e-6]
+
Train step of epoch 0:  49%|████▉     | 2748/5616 [35:00<37:15,  1.28it/s, gpt_loss=0.06, lr=2.94e-6]  
+
Train step of epoch 0:  49%|████▉     | 2749/5616 [35:00<35:50,  1.33it/s, gpt_loss=0.06, lr=2.94e-6]
+
Train step of epoch 0:  49%|████▉     | 2749/5616 [35:01<35:50,  1.33it/s, gpt_loss=0.0285, lr=2.94e-6]
+
Train step of epoch 0:  49%|████▉     | 2750/5616 [35:01<34:05,  1.40it/s, gpt_loss=0.0285, lr=2.94e-6]
+
Train step of epoch 0:  49%|████▉     | 2750/5616 [35:02<34:05,  1.40it/s, gpt_loss=0.119, lr=2.94e-6] 
+
Train step of epoch 0:  49%|████▉     | 2751/5616 [35:02<34:53,  1.37it/s, gpt_loss=0.119, lr=2.94e-6]
+
Train step of epoch 0:  49%|████▉     | 2751/5616 [35:03<34:53,  1.37it/s, gpt_loss=0.00893, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2752/5616 [35:03<36:31,  1.31it/s, gpt_loss=0.00893, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2752/5616 [35:03<36:31,  1.31it/s, gpt_loss=0.0431, lr=2.93e-6] 
+
Train step of epoch 0:  49%|████▉     | 2753/5616 [35:03<35:59,  1.33it/s, gpt_loss=0.0431, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2753/5616 [35:04<35:59,  1.33it/s, gpt_loss=0.0693, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2754/5616 [35:04<35:42,  1.34it/s, gpt_loss=0.0693, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2754/5616 [35:05<35:42,  1.34it/s, gpt_loss=0.0668, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2755/5616 [35:05<33:46,  1.41it/s, gpt_loss=0.0668, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2755/5616 [35:05<33:46,  1.41it/s, gpt_loss=0.0326, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2756/5616 [35:05<35:49,  1.33it/s, gpt_loss=0.0326, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2756/5616 [35:06<35:49,  1.33it/s, gpt_loss=0.016, lr=2.93e-6] 
+
Train step of epoch 0:  49%|████▉     | 2757/5616 [35:06<34:58,  1.36it/s, gpt_loss=0.016, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2757/5616 [35:07<34:58,  1.36it/s, gpt_loss=0.0653, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2758/5616 [35:07<33:56,  1.40it/s, gpt_loss=0.0653, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2758/5616 [35:08<33:56,  1.40it/s, gpt_loss=0.0724, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2759/5616 [35:08<36:02,  1.32it/s, gpt_loss=0.0724, lr=2.93e-6]
+
Train step of epoch 0:  49%|████▉     | 2759/5616 [35:09<36:02,  1.32it/s, gpt_loss=0.0254, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2760/5616 [35:09<36:51,  1.29it/s, gpt_loss=0.0254, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2760/5616 [35:09<36:51,  1.29it/s, gpt_loss=0.0554, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2761/5616 [35:09<37:08,  1.28it/s, gpt_loss=0.0554, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2761/5616 [35:10<37:08,  1.28it/s, gpt_loss=0.0491, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2762/5616 [35:10<37:31,  1.27it/s, gpt_loss=0.0491, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2762/5616 [35:11<37:31,  1.27it/s, gpt_loss=0.0192, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2763/5616 [35:11<36:32,  1.30it/s, gpt_loss=0.0192, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2763/5616 [35:12<36:32,  1.30it/s, gpt_loss=0.0197, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2764/5616 [35:12<36:02,  1.32it/s, gpt_loss=0.0197, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2764/5616 [35:12<36:02,  1.32it/s, gpt_loss=0.0307, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2765/5616 [35:12<37:19,  1.27it/s, gpt_loss=0.0307, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2765/5616 [35:13<37:19,  1.27it/s, gpt_loss=0.0817, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2766/5616 [35:13<36:30,  1.30it/s, gpt_loss=0.0817, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2766/5616 [35:14<36:30,  1.30it/s, gpt_loss=0.00699, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2767/5616 [35:14<36:02,  1.32it/s, gpt_loss=0.00699, lr=2.92e-6]
+
Train step of epoch 0:  49%|████▉     | 2767/5616 [35:15<36:02,  1.32it/s, gpt_loss=0.0878, lr=2.91e-6] 
+
Train step of epoch 0:  49%|████▉     | 2768/5616 [35:15<36:36,  1.30it/s, gpt_loss=0.0878, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2768/5616 [35:16<36:36,  1.30it/s, gpt_loss=0.0685, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2769/5616 [35:16<37:38,  1.26it/s, gpt_loss=0.0685, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2769/5616 [35:16<37:38,  1.26it/s, gpt_loss=0.0392, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2770/5616 [35:16<36:50,  1.29it/s, gpt_loss=0.0392, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2770/5616 [35:17<36:50,  1.29it/s, gpt_loss=0.0732, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2771/5616 [35:17<35:58,  1.32it/s, gpt_loss=0.0732, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2771/5616 [35:18<35:58,  1.32it/s, gpt_loss=0.0114, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2772/5616 [35:18<38:03,  1.25it/s, gpt_loss=0.0114, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2772/5616 [35:19<38:03,  1.25it/s, gpt_loss=0.0262, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2773/5616 [35:19<36:22,  1.30it/s, gpt_loss=0.0262, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2773/5616 [35:19<36:22,  1.30it/s, gpt_loss=0.0269, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2774/5616 [35:19<35:52,  1.32it/s, gpt_loss=0.0269, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2774/5616 [35:20<35:52,  1.32it/s, gpt_loss=0.0258, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2775/5616 [35:20<36:01,  1.31it/s, gpt_loss=0.0258, lr=2.91e-6]
+
Train step of epoch 0:  49%|████▉     | 2775/5616 [35:21<36:01,  1.31it/s, gpt_loss=0.0414, lr=2.9e-6] 
+
Train step of epoch 0:  49%|████▉     | 2776/5616 [35:21<37:42,  1.26it/s, gpt_loss=0.0414, lr=2.9e-6]
+
Train step of epoch 0:  49%|████▉     | 2776/5616 [35:22<37:42,  1.26it/s, gpt_loss=0.111, lr=2.9e-6] 
+
Train step of epoch 0:  49%|████▉     | 2777/5616 [35:22<38:00,  1.25it/s, gpt_loss=0.111, lr=2.9e-6]
+
Train step of epoch 0:  49%|████▉     | 2777/5616 [35:23<38:00,  1.25it/s, gpt_loss=0.0352, lr=2.9e-6]
+
Train step of epoch 0:  49%|████▉     | 2778/5616 [35:23<37:34,  1.26it/s, gpt_loss=0.0352, lr=2.9e-6]
+
Train step of epoch 0:  49%|████▉     | 2778/5616 [35:23<37:34,  1.26it/s, gpt_loss=0.0576, lr=2.9e-6]
+
Train step of epoch 0:  49%|████▉     | 2779/5616 [35:23<36:56,  1.28it/s, gpt_loss=0.0576, lr=2.9e-6]
+
Train step of epoch 0:  49%|████▉     | 2779/5616 [35:24<36:56,  1.28it/s, gpt_loss=0.0167, lr=2.9e-6]
+
Train step of epoch 0:  50%|████▉     | 2780/5616 [35:24<36:30,  1.29it/s, gpt_loss=0.0167, lr=2.9e-6]
+
Train step of epoch 0:  50%|████▉     | 2780/5616 [35:25<36:30,  1.29it/s, gpt_loss=0.0354, lr=2.9e-6]
+
Train step of epoch 0:  50%|████▉     | 2781/5616 [35:25<36:12,  1.30it/s, gpt_loss=0.0354, lr=2.9e-6]
+
Train step of epoch 0:  50%|████▉     | 2781/5616 [35:25<36:12,  1.30it/s, gpt_loss=0.0547, lr=2.9e-6]
+
Train step of epoch 0:  50%|████▉     | 2782/5616 [35:25<35:05,  1.35it/s, gpt_loss=0.0547, lr=2.9e-6]
+
Train step of epoch 0:  50%|████▉     | 2782/5616 [35:26<35:05,  1.35it/s, gpt_loss=0.0581, lr=2.9e-6]
+
Train step of epoch 0:  50%|████▉     | 2783/5616 [35:26<33:47,  1.40it/s, gpt_loss=0.0581, lr=2.9e-6]
+
Train step of epoch 0:  50%|████▉     | 2783/5616 [35:27<33:47,  1.40it/s, gpt_loss=0.0087, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2784/5616 [35:27<34:56,  1.35it/s, gpt_loss=0.0087, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2784/5616 [35:28<34:56,  1.35it/s, gpt_loss=0.0494, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2785/5616 [35:28<33:24,  1.41it/s, gpt_loss=0.0494, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2785/5616 [35:28<33:24,  1.41it/s, gpt_loss=0.0555, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2786/5616 [35:28<34:37,  1.36it/s, gpt_loss=0.0555, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2786/5616 [35:29<34:37,  1.36it/s, gpt_loss=0.0927, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2787/5616 [35:29<34:45,  1.36it/s, gpt_loss=0.0927, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2787/5616 [35:30<34:45,  1.36it/s, gpt_loss=0.0314, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2788/5616 [35:30<34:55,  1.35it/s, gpt_loss=0.0314, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2788/5616 [35:31<34:55,  1.35it/s, gpt_loss=0.0342, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2789/5616 [35:31<33:57,  1.39it/s, gpt_loss=0.0342, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2789/5616 [35:31<33:57,  1.39it/s, gpt_loss=0.0314, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2790/5616 [35:31<33:22,  1.41it/s, gpt_loss=0.0314, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2790/5616 [35:32<33:22,  1.41it/s, gpt_loss=0.0356, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2791/5616 [35:32<35:57,  1.31it/s, gpt_loss=0.0356, lr=2.89e-6]
+
Train step of epoch 0:  50%|████▉     | 2791/5616 [35:33<35:57,  1.31it/s, gpt_loss=0.018, lr=2.88e-6] 
+
Train step of epoch 0:  50%|████▉     | 2792/5616 [35:33<37:25,  1.26it/s, gpt_loss=0.018, lr=2.88e-6]
+
Train step of epoch 0:  50%|████▉     | 2792/5616 [35:34<37:25,  1.26it/s, gpt_loss=0.0763, lr=2.88e-6]
+
Train step of epoch 0:  50%|████▉     | 2793/5616 [35:34<35:30,  1.32it/s, gpt_loss=0.0763, lr=2.88e-6]
+
Train step of epoch 0:  50%|████▉     | 2793/5616 [35:34<35:30,  1.32it/s, gpt_loss=0.0896, lr=2.88e-6]
+
Train step of epoch 0:  50%|████▉     | 2794/5616 [35:34<34:15,  1.37it/s, gpt_loss=0.0896, lr=2.88e-6]
+
Train step of epoch 0:  50%|████▉     | 2794/5616 [35:35<34:15,  1.37it/s, gpt_loss=0.0869, lr=2.88e-6]
+
Train step of epoch 0:  50%|████▉     | 2795/5616 [35:35<36:48,  1.28it/s, gpt_loss=0.0869, lr=2.88e-6]
+
Train step of epoch 0:  50%|████▉     | 2795/5616 [35:36<36:48,  1.28it/s, gpt_loss=0.0241, lr=2.88e-6]
+
Train step of epoch 0:  50%|████▉     | 2796/5616 [35:36<35:59,  1.31it/s, gpt_loss=0.0241, lr=2.88e-6]
+
Train step of epoch 0:  50%|████▉     | 2796/5616 [35:37<35:59,  1.31it/s, gpt_loss=0.0071, lr=2.88e-6]
+
Train step of epoch 0:  50%|████▉     | 2797/5616 [35:37<34:21,  1.37it/s, gpt_loss=0.0071, lr=2.88e-6]
+
Train step of epoch 0:  50%|████▉     | 2797/5616 [35:37<34:21,  1.37it/s, gpt_loss=0.00263, lr=2.88e-6]
+
Train step of epoch 0:  50%|████▉     | 2798/5616 [35:37<33:21,  1.41it/s, gpt_loss=0.00263, lr=2.88e-6]
+
Train step of epoch 0:  50%|████▉     | 2798/5616 [35:38<33:21,  1.41it/s, gpt_loss=0.0291, lr=2.88e-6] 
+
Train step of epoch 0:  50%|████▉     | 2799/5616 [35:38<32:41,  1.44it/s, gpt_loss=0.0291, lr=2.88e-6][2025-06-30 07:13:47,681] [INFO] [logging.py:128:log_dist] [Rank 0] step=700, skipped=0, lr=[2.8719913733107376e-06, 2.8719913733107376e-06], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 07:13:47,682] [INFO] [timer.py:264:stop] epoch=0/micro_step=2800/global_step=700, RunningAvgSamplesPerSec=42.44696115442617, CurrSamplesPerSec=47.927002628637425, MemAllocated=15.83GB, MaxMemAllocated=51.79GB
+
+
Train step of epoch 0:  50%|████▉     | 2799/5616 [35:39<32:41,  1.44it/s, gpt_loss=0.0277, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2800/5616 [35:39<33:09,  1.42it/s, gpt_loss=0.0277, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2800/5616 [35:40<33:09,  1.42it/s, gpt_loss=0.0298, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2801/5616 [35:40<39:27,  1.19it/s, gpt_loss=0.0298, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2801/5616 [35:41<39:27,  1.19it/s, gpt_loss=0.0131, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2802/5616 [35:41<37:32,  1.25it/s, gpt_loss=0.0131, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2802/5616 [35:41<37:32,  1.25it/s, gpt_loss=0.0435, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2803/5616 [35:41<37:52,  1.24it/s, gpt_loss=0.0435, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2803/5616 [35:42<37:52,  1.24it/s, gpt_loss=0.0063, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2804/5616 [35:42<37:36,  1.25it/s, gpt_loss=0.0063, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2804/5616 [35:43<37:36,  1.25it/s, gpt_loss=0.0165, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2805/5616 [35:43<36:36,  1.28it/s, gpt_loss=0.0165, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2805/5616 [35:44<36:36,  1.28it/s, gpt_loss=0.0268, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2806/5616 [35:44<43:01,  1.09it/s, gpt_loss=0.0268, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2806/5616 [35:45<43:01,  1.09it/s, gpt_loss=0.0108, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2807/5616 [35:45<40:21,  1.16it/s, gpt_loss=0.0108, lr=2.87e-6]
+
Train step of epoch 0:  50%|████▉     | 2807/5616 [35:46<40:21,  1.16it/s, gpt_loss=0.00247, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2808/5616 [35:46<39:59,  1.17it/s, gpt_loss=0.00247, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2808/5616 [35:46<39:59,  1.17it/s, gpt_loss=0.00632, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2809/5616 [35:46<38:33,  1.21it/s, gpt_loss=0.00632, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2809/5616 [35:47<38:33,  1.21it/s, gpt_loss=0.00932, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2810/5616 [35:47<36:45,  1.27it/s, gpt_loss=0.00932, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2810/5616 [35:48<36:45,  1.27it/s, gpt_loss=0.0146, lr=2.86e-6] 
+
Train step of epoch 0:  50%|█████     | 2811/5616 [35:48<35:27,  1.32it/s, gpt_loss=0.0146, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2811/5616 [35:49<35:27,  1.32it/s, gpt_loss=0.136, lr=2.86e-6] 
+
Train step of epoch 0:  50%|█████     | 2812/5616 [35:49<37:29,  1.25it/s, gpt_loss=0.136, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2812/5616 [35:49<37:29,  1.25it/s, gpt_loss=0.0339, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2813/5616 [35:49<36:22,  1.28it/s, gpt_loss=0.0339, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2813/5616 [35:50<36:22,  1.28it/s, gpt_loss=0.0356, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2814/5616 [35:50<35:42,  1.31it/s, gpt_loss=0.0356, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2814/5616 [35:51<35:42,  1.31it/s, gpt_loss=0.0319, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2815/5616 [35:51<36:35,  1.28it/s, gpt_loss=0.0319, lr=2.86e-6]
+
Train step of epoch 0:  50%|█████     | 2815/5616 [35:52<36:35,  1.28it/s, gpt_loss=0.0287, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2816/5616 [35:52<36:53,  1.27it/s, gpt_loss=0.0287, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2816/5616 [35:52<36:53,  1.27it/s, gpt_loss=0.089, lr=2.85e-6] 
+
Train step of epoch 0:  50%|█████     | 2817/5616 [35:52<34:38,  1.35it/s, gpt_loss=0.089, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2817/5616 [35:53<34:38,  1.35it/s, gpt_loss=0.0185, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2818/5616 [35:53<35:00,  1.33it/s, gpt_loss=0.0185, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2818/5616 [35:54<35:00,  1.33it/s, gpt_loss=0.0276, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2819/5616 [35:54<35:24,  1.32it/s, gpt_loss=0.0276, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2819/5616 [35:55<35:24,  1.32it/s, gpt_loss=0.00965, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2820/5616 [35:55<37:03,  1.26it/s, gpt_loss=0.00965, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2820/5616 [35:56<37:03,  1.26it/s, gpt_loss=0.0693, lr=2.85e-6] 
+
Train step of epoch 0:  50%|█████     | 2821/5616 [35:56<37:10,  1.25it/s, gpt_loss=0.0693, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2821/5616 [35:56<37:10,  1.25it/s, gpt_loss=0.0216, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2822/5616 [35:56<35:44,  1.30it/s, gpt_loss=0.0216, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2822/5616 [35:57<35:44,  1.30it/s, gpt_loss=0.0331, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2823/5616 [35:57<35:41,  1.30it/s, gpt_loss=0.0331, lr=2.85e-6]
+
Train step of epoch 0:  50%|█████     | 2823/5616 [35:58<35:41,  1.30it/s, gpt_loss=0.0017, lr=2.84e-6]
+
Train step of epoch 0:  50%|█████     | 2824/5616 [35:58<36:23,  1.28it/s, gpt_loss=0.0017, lr=2.84e-6]
+
Train step of epoch 0:  50%|█████     | 2824/5616 [35:59<36:23,  1.28it/s, gpt_loss=0.0767, lr=2.84e-6]
+
Train step of epoch 0:  50%|���████     | 2825/5616 [35:59<34:48,  1.34it/s, gpt_loss=0.0767, lr=2.84e-6]
+
Train step of epoch 0:  50%|█████     | 2825/5616 [35:59<34:48,  1.34it/s, gpt_loss=0.159, lr=2.84e-6] 
+
Train step of epoch 0:  50%|█████     | 2826/5616 [35:59<34:32,  1.35it/s, gpt_loss=0.159, lr=2.84e-6]
+
Train step of epoch 0:  50%|█████     | 2826/5616 [36:00<34:32,  1.35it/s, gpt_loss=0.0414, lr=2.84e-6]
+
Train step of epoch 0:  50%|█████     | 2827/5616 [36:00<33:43,  1.38it/s, gpt_loss=0.0414, lr=2.84e-6]
+
Train step of epoch 0:  50%|█████     | 2827/5616 [36:01<33:43,  1.38it/s, gpt_loss=0.0463, lr=2.84e-6]
+
Train step of epoch 0:  50%|█████     | 2828/5616 [36:01<34:49,  1.33it/s, gpt_loss=0.0463, lr=2.84e-6]
+
Train step of epoch 0:  50%|█████     | 2828/5616 [36:02<34:49,  1.33it/s, gpt_loss=0.0757, lr=2.84e-6]
+
Train step of epoch 0:  50%|█████     | 2829/5616 [36:02<33:55,  1.37it/s, gpt_loss=0.0757, lr=2.84e-6]
+
Train step of epoch 0:  50%|█████     | 2829/5616 [36:02<33:55,  1.37it/s, gpt_loss=0.015, lr=2.84e-6] 
+
Train step of epoch 0:  50%|█████     | 2830/5616 [36:02<33:34,  1.38it/s, gpt_loss=0.015, lr=2.84e-6]
+
Train step of epoch 0:  50%|█████     | 2830/5616 [36:03<33:34,  1.38it/s, gpt_loss=0.00685, lr=2.84e-6]
+
Train step of epoch 0:  50%|█████     | 2831/5616 [36:03<35:01,  1.33it/s, gpt_loss=0.00685, lr=2.84e-6]
+
Train step of epoch 0:  50%|█████     | 2831/5616 [36:04<35:01,  1.33it/s, gpt_loss=0.0995, lr=2.83e-6] 
+
Train step of epoch 0:  50%|█████     | 2832/5616 [36:04<34:16,  1.35it/s, gpt_loss=0.0995, lr=2.83e-6]
+
Train step of epoch 0:  50%|█████     | 2832/5616 [36:05<34:16,  1.35it/s, gpt_loss=0.0288, lr=2.83e-6]
+
Train step of epoch 0:  50%|█████     | 2833/5616 [36:05<35:01,  1.32it/s, gpt_loss=0.0288, lr=2.83e-6]
+
Train step of epoch 0:  50%|█████     | 2833/5616 [36:05<35:01,  1.32it/s, gpt_loss=0.00687, lr=2.83e-6]
+
Train step of epoch 0:  50%|█████     | 2834/5616 [36:05<34:05,  1.36it/s, gpt_loss=0.00687, lr=2.83e-6]
+
Train step of epoch 0:  50%|█████     | 2834/5616 [36:06<34:05,  1.36it/s, gpt_loss=0.0336, lr=2.83e-6] 
+
Train step of epoch 0:  50%|█████     | 2835/5616 [36:06<33:58,  1.36it/s, gpt_loss=0.0336, lr=2.83e-6]
+
Train step of epoch 0:  50%|█████     | 2835/5616 [36:07<33:58,  1.36it/s, gpt_loss=0.0595, lr=2.83e-6]
+
Train step of epoch 0:  50%|█████     | 2836/5616 [36:07<34:49,  1.33it/s, gpt_loss=0.0595, lr=2.83e-6]
+
Train step of epoch 0:  50%|█████     | 2836/5616 [36:08<34:49,  1.33it/s, gpt_loss=0.0146, lr=2.83e-6]
+
Train step of epoch 0:  51%|█████     | 2837/5616 [36:08<34:46,  1.33it/s, gpt_loss=0.0146, lr=2.83e-6]
+
Train step of epoch 0:  51%|█████     | 2837/5616 [36:08<34:46,  1.33it/s, gpt_loss=0.0459, lr=2.83e-6]
+
Train step of epoch 0:  51%|█████     | 2838/5616 [36:08<34:45,  1.33it/s, gpt_loss=0.0459, lr=2.83e-6]
+
Train step of epoch 0:  51%|█████     | 2838/5616 [36:09<34:45,  1.33it/s, gpt_loss=0.075, lr=2.83e-6] 
+
Train step of epoch 0:  51%|█████     | 2839/5616 [36:09<33:23,  1.39it/s, gpt_loss=0.075, lr=2.83e-6]
+
Train step of epoch 0:  51%|█████     | 2839/5616 [36:10<33:23,  1.39it/s, gpt_loss=0.00158, lr=2.82e-6]
+
Train step of epoch 0:  51%|█████     | 2840/5616 [36:10<33:51,  1.37it/s, gpt_loss=0.00158, lr=2.82e-6]
+
Train step of epoch 0:  51%|█████     | 2840/5616 [36:10<33:51,  1.37it/s, gpt_loss=0.0242, lr=2.82e-6] 
+
Train step of epoch 0:  51%|█████     | 2841/5616 [36:10<33:20,  1.39it/s, gpt_loss=0.0242, lr=2.82e-6]
+
Train step of epoch 0:  51%|█████     | 2841/5616 [36:11<33:20,  1.39it/s, gpt_loss=0.0161, lr=2.82e-6]
+
Train step of epoch 0:  51%|█████     | 2842/5616 [36:11<32:50,  1.41it/s, gpt_loss=0.0161, lr=2.82e-6]
+
Train step of epoch 0:  51%|█████     | 2842/5616 [36:12<32:50,  1.41it/s, gpt_loss=0.00352, lr=2.82e-6]
+
Train step of epoch 0:  51%|█████     | 2843/5616 [36:12<34:09,  1.35it/s, gpt_loss=0.00352, lr=2.82e-6]
+
Train step of epoch 0:  51%|█████     | 2843/5616 [36:13<34:09,  1.35it/s, gpt_loss=0.0721, lr=2.81e-6] 
+
Train step of epoch 0:  51%|█████     | 2844/5616 [36:13<34:11,  1.35it/s, gpt_loss=0.0721, lr=2.81e-6]
+
Train step of epoch 0:  51%|█████     | 2844/5616 [36:13<34:11,  1.35it/s, gpt_loss=0.0965, lr=2.81e-6]
+
Train step of epoch 0:  51%|█████     | 2845/5616 [36:13<34:46,  1.33it/s, gpt_loss=0.0965, lr=2.81e-6]
+
Train step of epoch 0:  51%|█████     | 2845/5616 [36:14<34:46,  1.33it/s, gpt_loss=0.039, lr=2.81e-6] 
+
Train step of epoch 0:  51%|█████     | 2846/5616 [36:14<36:28,  1.27it/s, gpt_loss=0.039, lr=2.81e-6]
+
Train step of epoch 0:  51%|█████     | 2846/5616 [36:15<36:28,  1.27it/s, gpt_loss=0.0316, lr=2.81e-6]
+
Train step of epoch 0:  51%|█████     | 2847/5616 [36:15<34:51,  1.32it/s, gpt_loss=0.0316, lr=2.81e-6]
+
Train step of epoch 0:  51%|█████     | 2847/5616 [36:16<34:51,  1.32it/s, gpt_loss=0.0122, lr=2.81e-6]
+
Train step of epoch 0:  51%|█████     | 2848/5616 [36:16<35:39,  1.29it/s, gpt_loss=0.0122, lr=2.81e-6]
+
Train step of epoch 0:  51%|█████     | 2848/5616 [36:16<35:39,  1.29it/s, gpt_loss=0.147, lr=2.81e-6] 
+
Train step of epoch 0:  51%|█████     | 2849/5616 [36:16<34:06,  1.35it/s, gpt_loss=0.147, lr=2.81e-6]
+
Train step of epoch 0:  51%|█████     | 2849/5616 [36:17<34:06,  1.35it/s, gpt_loss=0.0201, lr=2.81e-6]
+
Train step of epoch 0:  51%|█████     | 2850/5616 [36:17<34:21,  1.34it/s, gpt_loss=0.0201, lr=2.81e-6]
+
Train step of epoch 0:  51%|█████     | 2850/5616 [36:18<34:21,  1.34it/s, gpt_loss=0.0296, lr=2.81e-6]
+
Train step of epoch 0:  51%|█████     | 2851/5616 [36:18<35:34,  1.30it/s, gpt_loss=0.0296, lr=2.81e-6]
+
Train step of epoch 0:  51%|█████     | 2851/5616 [36:19<35:34,  1.30it/s, gpt_loss=0.0356, lr=2.8e-6] 
+
Train step of epoch 0:  51%|█████     | 2852/5616 [36:19<35:23,  1.30it/s, gpt_loss=0.0356, lr=2.8e-6]
+
Train step of epoch 0:  51%|█████     | 2852/5616 [36:20<35:23,  1.30it/s, gpt_loss=0.0168, lr=2.8e-6]
+
Train step of epoch 0:  51%|█████     | 2853/5616 [36:20<36:43,  1.25it/s, gpt_loss=0.0168, lr=2.8e-6]
+
Train step of epoch 0:  51%|█████     | 2853/5616 [36:20<36:43,  1.25it/s, gpt_loss=0.0031, lr=2.8e-6]
+
Train step of epoch 0:  51%|█████     | 2854/5616 [36:20<35:00,  1.31it/s, gpt_loss=0.0031, lr=2.8e-6]
+
Train step of epoch 0:  51%|█████     | 2854/5616 [36:21<35:00,  1.31it/s, gpt_loss=0.0347, lr=2.8e-6]
+
Train step of epoch 0:  51%|█████     | 2855/5616 [36:21<33:25,  1.38it/s, gpt_loss=0.0347, lr=2.8e-6]
+
Train step of epoch 0:  51%|█████     | 2855/5616 [36:22<33:25,  1.38it/s, gpt_loss=0.00157, lr=2.8e-6]
+
Train step of epoch 0:  51%|█████     | 2856/5616 [36:22<33:40,  1.37it/s, gpt_loss=0.00157, lr=2.8e-6]
+
Train step of epoch 0:  51%|█████     | 2856/5616 [36:22<33:40,  1.37it/s, gpt_loss=0.0177, lr=2.8e-6] 
+
Train step of epoch 0:  51%|█████     | 2857/5616 [36:22<33:41,  1.36it/s, gpt_loss=0.0177, lr=2.8e-6]
+
Train step of epoch 0:  51%|█████     | 2857/5616 [36:23<33:41,  1.36it/s, gpt_loss=0.02, lr=2.8e-6]  
+
Train step of epoch 0:  51%|█████     | 2858/5616 [36:23<33:32,  1.37it/s, gpt_loss=0.02, lr=2.8e-6]
+
Train step of epoch 0:  51%|█████     | 2858/5616 [36:24<33:32,  1.37it/s, gpt_loss=0.0322, lr=2.8e-6]
+
Train step of epoch 0:  51%|█████     | 2859/5616 [36:24<33:47,  1.36it/s, gpt_loss=0.0322, lr=2.8e-6]
+
Train step of epoch 0:  51%|█████     | 2859/5616 [36:25<33:47,  1.36it/s, gpt_loss=0.137, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2860/5616 [36:25<33:17,  1.38it/s, gpt_loss=0.137, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2860/5616 [36:25<33:17,  1.38it/s, gpt_loss=0.0156, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2861/5616 [36:25<33:53,  1.36it/s, gpt_loss=0.0156, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2861/5616 [36:26<33:53,  1.36it/s, gpt_loss=0.0198, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2862/5616 [36:26<32:54,  1.39it/s, gpt_loss=0.0198, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2862/5616 [36:27<32:54,  1.39it/s, gpt_loss=0.00329, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2863/5616 [36:27<34:25,  1.33it/s, gpt_loss=0.00329, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2863/5616 [36:28<34:25,  1.33it/s, gpt_loss=0.0123, lr=2.79e-6] 
+
Train step of epoch 0:  51%|█████     | 2864/5616 [36:28<34:22,  1.33it/s, gpt_loss=0.0123, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2864/5616 [36:28<34:22,  1.33it/s, gpt_loss=0.0406, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2865/5616 [36:28<33:48,  1.36it/s, gpt_loss=0.0406, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2865/5616 [36:29<33:48,  1.36it/s, gpt_loss=0.0328, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2866/5616 [36:29<38:03,  1.20it/s, gpt_loss=0.0328, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2866/5616 [36:30<38:03,  1.20it/s, gpt_loss=0.0481, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2867/5616 [36:30<42:04,  1.09it/s, gpt_loss=0.0481, lr=2.79e-6]
+
Train step of epoch 0:  51%|█████     | 2867/5616 [36:31<42:04,  1.09it/s, gpt_loss=0.0117, lr=2.78e-6]
+
Train step of epoch 0:  51%|████���     | 2868/5616 [36:31<39:16,  1.17it/s, gpt_loss=0.0117, lr=2.78e-6]
+
Train step of epoch 0:  51%|█████     | 2868/5616 [36:32<39:16,  1.17it/s, gpt_loss=0.00694, lr=2.78e-6]
+
Train step of epoch 0:  51%|█████     | 2869/5616 [36:32<38:38,  1.19it/s, gpt_loss=0.00694, lr=2.78e-6]
+
Train step of epoch 0:  51%|█████     | 2869/5616 [36:33<38:38,  1.19it/s, gpt_loss=0.0159, lr=2.78e-6] 
+
Train step of epoch 0:  51%|█████     | 2870/5616 [36:33<36:40,  1.25it/s, gpt_loss=0.0159, lr=2.78e-6]
+
Train step of epoch 0:  51%|█████     | 2870/5616 [36:34<36:40,  1.25it/s, gpt_loss=0.0142, lr=2.78e-6]
+
Train step of epoch 0:  51%|█████     | 2871/5616 [36:34<36:49,  1.24it/s, gpt_loss=0.0142, lr=2.78e-6]
+
Train step of epoch 0:  51%|█████     | 2871/5616 [36:34<36:49,  1.24it/s, gpt_loss=0.0158, lr=2.78e-6]
+
Train step of epoch 0:  51%|█████     | 2872/5616 [36:34<38:55,  1.18it/s, gpt_loss=0.0158, lr=2.78e-6]
+
Train step of epoch 0:  51%|█████     | 2872/5616 [36:35<38:55,  1.18it/s, gpt_loss=0.0151, lr=2.78e-6]
+
Train step of epoch 0:  51%|█████     | 2873/5616 [36:35<38:49,  1.18it/s, gpt_loss=0.0151, lr=2.78e-6]
+
Train step of epoch 0:  51%|█████     | 2873/5616 [36:36<38:49,  1.18it/s, gpt_loss=0.00408, lr=2.78e-6]
+
Train step of epoch 0:  51%|█████     | 2874/5616 [36:36<37:26,  1.22it/s, gpt_loss=0.00408, lr=2.78e-6]
+
Train step of epoch 0:  51%|█████     | 2874/5616 [36:37<37:26,  1.22it/s, gpt_loss=0.0696, lr=2.78e-6] 
+
Train step of epoch 0:  51%|█████     | 2875/5616 [36:37<37:21,  1.22it/s, gpt_loss=0.0696, lr=2.78e-6]
+
Train step of epoch 0:  51%|█████     | 2875/5616 [36:38<37:21,  1.22it/s, gpt_loss=0.0321, lr=2.77e-6]
+
Train step of epoch 0:  51%|█████     | 2876/5616 [36:38<38:34,  1.18it/s, gpt_loss=0.0321, lr=2.77e-6]
+
Train step of epoch 0:  51%|█████     | 2876/5616 [36:39<38:34,  1.18it/s, gpt_loss=0.053, lr=2.77e-6] 
+
Train step of epoch 0:  51%|█████     | 2877/5616 [36:39<38:07,  1.20it/s, gpt_loss=0.053, lr=2.77e-6]
+
Train step of epoch 0:  51%|█████     | 2877/5616 [36:39<38:07,  1.20it/s, gpt_loss=0.0518, lr=2.77e-6]
+
Train step of epoch 0:  51%|█████     | 2878/5616 [36:39<36:45,  1.24it/s, gpt_loss=0.0518, lr=2.77e-6]
+
Train step of epoch 0:  51%|█████     | 2878/5616 [36:40<36:45,  1.24it/s, gpt_loss=0.06, lr=2.77e-6]  
+
Train step of epoch 0:  51%|█████▏    | 2879/5616 [36:40<35:51,  1.27it/s, gpt_loss=0.06, lr=2.77e-6]
+
Train step of epoch 0:  51%|█████▏    | 2879/5616 [36:41<35:51,  1.27it/s, gpt_loss=0.072, lr=2.77e-6]
+
Train step of epoch 0:  51%|█████▏    | 2880/5616 [36:41<38:52,  1.17it/s, gpt_loss=0.072, lr=2.77e-6]
+
Train step of epoch 0:  51%|█████▏    | 2880/5616 [36:42<38:52,  1.17it/s, gpt_loss=0.0578, lr=2.77e-6]
+
Train step of epoch 0:  51%|█████▏    | 2881/5616 [36:42<35:47,  1.27it/s, gpt_loss=0.0578, lr=2.77e-6]
+
Train step of epoch 0:  51%|█████▏    | 2881/5616 [36:42<35:47,  1.27it/s, gpt_loss=0.000316, lr=2.77e-6]
+
Train step of epoch 0:  51%|█████▏    | 2882/5616 [36:42<34:35,  1.32it/s, gpt_loss=0.000316, lr=2.77e-6]
+
Train step of epoch 0:  51%|█████▏    | 2882/5616 [36:43<34:35,  1.32it/s, gpt_loss=0.0147, lr=2.77e-6]  
+
Train step of epoch 0:  51%|█████▏    | 2883/5616 [36:43<36:01,  1.26it/s, gpt_loss=0.0147, lr=2.77e-6]
+
Train step of epoch 0:  51%|█████▏    | 2883/5616 [36:44<36:01,  1.26it/s, gpt_loss=0.00228, lr=2.76e-6]
+
Train step of epoch 0:  51%|█████▏    | 2884/5616 [36:44<35:36,  1.28it/s, gpt_loss=0.00228, lr=2.76e-6]
+
Train step of epoch 0:  51%|█████▏    | 2884/5616 [36:45<35:36,  1.28it/s, gpt_loss=0.086, lr=2.76e-6]  
+
Train step of epoch 0:  51%|█████▏    | 2885/5616 [36:45<34:30,  1.32it/s, gpt_loss=0.086, lr=2.76e-6]
+
Train step of epoch 0:  51%|█████▏    | 2885/5616 [36:45<34:30,  1.32it/s, gpt_loss=0.0218, lr=2.76e-6]
+
Train step of epoch 0:  51%|█████▏    | 2886/5616 [36:45<33:27,  1.36it/s, gpt_loss=0.0218, lr=2.76e-6]
+
Train step of epoch 0:  51%|█████▏    | 2886/5616 [36:46<33:27,  1.36it/s, gpt_loss=0.00801, lr=2.76e-6]
+
Train step of epoch 0:  51%|█████▏    | 2887/5616 [36:46<32:17,  1.41it/s, gpt_loss=0.00801, lr=2.76e-6]
+
Train step of epoch 0:  51%|█████▏    | 2887/5616 [36:47<32:17,  1.41it/s, gpt_loss=0.0232, lr=2.76e-6] 
+
Train step of epoch 0:  51%|█████▏    | 2888/5616 [36:47<34:03,  1.33it/s, gpt_loss=0.0232, lr=2.76e-6]
+
Train step of epoch 0:  51%|█████▏    | 2888/5616 [36:48<34:03,  1.33it/s, gpt_loss=0.00869, lr=2.76e-6]
+
Train step of epoch 0:  51%|█████▏    | 2889/5616 [36:48<32:25,  1.40it/s, gpt_loss=0.00869, lr=2.76e-6]
+
Train step of epoch 0:  51%|█████▏    | 2889/5616 [36:48<32:25,  1.40it/s, gpt_loss=0.0317, lr=2.76e-6] 
+
Train step of epoch 0:  51%|█████▏    | 2890/5616 [36:48<31:56,  1.42it/s, gpt_loss=0.0317, lr=2.76e-6]
+
Train step of epoch 0:  51%|█████▏    | 2890/5616 [36:49<31:56,  1.42it/s, gpt_loss=0.0812, lr=2.76e-6]
+
Train step of epoch 0:  51%|█████▏    | 2891/5616 [36:49<32:02,  1.42it/s, gpt_loss=0.0812, lr=2.76e-6]
+
Train step of epoch 0:  51%|█████▏    | 2891/5616 [36:50<32:02,  1.42it/s, gpt_loss=0.101, lr=2.75e-6] 
+
Train step of epoch 0:  51%|█████▏    | 2892/5616 [36:50<33:04,  1.37it/s, gpt_loss=0.101, lr=2.75e-6]
+
Train step of epoch 0:  51%|█████▏    | 2892/5616 [36:50<33:04,  1.37it/s, gpt_loss=0.072, lr=2.75e-6]
+
Train step of epoch 0:  52%|█████▏    | 2893/5616 [36:50<32:10,  1.41it/s, gpt_loss=0.072, lr=2.75e-6]
+
Train step of epoch 0:  52%|█████▏    | 2893/5616 [36:51<32:10,  1.41it/s, gpt_loss=0.0493, lr=2.75e-6]
+
Train step of epoch 0:  52%|█████▏    | 2894/5616 [36:51<31:59,  1.42it/s, gpt_loss=0.0493, lr=2.75e-6]
+
Train step of epoch 0:  52%|█████▏    | 2894/5616 [36:52<31:59,  1.42it/s, gpt_loss=0.0242, lr=2.75e-6]
+
Train step of epoch 0:  52%|█████▏    | 2895/5616 [36:52<31:45,  1.43it/s, gpt_loss=0.0242, lr=2.75e-6]
+
Train step of epoch 0:  52%|█████▏    | 2895/5616 [36:53<31:45,  1.43it/s, gpt_loss=0.0126, lr=2.75e-6]
+
Train step of epoch 0:  52%|█████▏    | 2896/5616 [36:53<33:39,  1.35it/s, gpt_loss=0.0126, lr=2.75e-6]
+
Train step of epoch 0:  52%|█████▏    | 2896/5616 [36:53<33:39,  1.35it/s, gpt_loss=0.107, lr=2.75e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2897/5616 [36:53<32:26,  1.40it/s, gpt_loss=0.107, lr=2.75e-6]
+
Train step of epoch 0:  52%|█████▏    | 2897/5616 [36:54<32:26,  1.40it/s, gpt_loss=0.057, lr=2.75e-6]
+
Train step of epoch 0:  52%|█████▏    | 2898/5616 [36:54<33:06,  1.37it/s, gpt_loss=0.057, lr=2.75e-6]
+
Train step of epoch 0:  52%|█████▏    | 2898/5616 [36:55<33:06,  1.37it/s, gpt_loss=0.00152, lr=2.75e-6]
+
Train step of epoch 0:  52%|█████▏    | 2899/5616 [36:55<32:16,  1.40it/s, gpt_loss=0.00152, lr=2.75e-6]
+
Train step of epoch 0:  52%|█████▏    | 2899/5616 [36:55<32:16,  1.40it/s, gpt_loss=0.0699, lr=2.74e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2900/5616 [36:55<32:29,  1.39it/s, gpt_loss=0.0699, lr=2.74e-6]
+
Train step of epoch 0:  52%|█████▏    | 2900/5616 [36:56<32:29,  1.39it/s, gpt_loss=0.0232, lr=2.74e-6]
+
Train step of epoch 0:  52%|█████▏    | 2901/5616 [36:56<31:15,  1.45it/s, gpt_loss=0.0232, lr=2.74e-6]
+
Train step of epoch 0:  52%|█████▏    | 2901/5616 [36:57<31:15,  1.45it/s, gpt_loss=0.032, lr=2.74e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2902/5616 [36:57<34:00,  1.33it/s, gpt_loss=0.032, lr=2.74e-6]
+
Train step of epoch 0:  52%|█████▏    | 2902/5616 [36:58<34:00,  1.33it/s, gpt_loss=0.0473, lr=2.74e-6]
+
Train step of epoch 0:  52%|█████▏    | 2903/5616 [36:58<33:07,  1.36it/s, gpt_loss=0.0473, lr=2.74e-6]
+
Train step of epoch 0:  52%|█████▏    | 2903/5616 [36:59<33:07,  1.36it/s, gpt_loss=0.0175, lr=2.74e-6]
+
Train step of epoch 0:  52%|█████▏    | 2904/5616 [36:59<37:12,  1.21it/s, gpt_loss=0.0175, lr=2.74e-6]
+
Train step of epoch 0:  52%|█████▏    | 2904/5616 [36:59<37:12,  1.21it/s, gpt_loss=0.00577, lr=2.74e-6]
+
Train step of epoch 0:  52%|█████▏    | 2905/5616 [36:59<36:09,  1.25it/s, gpt_loss=0.00577, lr=2.74e-6]
+
Train step of epoch 0:  52%|█████▏    | 2905/5616 [37:00<36:09,  1.25it/s, gpt_loss=0.0221, lr=2.74e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2906/5616 [37:00<36:47,  1.23it/s, gpt_loss=0.0221, lr=2.74e-6]
+
Train step of epoch 0:  52%|█████▏    | 2906/5616 [37:01<36:47,  1.23it/s, gpt_loss=0.0136, lr=2.74e-6]
+
Train step of epoch 0:  52%|█████▏    | 2907/5616 [37:01<36:35,  1.23it/s, gpt_loss=0.0136, lr=2.74e-6]
+
Train step of epoch 0:  52%|█████▏    | 2907/5616 [37:02<36:35,  1.23it/s, gpt_loss=0.00921, lr=2.73e-6]
+
Train step of epoch 0:  52%|█████▏    | 2908/5616 [37:02<35:55,  1.26it/s, gpt_loss=0.00921, lr=2.73e-6]
+
Train step of epoch 0:  52%|█████▏    | 2908/5616 [37:03<35:55,  1.26it/s, gpt_loss=0.0726, lr=2.73e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2909/5616 [37:03<35:06,  1.28it/s, gpt_loss=0.0726, lr=2.73e-6]
+
Train step of epoch 0:  52%|█████▏    | 2909/5616 [37:03<35:06,  1.28it/s, gpt_loss=0.0559, lr=2.73e-6]
+
Train step of epoch 0:  52%|█████▏    | 2910/5616 [37:03<33:00,  1.37it/s, gpt_loss=0.0559, lr=2.73e-6]
+
Train step of epoch 0:  52%|█████▏    | 2910/5616 [37:04<33:00,  1.37it/s, gpt_loss=0.115, lr=2.73e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2911/5616 [37:04<32:54,  1.37it/s, gpt_loss=0.115, lr=2.73e-6]
+
Train step of epoch 0:  52%|█████▏    | 2911/5616 [37:05<32:54,  1.37it/s, gpt_loss=0.0051, lr=2.73e-6]
+
Train step of epoch 0:  52%|█████▏    | 2912/5616 [37:05<33:14,  1.36it/s, gpt_loss=0.0051, lr=2.73e-6]
+
Train step of epoch 0:  52%|█████▏    | 2912/5616 [37:05<33:14,  1.36it/s, gpt_loss=0.0635, lr=2.73e-6]
+
Train step of epoch 0:  52%|█████▏    | 2913/5616 [37:05<31:42,  1.42it/s, gpt_loss=0.0635, lr=2.73e-6]
+
Train step of epoch 0:  52%|█████▏    | 2913/5616 [37:06<31:42,  1.42it/s, gpt_loss=0.0458, lr=2.73e-6]
+
Train step of epoch 0:  52%|█████▏    | 2914/5616 [37:06<35:53,  1.25it/s, gpt_loss=0.0458, lr=2.73e-6]
+
Train step of epoch 0:  52%|█████▏    | 2914/5616 [37:07<35:53,  1.25it/s, gpt_loss=0.02, lr=2.73e-6]  
+
Train step of epoch 0:  52%|█████▏    | 2915/5616 [37:07<35:21,  1.27it/s, gpt_loss=0.02, lr=2.73e-6]
+
Train step of epoch 0:  52%|█████▏    | 2915/5616 [37:08<35:21,  1.27it/s, gpt_loss=0.112, lr=2.72e-6]
+
Train step of epoch 0:  52%|█████▏    | 2916/5616 [37:08<34:58,  1.29it/s, gpt_loss=0.112, lr=2.72e-6]
+
Train step of epoch 0:  52%|█████▏    | 2916/5616 [37:09<34:58,  1.29it/s, gpt_loss=0.00964, lr=2.72e-6]
+
Train step of epoch 0:  52%|█████▏    | 2917/5616 [37:09<33:38,  1.34it/s, gpt_loss=0.00964, lr=2.72e-6]
+
Train step of epoch 0:  52%|█████▏    | 2917/5616 [37:09<33:38,  1.34it/s, gpt_loss=0.0631, lr=2.72e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2918/5616 [37:09<33:56,  1.32it/s, gpt_loss=0.0631, lr=2.72e-6]
+
Train step of epoch 0:  52%|█████▏    | 2918/5616 [37:10<33:56,  1.32it/s, gpt_loss=0.064, lr=2.72e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2919/5616 [37:10<36:49,  1.22it/s, gpt_loss=0.064, lr=2.72e-6]
+
Train step of epoch 0:  52%|█████▏    | 2919/5616 [37:11<36:49,  1.22it/s, gpt_loss=0.0121, lr=2.72e-6]
+
Train step of epoch 0:  52%|█████▏    | 2920/5616 [37:11<37:36,  1.19it/s, gpt_loss=0.0121, lr=2.72e-6]
+
Train step of epoch 0:  52%|█████▏    | 2920/5616 [37:12<37:36,  1.19it/s, gpt_loss=0.128, lr=2.72e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2921/5616 [37:12<35:38,  1.26it/s, gpt_loss=0.128, lr=2.72e-6]
+
Train step of epoch 0:  52%|█████▏    | 2921/5616 [37:13<35:38,  1.26it/s, gpt_loss=0.0214, lr=2.72e-6]
+
Train step of epoch 0:  52%|█████▏    | 2922/5616 [37:13<35:00,  1.28it/s, gpt_loss=0.0214, lr=2.72e-6]
+
Train step of epoch 0:  52%|█████▏    | 2922/5616 [37:13<35:00,  1.28it/s, gpt_loss=0.0121, lr=2.72e-6]
+
Train step of epoch 0:  52%|█████▏    | 2923/5616 [37:13<33:10,  1.35it/s, gpt_loss=0.0121, lr=2.72e-6]
+
Train step of epoch 0:  52%|█████▏    | 2923/5616 [37:14<33:10,  1.35it/s, gpt_loss=0.0313, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2924/5616 [37:14<34:22,  1.30it/s, gpt_loss=0.0313, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2924/5616 [37:15<34:22,  1.30it/s, gpt_loss=0.0634, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2925/5616 [37:15<33:19,  1.35it/s, gpt_loss=0.0634, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2925/5616 [37:15<33:19,  1.35it/s, gpt_loss=0.00632, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2926/5616 [37:15<33:18,  1.35it/s, gpt_loss=0.00632, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2926/5616 [37:16<33:18,  1.35it/s, gpt_loss=0.0465, lr=2.71e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2927/5616 [37:16<31:54,  1.40it/s, gpt_loss=0.0465, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2927/5616 [37:17<31:54,  1.40it/s, gpt_loss=0.00576, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2928/5616 [37:17<32:43,  1.37it/s, gpt_loss=0.00576, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2928/5616 [37:18<32:43,  1.37it/s, gpt_loss=0.0184, lr=2.71e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2929/5616 [37:18<32:39,  1.37it/s, gpt_loss=0.0184, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2929/5616 [37:18<32:39,  1.37it/s, gpt_loss=0.0141, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2930/5616 [37:18<32:40,  1.37it/s, gpt_loss=0.0141, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2930/5616 [37:19<32:40,  1.37it/s, gpt_loss=0.0408, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2931/5616 [37:19<34:14,  1.31it/s, gpt_loss=0.0408, lr=2.71e-6]
+
Train step of epoch 0:  52%|█████▏    | 2931/5616 [37:20<34:14,  1.31it/s, gpt_loss=0.0727, lr=2.7e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2932/5616 [37:20<34:23,  1.30it/s, gpt_loss=0.0727, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2932/5616 [37:21<34:23,  1.30it/s, gpt_loss=0.0187, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2933/5616 [37:21<35:27,  1.26it/s, gpt_loss=0.0187, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2933/5616 [37:22<35:27,  1.26it/s, gpt_loss=0.0741, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2934/5616 [37:22<34:11,  1.31it/s, gpt_loss=0.0741, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2934/5616 [37:22<34:11,  1.31it/s, gpt_loss=0.0302, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2935/5616 [37:22<34:08,  1.31it/s, gpt_loss=0.0302, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2935/5616 [37:23<34:08,  1.31it/s, gpt_loss=0.0185, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2936/5616 [37:23<33:33,  1.33it/s, gpt_loss=0.0185, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2936/5616 [37:24<33:33,  1.33it/s, gpt_loss=0.0971, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2937/5616 [37:24<34:20,  1.30it/s, gpt_loss=0.0971, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2937/5616 [37:25<34:20,  1.30it/s, gpt_loss=0.0247, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2938/5616 [37:25<33:12,  1.34it/s, gpt_loss=0.0247, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2938/5616 [37:25<33:12,  1.34it/s, gpt_loss=0.0536, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2939/5616 [37:25<33:29,  1.33it/s, gpt_loss=0.0536, lr=2.7e-6]
+
Train step of epoch 0:  52%|█████▏    | 2939/5616 [37:26<33:29,  1.33it/s, gpt_loss=0.0333, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2940/5616 [37:26<35:15,  1.27it/s, gpt_loss=0.0333, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2940/5616 [37:27<35:15,  1.27it/s, gpt_loss=0.0145, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2941/5616 [37:27<33:45,  1.32it/s, gpt_loss=0.0145, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2941/5616 [37:28<33:45,  1.32it/s, gpt_loss=0.0336, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2942/5616 [37:28<32:30,  1.37it/s, gpt_loss=0.0336, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2942/5616 [37:28<32:30,  1.37it/s, gpt_loss=0.121, lr=2.69e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2943/5616 [37:28<33:00,  1.35it/s, gpt_loss=0.121, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2943/5616 [37:29<33:00,  1.35it/s, gpt_loss=0.0122, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2944/5616 [37:29<34:27,  1.29it/s, gpt_loss=0.0122, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2944/5616 [37:30<34:27,  1.29it/s, gpt_loss=0.0926, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2945/5616 [37:30<33:00,  1.35it/s, gpt_loss=0.0926, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2945/5616 [37:30<33:00,  1.35it/s, gpt_loss=0.0692, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2946/5616 [37:30<32:02,  1.39it/s, gpt_loss=0.0692, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2946/5616 [37:31<32:02,  1.39it/s, gpt_loss=0.0358, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2947/5616 [37:31<31:53,  1.39it/s, gpt_loss=0.0358, lr=2.69e-6]
+
Train step of epoch 0:  52%|█████▏    | 2947/5616 [37:32<31:53,  1.39it/s, gpt_loss=0.154, lr=2.68e-6] 
+
Train step of epoch 0:  52%|█████▏    | 2948/5616 [37:32<34:54,  1.27it/s, gpt_loss=0.154, lr=2.68e-6]
+
Train step of epoch 0:  52%|█████▏    | 2948/5616 [37:33<34:54,  1.27it/s, gpt_loss=0.0852, lr=2.68e-6]
+
Train step of epoch 0:  53%|█████▎    | 2949/5616 [37:33<33:22,  1.33it/s, gpt_loss=0.0852, lr=2.68e-6]
+
Train step of epoch 0:  53%|█████▎    | 2949/5616 [37:34<33:22,  1.33it/s, gpt_loss=0.0758, lr=2.68e-6]
+
Train step of epoch 0:  53%|█████▎    | 2950/5616 [37:34<32:54,  1.35it/s, gpt_loss=0.0758, lr=2.68e-6]
+
Train step of epoch 0:  53%|█████▎    | 2950/5616 [37:34<32:54,  1.35it/s, gpt_loss=0.00917, lr=2.68e-6]
+
Train step of epoch 0:  53%|█████▎    | 2951/5616 [37:34<31:43,  1.40it/s, gpt_loss=0.00917, lr=2.68e-6]
+
Train step of epoch 0:  53%|█████▎    | 2951/5616 [37:35<31:43,  1.40it/s, gpt_loss=0.112, lr=2.67e-6]  
+
Train step of epoch 0:  53%|█████▎    | 2952/5616 [37:35<33:08,  1.34it/s, gpt_loss=0.112, lr=2.67e-6]
+
Train step of epoch 0:  53%|█████▎    | 2952/5616 [37:36<33:08,  1.34it/s, gpt_loss=0.0423, lr=2.67e-6]
+
Train step of epoch 0:  53%|█████▎    | 2953/5616 [37:36<34:51,  1.27it/s, gpt_loss=0.0423, lr=2.67e-6]
+
Train step of epoch 0:  53%|█████▎    | 2953/5616 [37:37<34:51,  1.27it/s, gpt_loss=0.0774, lr=2.67e-6]
+
Train step of epoch 0:  53%|█████▎    | 2954/5616 [37:37<33:22,  1.33it/s, gpt_loss=0.0774, lr=2.67e-6]
+
Train step of epoch 0:  53%|█████▎    | 2954/5616 [37:37<33:22,  1.33it/s, gpt_loss=0.0657, lr=2.67e-6]
+
Train step of epoch 0:  53%|█████▎    | 2955/5616 [37:37<32:57,  1.35it/s, gpt_loss=0.0657, lr=2.67e-6]
+
Train step of epoch 0:  53%|█████▎    | 2955/5616 [37:38<32:57,  1.35it/s, gpt_loss=0.038, lr=2.67e-6] 
+
Train step of epoch 0:  53%|█████▎    | 2956/5616 [37:38<34:18,  1.29it/s, gpt_loss=0.038, lr=2.67e-6]
+
Train step of epoch 0:  53%|█████▎    | 2956/5616 [37:39<34:18,  1.29it/s, gpt_loss=0.03, lr=2.67e-6] 
+
Train step of epoch 0:  53%|█████▎    | 2957/5616 [37:39<33:50,  1.31it/s, gpt_loss=0.03, lr=2.67e-6]
+
Train step of epoch 0:  53%|█████▎    | 2957/5616 [37:40<33:50,  1.31it/s, gpt_loss=0.0229, lr=2.67e-6]
+
Train step of epoch 0:  53%|█████▎    | 2958/5616 [37:40<32:35,  1.36it/s, gpt_loss=0.0229, lr=2.67e-6]
+
Train step of epoch 0:  53%|█████▎    | 2958/5616 [37:40<32:35,  1.36it/s, gpt_loss=0.0353, lr=2.67e-6]
+
Train step of epoch 0:  53%|█████▎    | 2959/5616 [37:40<33:33,  1.32it/s, gpt_loss=0.0353, lr=2.67e-6]
+
Train step of epoch 0:  53%|█████▎    | 2959/5616 [37:41<33:33,  1.32it/s, gpt_loss=0.0969, lr=2.66e-6]
+
Train step of epoch 0:  53%|█████▎    | 2960/5616 [37:41<34:32,  1.28it/s, gpt_loss=0.0969, lr=2.66e-6]
+
Train step of epoch 0:  53%|█████▎    | 2960/5616 [37:42<34:32,  1.28it/s, gpt_loss=0.015, lr=2.66e-6] 
+
Train step of epoch 0:  53%|█████▎    | 2961/5616 [37:42<34:02,  1.30it/s, gpt_loss=0.015, lr=2.66e-6]
+
Train step of epoch 0:  53%|█████▎    | 2961/5616 [37:43<34:02,  1.30it/s, gpt_loss=0.00536, lr=2.66e-6]
+
Train step of epoch 0:  53%|█████▎    | 2962/5616 [37:43<32:40,  1.35it/s, gpt_loss=0.00536, lr=2.66e-6]
+
Train step of epoch 0:  53%|█████▎    | 2962/5616 [37:43<32:40,  1.35it/s, gpt_loss=0.00974, lr=2.66e-6]
+
Train step of epoch 0:  53%|█████▎    | 2963/5616 [37:43<33:03,  1.34it/s, gpt_loss=0.00974, lr=2.66e-6]
+
Train step of epoch 0:  53%|█████▎    | 2963/5616 [37:44<33:03,  1.34it/s, gpt_loss=0.0317, lr=2.66e-6] 
+
Train step of epoch 0:  53%|█████▎    | 2964/5616 [37:44<33:14,  1.33it/s, gpt_loss=0.0317, lr=2.66e-6]
+
Train step of epoch 0:  53%|█████▎    | 2964/5616 [37:45<33:14,  1.33it/s, gpt_loss=0.0684, lr=2.66e-6]
+
Train step of epoch 0:  53%|█████▎    | 2965/5616 [37:45<33:11,  1.33it/s, gpt_loss=0.0684, lr=2.66e-6]
+
Train step of epoch 0:  53%|█████▎    | 2965/5616 [37:46<33:11,  1.33it/s, gpt_loss=0.021, lr=2.66e-6] 
+
Train step of epoch 0:  53%|█████▎    | 2966/5616 [37:46<32:12,  1.37it/s, gpt_loss=0.021, lr=2.66e-6]
+
Train step of epoch 0:  53%|█████▎    | 2966/5616 [37:46<32:12,  1.37it/s, gpt_loss=0.0512, lr=2.66e-6]
+
Train step of epoch 0:  53%|█████▎    | 2967/5616 [37:46<32:44,  1.35it/s, gpt_loss=0.0512, lr=2.66e-6]
+
Train step of epoch 0:  53%|█████▎    | 2967/5616 [37:47<32:44,  1.35it/s, gpt_loss=0.0122, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2968/5616 [37:47<33:51,  1.30it/s, gpt_loss=0.0122, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2968/5616 [37:48<33:51,  1.30it/s, gpt_loss=0.0575, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2969/5616 [37:48<33:15,  1.33it/s, gpt_loss=0.0575, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2969/5616 [37:49<33:15,  1.33it/s, gpt_loss=0.0254, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2970/5616 [37:49<32:07,  1.37it/s, gpt_loss=0.0254, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2970/5616 [37:49<32:07,  1.37it/s, gpt_loss=0.0599, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2971/5616 [37:49<31:32,  1.40it/s, gpt_loss=0.0599, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2971/5616 [37:50<31:32,  1.40it/s, gpt_loss=0.0521, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2972/5616 [37:50<32:59,  1.34it/s, gpt_loss=0.0521, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2972/5616 [37:51<32:59,  1.34it/s, gpt_loss=0.012, lr=2.65e-6] 
+
Train step of epoch 0:  53%|█████▎    | 2973/5616 [37:51<32:07,  1.37it/s, gpt_loss=0.012, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2973/5616 [37:52<32:07,  1.37it/s, gpt_loss=0.0527, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2974/5616 [37:52<33:25,  1.32it/s, gpt_loss=0.0527, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2974/5616 [37:52<33:25,  1.32it/s, gpt_loss=0.0331, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2975/5616 [37:52<31:57,  1.38it/s, gpt_loss=0.0331, lr=2.65e-6]
+
Train step of epoch 0:  53%|█████▎    | 2975/5616 [37:53<31:57,  1.38it/s, gpt_loss=0.0382, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2976/5616 [37:53<33:17,  1.32it/s, gpt_loss=0.0382, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2976/5616 [37:54<33:17,  1.32it/s, gpt_loss=0.0179, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2977/5616 [37:54<32:47,  1.34it/s, gpt_loss=0.0179, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2977/5616 [37:55<32:47,  1.34it/s, gpt_loss=0.0832, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2978/5616 [37:55<33:45,  1.30it/s, gpt_loss=0.0832, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2978/5616 [37:55<33:45,  1.30it/s, gpt_loss=0.00886, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2979/5616 [37:55<32:16,  1.36it/s, gpt_loss=0.00886, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2979/5616 [37:56<32:16,  1.36it/s, gpt_loss=0.0332, lr=2.64e-6] 
+
Train step of epoch 0:  53%|█████▎    | 2980/5616 [37:56<33:06,  1.33it/s, gpt_loss=0.0332, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2980/5616 [37:57<33:06,  1.33it/s, gpt_loss=0.0487, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2981/5616 [37:57<32:26,  1.35it/s, gpt_loss=0.0487, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2981/5616 [37:57<32:26,  1.35it/s, gpt_loss=0.0571, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2982/5616 [37:57<31:38,  1.39it/s, gpt_loss=0.0571, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2982/5616 [37:58<31:38,  1.39it/s, gpt_loss=0.0233, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2983/5616 [37:58<30:37,  1.43it/s, gpt_loss=0.0233, lr=2.64e-6]
+
Train step of epoch 0:  53%|█████▎    | 2983/5616 [37:59<30:37,  1.43it/s, gpt_loss=0.0208, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2984/5616 [37:59<32:39,  1.34it/s, gpt_loss=0.0208, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2984/5616 [38:00<32:39,  1.34it/s, gpt_loss=0.0711, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2985/5616 [38:00<31:46,  1.38it/s, gpt_loss=0.0711, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2985/5616 [38:00<31:46,  1.38it/s, gpt_loss=0.0128, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2986/5616 [38:00<31:50,  1.38it/s, gpt_loss=0.0128, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2986/5616 [38:01<31:50,  1.38it/s, gpt_loss=0.0472, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2987/5616 [38:01<32:13,  1.36it/s, gpt_loss=0.0472, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2987/5616 [38:02<32:13,  1.36it/s, gpt_loss=0.0349, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2988/5616 [38:02<33:19,  1.31it/s, gpt_loss=0.0349, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2988/5616 [38:03<33:19,  1.31it/s, gpt_loss=0.0479, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2989/5616 [38:03<39:46,  1.10it/s, gpt_loss=0.0479, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2989/5616 [38:04<39:46,  1.10it/s, gpt_loss=0.0148, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2990/5616 [38:04<37:28,  1.17it/s, gpt_loss=0.0148, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2990/5616 [38:05<37:28,  1.17it/s, gpt_loss=0.0492, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2991/5616 [38:05<34:58,  1.25it/s, gpt_loss=0.0492, lr=2.63e-6]
+
Train step of epoch 0:  53%|█████▎    | 2991/5616 [38:05<34:58,  1.25it/s, gpt_loss=0.0998, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2992/5616 [38:05<34:11,  1.28it/s, gpt_loss=0.0998, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2992/5616 [38:06<34:11,  1.28it/s, gpt_loss=0.0441, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2993/5616 [38:06<35:55,  1.22it/s, gpt_loss=0.0441, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2993/5616 [38:07<35:55,  1.22it/s, gpt_loss=0.0475, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2994/5616 [38:07<35:12,  1.24it/s, gpt_loss=0.0475, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2994/5616 [38:08<35:12,  1.24it/s, gpt_loss=0.0538, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2995/5616 [38:08<34:19,  1.27it/s, gpt_loss=0.0538, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2995/5616 [38:08<34:19,  1.27it/s, gpt_loss=0.0717, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2996/5616 [38:08<33:35,  1.30it/s, gpt_loss=0.0717, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2996/5616 [38:09<33:35,  1.30it/s, gpt_loss=0.0297, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2997/5616 [38:09<34:34,  1.26it/s, gpt_loss=0.0297, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2997/5616 [38:10<34:34,  1.26it/s, gpt_loss=0.029, lr=2.62e-6] 
+
Train step of epoch 0:  53%|█████▎    | 2998/5616 [38:10<33:22,  1.31it/s, gpt_loss=0.029, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2998/5616 [38:11<33:22,  1.31it/s, gpt_loss=0.025, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2999/5616 [38:11<33:10,  1.31it/s, gpt_loss=0.025, lr=2.62e-6]
+
Train step of epoch 0:  53%|█████▎    | 2999/5616 [38:11<33:10,  1.31it/s, gpt_loss=0.00253, lr=2.61e-6]
+
Train step of epoch 0:  53%|█████▎    | 3000/5616 [38:11<33:21,  1.31it/s, gpt_loss=0.00253, lr=2.61e-6]
+
Train step of epoch 0:  53%|█████▎    | 3000/5616 [38:12<33:21,  1.31it/s, gpt_loss=0.0102, lr=2.61e-6] 
+
Train step of epoch 0:  53%|█████▎    | 3001/5616 [38:12<31:41,  1.38it/s, gpt_loss=0.0102, lr=2.61e-6]
+
Train step of epoch 0:  53%|█████▎    | 3001/5616 [38:13<31:41,  1.38it/s, gpt_loss=0.0933, lr=2.61e-6]
+
Train step of epoch 0:  53%|█████▎    | 3002/5616 [38:13<31:02,  1.40it/s, gpt_loss=0.0933, lr=2.61e-6]
+
Train step of epoch 0:  53%|█████▎    | 3002/5616 [38:13<31:02,  1.40it/s, gpt_loss=0.0114, lr=2.61e-6]
+
Train step of epoch 0:  53%|█████▎    | 3003/5616 [38:13<30:27,  1.43it/s, gpt_loss=0.0114, lr=2.61e-6]
+
Train step of epoch 0:  53%|█████▎    | 3003/5616 [38:14<30:27,  1.43it/s, gpt_loss=0.106, lr=2.61e-6] 
+
Train step of epoch 0:  53%|█████▎    | 3004/5616 [38:14<31:21,  1.39it/s, gpt_loss=0.106, lr=2.61e-6]
+
Train step of epoch 0:  53%|█████▎    | 3004/5616 [38:15<31:21,  1.39it/s, gpt_loss=0.0147, lr=2.61e-6]
+
Train step of epoch 0:  54%|█████▎    | 3005/5616 [38:15<31:45,  1.37it/s, gpt_loss=0.0147, lr=2.61e-6]
+
Train step of epoch 0:  54%|█████▎    | 3005/5616 [38:16<31:45,  1.37it/s, gpt_loss=0.0513, lr=2.61e-6]
+
Train step of epoch 0:  54%|█████▎    | 3006/5616 [38:16<30:48,  1.41it/s, gpt_loss=0.0513, lr=2.61e-6]
+
Train step of epoch 0:  54%|█████▎    | 3006/5616 [38:16<30:48,  1.41it/s, gpt_loss=0.0713, lr=2.61e-6]
+
Train step of epoch 0:  54%|█████▎    | 3007/5616 [38:16<30:12,  1.44it/s, gpt_loss=0.0713, lr=2.61e-6]
+
Train step of epoch 0:  54%|█████▎    | 3007/5616 [38:17<30:12,  1.44it/s, gpt_loss=0.0394, lr=2.6e-6] 
+
Train step of epoch 0:  54%|█████▎    | 3008/5616 [38:17<30:58,  1.40it/s, gpt_loss=0.0394, lr=2.6e-6]
+
Train step of epoch 0:  54%|█████▎    | 3008/5616 [38:18<30:58,  1.40it/s, gpt_loss=0.0673, lr=2.6e-6]
+
Train step of epoch 0:  54%|█████▎    | 3009/5616 [38:18<29:56,  1.45it/s, gpt_loss=0.0673, lr=2.6e-6]
+
Train step of epoch 0:  54%|█████▎    | 3009/5616 [38:18<29:56,  1.45it/s, gpt_loss=0.0309, lr=2.6e-6]
+
Train step of epoch 0:  54%|█████▎    | 3010/5616 [38:18<29:41,  1.46it/s, gpt_loss=0.0309, lr=2.6e-6]
+
Train step of epoch 0:  54%|█████▎    | 3010/5616 [38:19<29:41,  1.46it/s, gpt_loss=0.0512, lr=2.6e-6]
+
Train step of epoch 0:  54%|█████▎    | 3011/5616 [38:19<30:10,  1.44it/s, gpt_loss=0.0512, lr=2.6e-6]
+
Train step of epoch 0:  54%|█████▎    | 3011/5616 [38:20<30:10,  1.44it/s, gpt_loss=0.0179, lr=2.6e-6]
+
Train step of epoch 0:  54%|█████▎    | 3012/5616 [38:20<31:52,  1.36it/s, gpt_loss=0.0179, lr=2.6e-6]
+
Train step of epoch 0:  54%|█████▎    | 3012/5616 [38:21<31:52,  1.36it/s, gpt_loss=0.0814, lr=2.6e-6]
+
Train step of epoch 0:  54%|█████▎    | 3013/5616 [38:21<32:06,  1.35it/s, gpt_loss=0.0814, lr=2.6e-6]
+
Train step of epoch 0:  54%|█████▎    | 3013/5616 [38:21<32:06,  1.35it/s, gpt_loss=0.012, lr=2.6e-6] 
+
Train step of epoch 0:  54%|█████▎    | 3014/5616 [38:21<32:20,  1.34it/s, gpt_loss=0.012, lr=2.6e-6]
+
Train step of epoch 0:  54%|█████▎    | 3014/5616 [38:22<32:20,  1.34it/s, gpt_loss=0.18, lr=2.6e-6] 
+
Train step of epoch 0:  54%|█████▎    | 3015/5616 [38:22<32:34,  1.33it/s, gpt_loss=0.18, lr=2.6e-6]
+
Train step of epoch 0:  54%|█████▎    | 3015/5616 [38:23<32:34,  1.33it/s, gpt_loss=0.0324, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▎    | 3016/5616 [38:23<34:18,  1.26it/s, gpt_loss=0.0324, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▎    | 3016/5616 [38:24<34:18,  1.26it/s, gpt_loss=0.0105, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▎    | 3017/5616 [38:24<36:33,  1.18it/s, gpt_loss=0.0105, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▎    | 3017/5616 [38:25<36:33,  1.18it/s, gpt_loss=0.0413, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▎    | 3018/5616 [38:25<33:48,  1.28it/s, gpt_loss=0.0413, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▎    | 3018/5616 [38:25<33:48,  1.28it/s, gpt_loss=0.0214, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▍    | 3019/5616 [38:25<32:46,  1.32it/s, gpt_loss=0.0214, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▍    | 3019/5616 [38:26<32:46,  1.32it/s, gpt_loss=0.0371, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▍    | 3020/5616 [38:26<33:34,  1.29it/s, gpt_loss=0.0371, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▍    | 3020/5616 [38:27<33:34,  1.29it/s, gpt_loss=0.0271, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▍    | 3021/5616 [38:27<32:15,  1.34it/s, gpt_loss=0.0271, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▍    | 3021/5616 [38:28<32:15,  1.34it/s, gpt_loss=0.0518, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▍    | 3022/5616 [38:28<32:17,  1.34it/s, gpt_loss=0.0518, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▍    | 3022/5616 [38:28<32:17,  1.34it/s, gpt_loss=0.0362, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▍    | 3023/5616 [38:28<30:47,  1.40it/s, gpt_loss=0.0362, lr=2.59e-6]
+
Train step of epoch 0:  54%|█████▍    | 3023/5616 [38:29<30:47,  1.40it/s, gpt_loss=0.0828, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3024/5616 [38:29<31:58,  1.35it/s, gpt_loss=0.0828, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3024/5616 [38:30<31:58,  1.35it/s, gpt_loss=0.0158, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3025/5616 [38:30<31:01,  1.39it/s, gpt_loss=0.0158, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3025/5616 [38:31<31:01,  1.39it/s, gpt_loss=0.00951, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3026/5616 [38:31<32:19,  1.34it/s, gpt_loss=0.00951, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3026/5616 [38:32<32:19,  1.34it/s, gpt_loss=0.0468, lr=2.58e-6] 
+
Train step of epoch 0:  54%|█████▍    | 3027/5616 [38:32<36:16,  1.19it/s, gpt_loss=0.0468, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3027/5616 [38:32<36:16,  1.19it/s, gpt_loss=0.00599, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3028/5616 [38:32<34:58,  1.23it/s, gpt_loss=0.00599, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3028/5616 [38:33<34:58,  1.23it/s, gpt_loss=0.105, lr=2.58e-6]  
+
Train step of epoch 0:  54%|█████▍    | 3029/5616 [38:33<33:55,  1.27it/s, gpt_loss=0.105, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3029/5616 [38:34<33:55,  1.27it/s, gpt_loss=0.00974, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3030/5616 [38:34<32:35,  1.32it/s, gpt_loss=0.00974, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3030/5616 [38:35<32:35,  1.32it/s, gpt_loss=0.00933, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3031/5616 [38:35<32:43,  1.32it/s, gpt_loss=0.00933, lr=2.58e-6]
+
Train step of epoch 0:  54%|█████▍    | 3031/5616 [38:36<32:43,  1.32it/s, gpt_loss=0.00914, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3032/5616 [38:36<39:16,  1.10it/s, gpt_loss=0.00914, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3032/5616 [38:36<39:16,  1.10it/s, gpt_loss=0.0558, lr=2.57e-6] 
+
Train step of epoch 0:  54%|█████▍    | 3033/5616 [38:36<36:05,  1.19it/s, gpt_loss=0.0558, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3033/5616 [38:37<36:05,  1.19it/s, gpt_loss=0.0319, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3034/5616 [38:37<33:50,  1.27it/s, gpt_loss=0.0319, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3034/5616 [38:38<33:50,  1.27it/s, gpt_loss=0.0339, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3035/5616 [38:38<32:43,  1.31it/s, gpt_loss=0.0339, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3035/5616 [38:39<32:43,  1.31it/s, gpt_loss=0.0756, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3036/5616 [38:39<32:43,  1.31it/s, gpt_loss=0.0756, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3036/5616 [38:39<32:43,  1.31it/s, gpt_loss=0.0674, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3037/5616 [38:39<32:43,  1.31it/s, gpt_loss=0.0674, lr=2.57e-6]
+
Train step of epoch 0:  54%|���████▍    | 3037/5616 [38:40<32:43,  1.31it/s, gpt_loss=0.0139, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3038/5616 [38:40<33:09,  1.30it/s, gpt_loss=0.0139, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3038/5616 [38:41<33:09,  1.30it/s, gpt_loss=0.0849, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3039/5616 [38:41<32:18,  1.33it/s, gpt_loss=0.0849, lr=2.57e-6]
+
Train step of epoch 0:  54%|█████▍    | 3039/5616 [38:42<32:18,  1.33it/s, gpt_loss=0.127, lr=2.56e-6] 
+
Train step of epoch 0:  54%|█████▍    | 3040/5616 [38:42<32:54,  1.30it/s, gpt_loss=0.127, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3040/5616 [38:42<32:54,  1.30it/s, gpt_loss=0.0212, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3041/5616 [38:42<31:46,  1.35it/s, gpt_loss=0.0212, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3041/5616 [38:44<31:46,  1.35it/s, gpt_loss=0.0493, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3042/5616 [38:44<40:09,  1.07it/s, gpt_loss=0.0493, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3042/5616 [38:44<40:09,  1.07it/s, gpt_loss=0.0122, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3043/5616 [38:44<37:03,  1.16it/s, gpt_loss=0.0122, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3043/5616 [38:45<37:03,  1.16it/s, gpt_loss=0.0163, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3044/5616 [38:45<38:26,  1.11it/s, gpt_loss=0.0163, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3044/5616 [38:46<38:26,  1.11it/s, gpt_loss=0.079, lr=2.56e-6] 
+
Train step of epoch 0:  54%|█████▍    | 3045/5616 [38:46<36:13,  1.18it/s, gpt_loss=0.079, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3045/5616 [38:47<36:13,  1.18it/s, gpt_loss=0.0151, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3046/5616 [38:47<33:34,  1.28it/s, gpt_loss=0.0151, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3046/5616 [38:47<33:34,  1.28it/s, gpt_loss=0.00446, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3047/5616 [38:47<32:49,  1.30it/s, gpt_loss=0.00446, lr=2.56e-6]
+
Train step of epoch 0:  54%|█████▍    | 3047/5616 [38:48<32:49,  1.30it/s, gpt_loss=0.00894, lr=2.55e-6]
+
Train step of epoch 0:  54%|█████▍    | 3048/5616 [38:48<32:16,  1.33it/s, gpt_loss=0.00894, lr=2.55e-6]
+
Train step of epoch 0:  54%|█████▍    | 3048/5616 [38:49<32:16,  1.33it/s, gpt_loss=0.0173, lr=2.55e-6] 
+
Train step of epoch 0:  54%|█████▍    | 3049/5616 [38:49<31:37,  1.35it/s, gpt_loss=0.0173, lr=2.55e-6]
+
Train step of epoch 0:  54%|█████▍    | 3049/5616 [38:50<31:37,  1.35it/s, gpt_loss=0.00976, lr=2.55e-6]
+
Train step of epoch 0:  54%|█████▍    | 3050/5616 [38:50<32:26,  1.32it/s, gpt_loss=0.00976, lr=2.55e-6]
+
Train step of epoch 0:  54%|█████▍    | 3050/5616 [38:51<32:26,  1.32it/s, gpt_loss=0.0157, lr=2.55e-6] 
+
Train step of epoch 0:  54%|█████▍    | 3051/5616 [38:51<34:07,  1.25it/s, gpt_loss=0.0157, lr=2.55e-6]
+
Train step of epoch 0:  54%|█████▍    | 3051/5616 [38:51<34:07,  1.25it/s, gpt_loss=0.0213, lr=2.55e-6]
+
Train step of epoch 0:  54%|█████▍    | 3052/5616 [38:51<34:52,  1.23it/s, gpt_loss=0.0213, lr=2.55e-6]
+
Train step of epoch 0:  54%|█████▍    | 3052/5616 [38:52<34:52,  1.23it/s, gpt_loss=0.0393, lr=2.55e-6]
+
Train step of epoch 0:  54%|█████▍    | 3053/5616 [38:52<33:23,  1.28it/s, gpt_loss=0.0393, lr=2.55e-6]
+
Train step of epoch 0:  54%|█████▍    | 3053/5616 [38:53<33:23,  1.28it/s, gpt_loss=0.113, lr=2.55e-6] 
+
Train step of epoch 0:  54%|█████▍    | 3054/5616 [38:53<33:03,  1.29it/s, gpt_loss=0.113, lr=2.55e-6]
+
Train step of epoch 0:  54%|█████▍    | 3054/5616 [38:54<33:03,  1.29it/s, gpt_loss=0.154, lr=2.55e-6]
+
Train step of epoch 0:  54%|█████▍    | 3055/5616 [38:54<31:58,  1.34it/s, gpt_loss=0.154, lr=2.55e-6]
+
Train step of epoch 0:  54%|█████▍    | 3055/5616 [38:54<31:58,  1.34it/s, gpt_loss=0.0809, lr=2.54e-6]
+
Train step of epoch 0:  54%|█████▍    | 3056/5616 [38:54<32:52,  1.30it/s, gpt_loss=0.0809, lr=2.54e-6]
+
Train step of epoch 0:  54%|█████▍    | 3056/5616 [38:55<32:52,  1.30it/s, gpt_loss=0.0335, lr=2.54e-6]
+
Train step of epoch 0:  54%|█████▍    | 3057/5616 [38:55<32:52,  1.30it/s, gpt_loss=0.0335, lr=2.54e-6]
+
Train step of epoch 0:  54%|█████▍    | 3057/5616 [38:56<32:52,  1.30it/s, gpt_loss=0.031, lr=2.54e-6] 
+
Train step of epoch 0:  54%|█████▍    | 3058/5616 [38:56<33:24,  1.28it/s, gpt_loss=0.031, lr=2.54e-6]
+
Train step of epoch 0:  54%|█████▍    | 3058/5616 [38:57<33:24,  1.28it/s, gpt_loss=0.0345, lr=2.54e-6]
+
Train step of epoch 0:  54%|█████▍    | 3059/5616 [38:57<32:17,  1.32it/s, gpt_loss=0.0345, lr=2.54e-6]
+
Train step of epoch 0:  54%|█████▍    | 3059/5616 [38:57<32:17,  1.32it/s, gpt_loss=0.00821, lr=2.53e-6]
+
Train step of epoch 0:  54%|█████▍    | 3060/5616 [38:57<31:54,  1.33it/s, gpt_loss=0.00821, lr=2.53e-6]
+
Train step of epoch 0:  54%|█████▍    | 3060/5616 [38:58<31:54,  1.33it/s, gpt_loss=0.0189, lr=2.53e-6] 
+
Train step of epoch 0:  55%|█████▍    | 3061/5616 [38:58<30:40,  1.39it/s, gpt_loss=0.0189, lr=2.53e-6]
+
Train step of epoch 0:  55%|█████▍    | 3061/5616 [38:59<30:40,  1.39it/s, gpt_loss=0.0314, lr=2.53e-6]
+
Train step of epoch 0:  55%|█████▍    | 3062/5616 [38:59<30:18,  1.40it/s, gpt_loss=0.0314, lr=2.53e-6]
+
Train step of epoch 0:  55%|█████▍    | 3062/5616 [38:59<30:18,  1.40it/s, gpt_loss=0.0476, lr=2.53e-6]
+
Train step of epoch 0:  55%|█████▍    | 3063/5616 [38:59<29:51,  1.43it/s, gpt_loss=0.0476, lr=2.53e-6]
+
Train step of epoch 0:  55%|█████▍    | 3063/5616 [39:00<29:51,  1.43it/s, gpt_loss=0.0289, lr=2.53e-6]
+
Train step of epoch 0:  55%|█████▍    | 3064/5616 [39:00<30:43,  1.38it/s, gpt_loss=0.0289, lr=2.53e-6]
+
Train step of epoch 0:  55%|█████▍    | 3064/5616 [39:01<30:43,  1.38it/s, gpt_loss=0.0578, lr=2.53e-6]
+
Train step of epoch 0:  55%|█████▍    | 3065/5616 [39:01<31:23,  1.35it/s, gpt_loss=0.0578, lr=2.53e-6]
+
Train step of epoch 0:  55%|█████▍    | 3065/5616 [39:02<31:23,  1.35it/s, gpt_loss=0.0116, lr=2.53e-6]
+
Train step of epoch 0:  55%|█████▍    | 3066/5616 [39:02<30:46,  1.38it/s, gpt_loss=0.0116, lr=2.53e-6]
+
Train step of epoch 0:  55%|█████▍    | 3066/5616 [39:02<30:46,  1.38it/s, gpt_loss=0.0211, lr=2.53e-6]
+
Train step of epoch 0:  55%|█████▍    | 3067/5616 [39:02<30:31,  1.39it/s, gpt_loss=0.0211, lr=2.53e-6]
+
Train step of epoch 0:  55%|█████▍    | 3067/5616 [39:03<30:31,  1.39it/s, gpt_loss=0.00486, lr=2.52e-6]
+
Train step of epoch 0:  55%|█████▍    | 3068/5616 [39:03<30:56,  1.37it/s, gpt_loss=0.00486, lr=2.52e-6]
+
Train step of epoch 0:  55%|█████▍    | 3068/5616 [39:04<30:56,  1.37it/s, gpt_loss=0.0632, lr=2.52e-6] 
+
Train step of epoch 0:  55%|█████▍    | 3069/5616 [39:04<30:38,  1.39it/s, gpt_loss=0.0632, lr=2.52e-6]
+
Train step of epoch 0:  55%|█████▍    | 3069/5616 [39:05<30:38,  1.39it/s, gpt_loss=0.033, lr=2.52e-6] 
+
Train step of epoch 0:  55%|█████▍    | 3070/5616 [39:05<30:32,  1.39it/s, gpt_loss=0.033, lr=2.52e-6]
+
Train step of epoch 0:  55%|█████▍    | 3070/5616 [39:05<30:32,  1.39it/s, gpt_loss=0.0423, lr=2.52e-6]
+
Train step of epoch 0:  55%|█████▍    | 3071/5616 [39:05<29:30,  1.44it/s, gpt_loss=0.0423, lr=2.52e-6]
+
Train step of epoch 0:  55%|█████▍    | 3071/5616 [39:06<29:30,  1.44it/s, gpt_loss=0.0149, lr=2.52e-6]
+
Train step of epoch 0:  55%|█████▍    | 3072/5616 [39:06<30:45,  1.38it/s, gpt_loss=0.0149, lr=2.52e-6]
+
Train step of epoch 0:  55%|█████▍    | 3072/5616 [39:07<30:45,  1.38it/s, gpt_loss=0.00942, lr=2.52e-6]
+
Train step of epoch 0:  55%|█████▍    | 3073/5616 [39:07<29:48,  1.42it/s, gpt_loss=0.00942, lr=2.52e-6]
+
Train step of epoch 0:  55%|█████▍    | 3073/5616 [39:07<29:48,  1.42it/s, gpt_loss=0.0265, lr=2.52e-6] 
+
Train step of epoch 0:  55%|█████▍    | 3074/5616 [39:07<29:45,  1.42it/s, gpt_loss=0.0265, lr=2.52e-6]
+
Train step of epoch 0:  55%|█████▍    | 3074/5616 [39:08<29:45,  1.42it/s, gpt_loss=0.0114, lr=2.52e-6]
+
Train step of epoch 0:  55%|█████▍    | 3075/5616 [39:08<29:46,  1.42it/s, gpt_loss=0.0114, lr=2.52e-6]
+
Train step of epoch 0:  55%|█████▍    | 3075/5616 [39:09<29:46,  1.42it/s, gpt_loss=0.0248, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3076/5616 [39:09<30:24,  1.39it/s, gpt_loss=0.0248, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3076/5616 [39:10<30:24,  1.39it/s, gpt_loss=0.00617, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3077/5616 [39:10<30:35,  1.38it/s, gpt_loss=0.00617, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3077/5616 [39:10<30:35,  1.38it/s, gpt_loss=0.0399, lr=2.51e-6] 
+
Train step of epoch 0:  55%|█████▍    | 3078/5616 [39:10<30:34,  1.38it/s, gpt_loss=0.0399, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3078/5616 [39:11<30:34,  1.38it/s, gpt_loss=0.0227, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3079/5616 [39:11<29:17,  1.44it/s, gpt_loss=0.0227, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3079/5616 [39:12<29:17,  1.44it/s, gpt_loss=0.0654, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3080/5616 [39:12<31:52,  1.33it/s, gpt_loss=0.0654, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3080/5616 [39:12<31:52,  1.33it/s, gpt_loss=0.0138, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3081/5616 [39:12<30:53,  1.37it/s, gpt_loss=0.0138, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3081/5616 [39:13<30:53,  1.37it/s, gpt_loss=0.0131, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3082/5616 [39:13<30:31,  1.38it/s, gpt_loss=0.0131, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3082/5616 [39:14<30:31,  1.38it/s, gpt_loss=0.0387, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3083/5616 [39:14<30:07,  1.40it/s, gpt_loss=0.0387, lr=2.51e-6]
+
Train step of epoch 0:  55%|█████▍    | 3083/5616 [39:15<30:07,  1.40it/s, gpt_loss=0.0272, lr=2.5e-6] 
+
Train step of epoch 0:  55%|█████▍    | 3084/5616 [39:15<32:26,  1.30it/s, gpt_loss=0.0272, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▍    | 3084/5616 [39:16<32:26,  1.30it/s, gpt_loss=0.0648, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▍    | 3085/5616 [39:16<32:46,  1.29it/s, gpt_loss=0.0648, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▍    | 3085/5616 [39:16<32:46,  1.29it/s, gpt_loss=0.0281, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▍    | 3086/5616 [39:16<32:38,  1.29it/s, gpt_loss=0.0281, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▍    | 3086/5616 [39:17<32:38,  1.29it/s, gpt_loss=0.00485, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▍    | 3087/5616 [39:17<31:35,  1.33it/s, gpt_loss=0.00485, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▍    | 3087/5616 [39:18<31:35,  1.33it/s, gpt_loss=0.0467, lr=2.5e-6] 
+
Train step of epoch 0:  55%|█████▍    | 3088/5616 [39:18<31:56,  1.32it/s, gpt_loss=0.0467, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▍    | 3088/5616 [39:18<31:56,  1.32it/s, gpt_loss=0.0728, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▌    | 3089/5616 [39:18<30:28,  1.38it/s, gpt_loss=0.0728, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▌    | 3089/5616 [39:19<30:28,  1.38it/s, gpt_loss=0.0165, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▌    | 3090/5616 [39:19<30:44,  1.37it/s, gpt_loss=0.0165, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▌    | 3090/5616 [39:20<30:44,  1.37it/s, gpt_loss=0.0725, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▌    | 3091/5616 [39:20<30:23,  1.38it/s, gpt_loss=0.0725, lr=2.5e-6]
+
Train step of epoch 0:  55%|█████▌    | 3091/5616 [39:21<30:23,  1.38it/s, gpt_loss=0.0582, lr=2.49e-6]
+
Train step of epoch 0:  55%|█████▌    | 3092/5616 [39:21<31:28,  1.34it/s, gpt_loss=0.0582, lr=2.49e-6]
+
Train step of epoch 0:  55%|█████▌    | 3092/5616 [39:21<31:28,  1.34it/s, gpt_loss=0.0529, lr=2.49e-6]
+
Train step of epoch 0:  55%|█████▌    | 3093/5616 [39:21<30:35,  1.37it/s, gpt_loss=0.0529, lr=2.49e-6]
+
Train step of epoch 0:  55%|█████▌    | 3093/5616 [39:22<30:35,  1.37it/s, gpt_loss=0.0334, lr=2.49e-6]
+
Train step of epoch 0:  55%|█████▌    | 3094/5616 [39:22<31:02,  1.35it/s, gpt_loss=0.0334, lr=2.49e-6]
+
Train step of epoch 0:  55%|█████▌    | 3094/5616 [39:23<31:02,  1.35it/s, gpt_loss=0.021, lr=2.49e-6] 
+
Train step of epoch 0:  55%|█████▌    | 3095/5616 [39:23<30:31,  1.38it/s, gpt_loss=0.021, lr=2.49e-6]
+
Train step of epoch 0:  55%|█████▌    | 3095/5616 [39:24<30:31,  1.38it/s, gpt_loss=0.0102, lr=2.49e-6]
+
Train step of epoch 0:  55%|█████▌    | 3096/5616 [39:24<31:13,  1.34it/s, gpt_loss=0.0102, lr=2.49e-6]
+
Train step of epoch 0:  55%|█████▌    | 3096/5616 [39:24<31:13,  1.34it/s, gpt_loss=0.00681, lr=2.49e-6]
+
Train step of epoch 0:  55%|█████▌    | 3097/5616 [39:24<31:01,  1.35it/s, gpt_loss=0.00681, lr=2.49e-6]
+
Train step of epoch 0:  55%|█████▌    | 3097/5616 [39:25<31:01,  1.35it/s, gpt_loss=0.0171, lr=2.49e-6] 
+
Train step of epoch 0:  55%|█████▌    | 3098/5616 [39:25<31:46,  1.32it/s, gpt_loss=0.0171, lr=2.49e-6]
+
Train step of epoch 0:  55%|█████▌    | 3098/5616 [39:26<31:46,  1.32it/s, gpt_loss=0.054, lr=2.49e-6] 
+
Train step of epoch 0:  55%|█████▌    | 3099/5616 [39:26<32:09,  1.30it/s, gpt_loss=0.054, lr=2.49e-6]
+
Train step of epoch 0:  55%|█████▌    | 3099/5616 [39:27<32:09,  1.30it/s, gpt_loss=0.0942, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3100/5616 [39:27<32:52,  1.28it/s, gpt_loss=0.0942, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3100/5616 [39:28<32:52,  1.28it/s, gpt_loss=0.108, lr=2.48e-6] 
+
Train step of epoch 0:  55%|█████▌    | 3101/5616 [39:28<32:41,  1.28it/s, gpt_loss=0.108, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3101/5616 [39:28<32:41,  1.28it/s, gpt_loss=0.0867, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3102/5616 [39:28<32:33,  1.29it/s, gpt_loss=0.0867, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3102/5616 [39:29<32:33,  1.29it/s, gpt_loss=0.0635, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3103/5616 [39:29<31:11,  1.34it/s, gpt_loss=0.0635, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3103/5616 [39:30<31:11,  1.34it/s, gpt_loss=0.0427, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3104/5616 [39:30<31:21,  1.34it/s, gpt_loss=0.0427, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3104/5616 [39:30<31:21,  1.34it/s, gpt_loss=0.0044, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3105/5616 [39:30<31:11,  1.34it/s, gpt_loss=0.0044, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3105/5616 [39:31<31:11,  1.34it/s, gpt_loss=0.0209, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3106/5616 [39:31<32:29,  1.29it/s, gpt_loss=0.0209, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3106/5616 [39:32<32:29,  1.29it/s, gpt_loss=0.00599, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3107/5616 [39:32<32:01,  1.31it/s, gpt_loss=0.00599, lr=2.48e-6]
+
Train step of epoch 0:  55%|█████▌    | 3107/5616 [39:33<32:01,  1.31it/s, gpt_loss=0.018, lr=2.47e-6]  
+
Train step of epoch 0:  55%|█████▌    | 3108/5616 [39:33<32:19,  1.29it/s, gpt_loss=0.018, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3108/5616 [39:34<32:19,  1.29it/s, gpt_loss=0.0166, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3109/5616 [39:34<30:59,  1.35it/s, gpt_loss=0.0166, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3109/5616 [39:34<30:59,  1.35it/s, gpt_loss=0.0341, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3110/5616 [39:34<30:04,  1.39it/s, gpt_loss=0.0341, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3110/5616 [39:35<30:04,  1.39it/s, gpt_loss=0.101, lr=2.47e-6] 
+
Train step of epoch 0:  55%|█████▌    | 3111/5616 [39:35<31:42,  1.32it/s, gpt_loss=0.101, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3111/5616 [39:36<31:42,  1.32it/s, gpt_loss=0.0162, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3112/5616 [39:36<36:17,  1.15it/s, gpt_loss=0.0162, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3112/5616 [39:37<36:17,  1.15it/s, gpt_loss=0.0222, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3113/5616 [39:37<34:25,  1.21it/s, gpt_loss=0.0222, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3113/5616 [39:38<34:25,  1.21it/s, gpt_loss=0.0664, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3114/5616 [39:38<32:47,  1.27it/s, gpt_loss=0.0664, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3114/5616 [39:38<32:47,  1.27it/s, gpt_loss=0.0101, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3115/5616 [39:38<32:06,  1.30it/s, gpt_loss=0.0101, lr=2.47e-6]
+
Train step of epoch 0:  55%|█████▌    | 3115/5616 [39:39<32:06,  1.30it/s, gpt_loss=0.0876, lr=2.46e-6]
+
Train step of epoch 0:  55%|█████▌    | 3116/5616 [39:39<33:01,  1.26it/s, gpt_loss=0.0876, lr=2.46e-6]
+
Train step of epoch 0:  55%|█████▌    | 3116/5616 [39:40<33:01,  1.26it/s, gpt_loss=0.0465, lr=2.46e-6]
+
Train step of epoch 0:  56%|█████▌    | 3117/5616 [39:40<31:55,  1.30it/s, gpt_loss=0.0465, lr=2.46e-6]
+
Train step of epoch 0:  56%|█████▌    | 3117/5616 [39:41<31:55,  1.30it/s, gpt_loss=0.113, lr=2.46e-6] 
+
Train step of epoch 0:  56%|█████▌    | 3118/5616 [39:41<32:21,  1.29it/s, gpt_loss=0.113, lr=2.46e-6]
+
Train step of epoch 0:  56%|█████▌    | 3118/5616 [39:41<32:21,  1.29it/s, gpt_loss=0.032, lr=2.46e-6]
+
Train step of epoch 0:  56%|█████▌    | 3119/5616 [39:41<31:24,  1.33it/s, gpt_loss=0.032, lr=2.46e-6]
+
Train step of epoch 0:  56%|█████▌    | 3119/5616 [39:42<31:24,  1.33it/s, gpt_loss=0.0292, lr=2.46e-6]
+
Train step of epoch 0:  56%|█████▌    | 3120/5616 [39:42<31:29,  1.32it/s, gpt_loss=0.0292, lr=2.46e-6]
+
Train step of epoch 0:  56%|█████▌    | 3120/5616 [39:43<31:29,  1.32it/s, gpt_loss=0.0341, lr=2.46e-6]
+
Train step of epoch 0:  56%|█████▌    | 3121/5616 [39:43<31:21,  1.33it/s, gpt_loss=0.0341, lr=2.46e-6]
+
Train step of epoch 0:  56%|█████▌    | 3121/5616 [39:44<31:21,  1.33it/s, gpt_loss=0.0768, lr=2.46e-6]
+
Train step of epoch 0:  56%|█████▌    | 3122/5616 [39:44<31:12,  1.33it/s, gpt_loss=0.0768, lr=2.46e-6]
+
Train step of epoch 0:  56%|█████▌    | 3122/5616 [39:44<31:12,  1.33it/s, gpt_loss=0.0195, lr=2.46e-6]
+
Train step of epoch 0:  56%|█████▌    | 3123/5616 [39:44<31:09,  1.33it/s, gpt_loss=0.0195, lr=2.46e-6]
+
Train step of epoch 0:  56%|█████▌    | 3123/5616 [39:45<31:09,  1.33it/s, gpt_loss=0.033, lr=2.45e-6] 
+
Train step of epoch 0:  56%|█████▌    | 3124/5616 [39:45<32:32,  1.28it/s, gpt_loss=0.033, lr=2.45e-6]
+
Train step of epoch 0:  56%|█████▌    | 3124/5616 [39:46<32:32,  1.28it/s, gpt_loss=0.0122, lr=2.45e-6]
+
Train step of epoch 0:  56%|█████▌    | 3125/5616 [39:46<31:49,  1.30it/s, gpt_loss=0.0122, lr=2.45e-6]
+
Train step of epoch 0:  56%|█████▌    | 3125/5616 [39:47<31:49,  1.30it/s, gpt_loss=0.09, lr=2.45e-6]  
+
Train step of epoch 0:  56%|█████▌    | 3126/5616 [39:47<30:11,  1.37it/s, gpt_loss=0.09, lr=2.45e-6]
+
Train step of epoch 0:  56%|█████▌    | 3126/5616 [39:47<30:11,  1.37it/s, gpt_loss=0.0208, lr=2.45e-6]
+
Train step of epoch 0:  56%|█████▌    | 3127/5616 [39:47<29:33,  1.40it/s, gpt_loss=0.0208, lr=2.45e-6]
+
Train step of epoch 0:  56%|█████▌    | 3127/5616 [39:48<29:33,  1.40it/s, gpt_loss=0.0779, lr=2.45e-6]
+
Train step of epoch 0:  56%|█████▌    | 3128/5616 [39:48<31:02,  1.34it/s, gpt_loss=0.0779, lr=2.45e-6]
+
Train step of epoch 0:  56%|█████▌    | 3128/5616 [39:49<31:02,  1.34it/s, gpt_loss=0.0152, lr=2.45e-6]
+
Train step of epoch 0:  56%|█████▌    | 3129/5616 [39:49<30:16,  1.37it/s, gpt_loss=0.0152, lr=2.45e-6]
+
Train step of epoch 0:  56%|█████▌    | 3129/5616 [39:49<30:16,  1.37it/s, gpt_loss=0.017, lr=2.45e-6] 
+
Train step of epoch 0:  56%|█████▌    | 3130/5616 [39:49<29:05,  1.42it/s, gpt_loss=0.017, lr=2.45e-6]
+
Train step of epoch 0:  56%|█████▌    | 3130/5616 [39:50<29:05,  1.42it/s, gpt_loss=0.00926, lr=2.45e-6]
+
Train step of epoch 0:  56%|█████▌    | 3131/5616 [39:50<29:32,  1.40it/s, gpt_loss=0.00926, lr=2.45e-6]
+
Train step of epoch 0:  56%|█████▌    | 3131/5616 [39:51<29:32,  1.40it/s, gpt_loss=0.0234, lr=2.44e-6] 
+
Train step of epoch 0:  56%|█████▌    | 3132/5616 [39:51<30:33,  1.36it/s, gpt_loss=0.0234, lr=2.44e-6]
+
Train step of epoch 0:  56%|█████▌    | 3132/5616 [39:52<30:33,  1.36it/s, gpt_loss=0.0189, lr=2.44e-6]
+
Train step of epoch 0:  56%|█████▌    | 3133/5616 [39:52<30:06,  1.37it/s, gpt_loss=0.0189, lr=2.44e-6]
+
Train step of epoch 0:  56%|█████▌    | 3133/5616 [39:52<30:06,  1.37it/s, gpt_loss=0.0106, lr=2.44e-6]
+
Train step of epoch 0:  56%|█████▌    | 3134/5616 [39:52<31:02,  1.33it/s, gpt_loss=0.0106, lr=2.44e-6]
+
Train step of epoch 0:  56%|█████▌    | 3134/5616 [39:53<31:02,  1.33it/s, gpt_loss=0.0168, lr=2.44e-6]
+
Train step of epoch 0:  56%|█████▌    | 3135/5616 [39:53<30:16,  1.37it/s, gpt_loss=0.0168, lr=2.44e-6]
+
Train step of epoch 0:  56%|█████▌    | 3135/5616 [39:54<30:16,  1.37it/s, gpt_loss=0.089, lr=2.44e-6] 
+
Train step of epoch 0:  56%|█████▌    | 3136/5616 [39:54<30:44,  1.34it/s, gpt_loss=0.089, lr=2.44e-6]
+
Train step of epoch 0:  56%|█████▌    | 3136/5616 [39:55<30:44,  1.34it/s, gpt_loss=0.0125, lr=2.44e-6]
+
Train step of epoch 0:  56%|█████▌    | 3137/5616 [39:55<30:15,  1.37it/s, gpt_loss=0.0125, lr=2.44e-6]
+
Train step of epoch 0:  56%|█████▌    | 3137/5616 [39:55<30:15,  1.37it/s, gpt_loss=0.00747, lr=2.44e-6]
+
Train step of epoch 0:  56%|█████▌    | 3138/5616 [39:55<29:22,  1.41it/s, gpt_loss=0.00747, lr=2.44e-6]
+
Train step of epoch 0:  56%|█████▌    | 3138/5616 [39:56<29:22,  1.41it/s, gpt_loss=0.0962, lr=2.44e-6] 
+
Train step of epoch 0:  56%|█████▌    | 3139/5616 [39:56<29:10,  1.42it/s, gpt_loss=0.0962, lr=2.44e-6]
+
Train step of epoch 0:  56%|█████▌    | 3139/5616 [39:57<29:10,  1.42it/s, gpt_loss=0.0165, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3140/5616 [39:57<36:21,  1.13it/s, gpt_loss=0.0165, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3140/5616 [39:58<36:21,  1.13it/s, gpt_loss=0.0428, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3141/5616 [39:58<33:58,  1.21it/s, gpt_loss=0.0428, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3141/5616 [39:59<33:58,  1.21it/s, gpt_loss=0.0337, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3142/5616 [39:59<32:07,  1.28it/s, gpt_loss=0.0337, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3142/5616 [40:00<32:07,  1.28it/s, gpt_loss=0.0371, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3143/5616 [40:00<34:00,  1.21it/s, gpt_loss=0.0371, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3143/5616 [40:01<34:00,  1.21it/s, gpt_loss=0.0127, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3144/5616 [40:01<35:09,  1.17it/s, gpt_loss=0.0127, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3144/5616 [40:02<35:09,  1.17it/s, gpt_loss=0.0186, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3145/5616 [40:02<37:31,  1.10it/s, gpt_loss=0.0186, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3145/5616 [40:02<37:31,  1.10it/s, gpt_loss=0.0437, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3146/5616 [40:02<34:45,  1.18it/s, gpt_loss=0.0437, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3146/5616 [40:03<34:45,  1.18it/s, gpt_loss=0.0413, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3147/5616 [40:03<33:37,  1.22it/s, gpt_loss=0.0413, lr=2.43e-6]
+
Train step of epoch 0:  56%|█████▌    | 3147/5616 [40:04<33:37,  1.22it/s, gpt_loss=0.129, lr=2.42e-6] 
+
Train step of epoch 0:  56%|█████▌    | 3148/5616 [40:04<32:11,  1.28it/s, gpt_loss=0.129, lr=2.42e-6]
+
Train step of epoch 0:  56%|█████▌    | 3148/5616 [40:04<32:11,  1.28it/s, gpt_loss=0.00295, lr=2.42e-6]
+
Train step of epoch 0:  56%|█████▌    | 3149/5616 [40:04<30:56,  1.33it/s, gpt_loss=0.00295, lr=2.42e-6]
+
Train step of epoch 0:  56%|█████▌    | 3149/5616 [40:05<30:56,  1.33it/s, gpt_loss=0.191, lr=2.42e-6]  
+
Train step of epoch 0:  56%|█████▌    | 3150/5616 [40:05<30:03,  1.37it/s, gpt_loss=0.191, lr=2.42e-6]
+
Train step of epoch 0:  56%|█████▌    | 3150/5616 [40:06<30:03,  1.37it/s, gpt_loss=0.122, lr=2.42e-6]
+
Train step of epoch 0:  56%|█████▌    | 3151/5616 [40:06<29:38,  1.39it/s, gpt_loss=0.122, lr=2.42e-6]
+
Train step of epoch 0:  56%|█████▌    | 3151/5616 [40:07<29:38,  1.39it/s, gpt_loss=0.0122, lr=2.42e-6]
+
Train step of epoch 0:  56%|█████▌    | 3152/5616 [40:07<29:49,  1.38it/s, gpt_loss=0.0122, lr=2.42e-6]
+
Train step of epoch 0:  56%|█████▌    | 3152/5616 [40:07<29:49,  1.38it/s, gpt_loss=0.0272, lr=2.42e-6]
+
Train step of epoch 0:  56%|█████▌    | 3153/5616 [40:07<29:43,  1.38it/s, gpt_loss=0.0272, lr=2.42e-6]
+
Train step of epoch 0:  56%|█████▌    | 3153/5616 [40:08<29:43,  1.38it/s, gpt_loss=0.00935, lr=2.42e-6]
+
Train step of epoch 0:  56%|█████▌    | 3154/5616 [40:08<28:52,  1.42it/s, gpt_loss=0.00935, lr=2.42e-6]
+
Train step of epoch 0:  56%|█████▌    | 3154/5616 [40:09<28:52,  1.42it/s, gpt_loss=0.0223, lr=2.42e-6] 
+
Train step of epoch 0:  56%|█████▌    | 3155/5616 [40:09<28:08,  1.46it/s, gpt_loss=0.0223, lr=2.42e-6]
+
Train step of epoch 0:  56%|█████▌    | 3155/5616 [40:09<28:08,  1.46it/s, gpt_loss=0.00656, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▌    | 3156/5616 [40:09<29:29,  1.39it/s, gpt_loss=0.00656, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▌    | 3156/5616 [40:10<29:29,  1.39it/s, gpt_loss=0.0116, lr=2.41e-6] 
+
Train step of epoch 0:  56%|█████▌    | 3157/5616 [40:10<28:38,  1.43it/s, gpt_loss=0.0116, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▌    | 3157/5616 [40:11<28:38,  1.43it/s, gpt_loss=0.0985, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▌    | 3158/5616 [40:11<28:25,  1.44it/s, gpt_loss=0.0985, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▌    | 3158/5616 [40:11<28:25,  1.44it/s, gpt_loss=0.0244, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▋    | 3159/5616 [40:11<29:03,  1.41it/s, gpt_loss=0.0244, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▋    | 3159/5616 [40:12<29:03,  1.41it/s, gpt_loss=0.00177, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▋    | 3160/5616 [40:12<29:09,  1.40it/s, gpt_loss=0.00177, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▋    | 3160/5616 [40:13<29:09,  1.40it/s, gpt_loss=0.0105, lr=2.41e-6] 
+
Train step of epoch 0:  56%|█████▋    | 3161/5616 [40:13<31:57,  1.28it/s, gpt_loss=0.0105, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▋    | 3161/5616 [40:14<31:57,  1.28it/s, gpt_loss=0.0511, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▋    | 3162/5616 [40:14<31:57,  1.28it/s, gpt_loss=0.0511, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▋    | 3162/5616 [40:15<31:57,  1.28it/s, gpt_loss=0.0154, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▋    | 3163/5616 [40:15<30:38,  1.33it/s, gpt_loss=0.0154, lr=2.41e-6]
+
Train step of epoch 0:  56%|█████▋    | 3163/5616 [40:15<30:38,  1.33it/s, gpt_loss=0.0244, lr=2.4e-6] 
+
Train step of epoch 0:  56%|█████▋    | 3164/5616 [40:15<31:43,  1.29it/s, gpt_loss=0.0244, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3164/5616 [40:16<31:43,  1.29it/s, gpt_loss=0.0353, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3165/5616 [40:16<30:33,  1.34it/s, gpt_loss=0.0353, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3165/5616 [40:17<30:33,  1.34it/s, gpt_loss=0.00933, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3166/5616 [40:17<30:04,  1.36it/s, gpt_loss=0.00933, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3166/5616 [40:18<30:04,  1.36it/s, gpt_loss=0.0304, lr=2.4e-6] 
+
Train step of epoch 0:  56%|█████▋    | 3167/5616 [40:18<30:49,  1.32it/s, gpt_loss=0.0304, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3167/5616 [40:18<30:49,  1.32it/s, gpt_loss=0.0669, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3168/5616 [40:18<31:19,  1.30it/s, gpt_loss=0.0669, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3168/5616 [40:19<31:19,  1.30it/s, gpt_loss=0.0452, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3169/5616 [40:19<30:24,  1.34it/s, gpt_loss=0.0452, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3169/5616 [40:20<30:24,  1.34it/s, gpt_loss=0.0102, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3170/5616 [40:20<29:39,  1.37it/s, gpt_loss=0.0102, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3170/5616 [40:20<29:39,  1.37it/s, gpt_loss=0.0453, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3171/5616 [40:20<29:07,  1.40it/s, gpt_loss=0.0453, lr=2.4e-6]
+
Train step of epoch 0:  56%|█████▋    | 3171/5616 [40:21<29:07,  1.40it/s, gpt_loss=0.0985, lr=2.39e-6]
+
Train step of epoch 0:  56%|█████▋    | 3172/5616 [40:21<29:07,  1.40it/s, gpt_loss=0.0985, lr=2.39e-6]
+
Train step of epoch 0:  56%|█████▋    | 3172/5616 [40:22<29:07,  1.40it/s, gpt_loss=0.0424, lr=2.39e-6]
+
Train step of epoch 0:  56%|█████▋    | 3173/5616 [40:22<28:43,  1.42it/s, gpt_loss=0.0424, lr=2.39e-6]
+
Train step of epoch 0:  56%|█████▋    | 3173/5616 [40:23<28:43,  1.42it/s, gpt_loss=0.168, lr=2.39e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3174/5616 [40:23<28:24,  1.43it/s, gpt_loss=0.168, lr=2.39e-6]
+
Train step of epoch 0:  57%|█████▋    | 3174/5616 [40:23<28:24,  1.43it/s, gpt_loss=0.00998, lr=2.39e-6]
+
Train step of epoch 0:  57%|█████▋    | 3175/5616 [40:23<27:56,  1.46it/s, gpt_loss=0.00998, lr=2.39e-6]
+
Train step of epoch 0:  57%|█████▋    | 3175/5616 [40:24<27:56,  1.46it/s, gpt_loss=0.00104, lr=2.39e-6]
+
Train step of epoch 0:  57%|█████▋    | 3176/5616 [40:24<28:40,  1.42it/s, gpt_loss=0.00104, lr=2.39e-6]
+
Train step of epoch 0:  57%|█████▋    | 3176/5616 [40:25<28:40,  1.42it/s, gpt_loss=0.0294, lr=2.39e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3177/5616 [40:25<29:13,  1.39it/s, gpt_loss=0.0294, lr=2.39e-6]
+
Train step of epoch 0:  57%|█████▋    | 3177/5616 [40:25<29:13,  1.39it/s, gpt_loss=0.00871, lr=2.39e-6]
+
Train step of epoch 0:  57%|█████▋    | 3178/5616 [40:25<28:23,  1.43it/s, gpt_loss=0.00871, lr=2.39e-6]
+
Train step of epoch 0:  57%|█████▋    | 3178/5616 [40:26<28:23,  1.43it/s, gpt_loss=0.0452, lr=2.39e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3179/5616 [40:26<29:12,  1.39it/s, gpt_loss=0.0452, lr=2.39e-6]
+
Train step of epoch 0:  57%|█████▋    | 3179/5616 [40:27<29:12,  1.39it/s, gpt_loss=0.0629, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3180/5616 [40:27<29:27,  1.38it/s, gpt_loss=0.0629, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3180/5616 [40:28<29:27,  1.38it/s, gpt_loss=0.00308, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3181/5616 [40:28<29:24,  1.38it/s, gpt_loss=0.00308, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3181/5616 [40:28<29:24,  1.38it/s, gpt_loss=0.0411, lr=2.38e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3182/5616 [40:28<29:19,  1.38it/s, gpt_loss=0.0411, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3182/5616 [40:29<29:19,  1.38it/s, gpt_loss=0.0146, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3183/5616 [40:29<29:17,  1.38it/s, gpt_loss=0.0146, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3183/5616 [40:30<29:17,  1.38it/s, gpt_loss=0.0133, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3184/5616 [40:30<30:14,  1.34it/s, gpt_loss=0.0133, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3184/5616 [40:31<30:14,  1.34it/s, gpt_loss=0.0457, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3185/5616 [40:31<32:25,  1.25it/s, gpt_loss=0.0457, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3185/5616 [40:31<32:25,  1.25it/s, gpt_loss=0.0189, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3186/5616 [40:31<31:33,  1.28it/s, gpt_loss=0.0189, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3186/5616 [40:32<31:33,  1.28it/s, gpt_loss=0.0196, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3187/5616 [40:32<30:22,  1.33it/s, gpt_loss=0.0196, lr=2.38e-6]
+
Train step of epoch 0:  57%|█████▋    | 3187/5616 [40:33<30:22,  1.33it/s, gpt_loss=0.0515, lr=2.37e-6]
+
Train step of epoch 0:  57%|█████▋    | 3188/5616 [40:33<31:24,  1.29it/s, gpt_loss=0.0515, lr=2.37e-6]
+
Train step of epoch 0:  57%|█████▋    | 3188/5616 [40:34<31:24,  1.29it/s, gpt_loss=0.0522, lr=2.37e-6]
+
Train step of epoch 0:  57%|█████▋    | 3189/5616 [40:34<29:51,  1.35it/s, gpt_loss=0.0522, lr=2.37e-6]
+
Train step of epoch 0:  57%|█████▋    | 3189/5616 [40:34<29:51,  1.35it/s, gpt_loss=0.155, lr=2.37e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3190/5616 [40:34<29:22,  1.38it/s, gpt_loss=0.155, lr=2.37e-6]
+
Train step of epoch 0:  57%|█████▋    | 3190/5616 [40:35<29:22,  1.38it/s, gpt_loss=0.0953, lr=2.37e-6]
+
Train step of epoch 0:  57%|█████▋    | 3191/5616 [40:35<28:51,  1.40it/s, gpt_loss=0.0953, lr=2.37e-6]
+
Train step of epoch 0:  57%|█████▋    | 3191/5616 [40:36<28:51,  1.40it/s, gpt_loss=0.0462, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3192/5616 [40:36<30:12,  1.34it/s, gpt_loss=0.0462, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3192/5616 [40:37<30:12,  1.34it/s, gpt_loss=0.0939, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3193/5616 [40:37<29:28,  1.37it/s, gpt_loss=0.0939, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3193/5616 [40:37<29:28,  1.37it/s, gpt_loss=0.0708, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3194/5616 [40:37<29:31,  1.37it/s, gpt_loss=0.0708, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3194/5616 [40:38<29:31,  1.37it/s, gpt_loss=0.0584, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3195/5616 [40:38<29:56,  1.35it/s, gpt_loss=0.0584, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3195/5616 [40:39<29:56,  1.35it/s, gpt_loss=0.0107, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3196/5616 [40:39<29:37,  1.36it/s, gpt_loss=0.0107, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3196/5616 [40:39<29:37,  1.36it/s, gpt_loss=0.00836, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3197/5616 [40:39<29:02,  1.39it/s, gpt_loss=0.00836, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3197/5616 [40:40<29:02,  1.39it/s, gpt_loss=0.0088, lr=2.36e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3198/5616 [40:40<30:16,  1.33it/s, gpt_loss=0.0088, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3198/5616 [40:41<30:16,  1.33it/s, gpt_loss=0.0116, lr=2.36e-6]
+
Train step of epoch 0:  57%|█████▋    | 3199/5616 [40:41<29:12,  1.38it/s, gpt_loss=0.0116, lr=2.36e-6][2025-06-30 07:18:50,678] [INFO] [logging.py:128:log_dist] [Rank 0] step=800, skipped=0, lr=[2.3547459676779318e-06, 2.3547459676779318e-06], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 07:18:50,679] [INFO] [timer.py:264:stop] epoch=0/micro_step=3200/global_step=800, RunningAvgSamplesPerSec=42.48175532995403, CurrSamplesPerSec=44.38340539251035, MemAllocated=15.92GB, MaxMemAllocated=51.79GB
+
+
Train step of epoch 0:  57%|█████▋    | 3199/5616 [40:42<29:12,  1.38it/s, gpt_loss=0.00528, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3200/5616 [40:42<29:22,  1.37it/s, gpt_loss=0.00528, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3200/5616 [40:42<29:22,  1.37it/s, gpt_loss=0.0262, lr=2.35e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3201/5616 [40:42<28:51,  1.39it/s, gpt_loss=0.0262, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3201/5616 [40:43<28:51,  1.39it/s, gpt_loss=0.00548, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3202/5616 [40:43<30:53,  1.30it/s, gpt_loss=0.00548, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3202/5616 [40:44<30:53,  1.30it/s, gpt_loss=0.00676, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3203/5616 [40:44<29:03,  1.38it/s, gpt_loss=0.00676, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3203/5616 [40:45<29:03,  1.38it/s, gpt_loss=0.0564, lr=2.35e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3204/5616 [40:45<31:04,  1.29it/s, gpt_loss=0.0564, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3204/5616 [40:46<31:04,  1.29it/s, gpt_loss=0.0153, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3205/5616 [40:46<32:58,  1.22it/s, gpt_loss=0.0153, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3205/5616 [40:46<32:58,  1.22it/s, gpt_loss=0.0599, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3206/5616 [40:46<30:55,  1.30it/s, gpt_loss=0.0599, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3206/5616 [40:47<30:55,  1.30it/s, gpt_loss=0.00781, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3207/5616 [40:47<29:34,  1.36it/s, gpt_loss=0.00781, lr=2.35e-6]
+
Train step of epoch 0:  57%|█████▋    | 3207/5616 [40:48<29:34,  1.36it/s, gpt_loss=0.0385, lr=2.34e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3208/5616 [40:48<30:30,  1.32it/s, gpt_loss=0.0385, lr=2.34e-6]
+
Train step of epoch 0:  57%|█████▋    | 3208/5616 [40:49<30:30,  1.32it/s, gpt_loss=0.013, lr=2.34e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3209/5616 [40:49<31:25,  1.28it/s, gpt_loss=0.013, lr=2.34e-6]
+
Train step of epoch 0:  57%|█████▋    | 3209/5616 [40:49<31:25,  1.28it/s, gpt_loss=0.101, lr=2.34e-6]
+
Train step of epoch 0:  57%|█████▋    | 3210/5616 [40:49<30:39,  1.31it/s, gpt_loss=0.101, lr=2.34e-6]
+
Train step of epoch 0:  57%|█████▋    | 3210/5616 [40:50<30:39,  1.31it/s, gpt_loss=0.00134, lr=2.34e-6]
+
Train step of epoch 0:  57%|█████▋    | 3211/5616 [40:50<29:36,  1.35it/s, gpt_loss=0.00134, lr=2.34e-6]
+
Train step of epoch 0:  57%|█████▋    | 3211/5616 [40:51<29:36,  1.35it/s, gpt_loss=0.0817, lr=2.34e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3212/5616 [40:51<29:13,  1.37it/s, gpt_loss=0.0817, lr=2.34e-6]
+
Train step of epoch 0:  57%|█████▋    | 3212/5616 [40:51<29:13,  1.37it/s, gpt_loss=0.0588, lr=2.34e-6]
+
Train step of epoch 0:  57%|█████▋    | 3213/5616 [40:51<28:53,  1.39it/s, gpt_loss=0.0588, lr=2.34e-6]
+
Train step of epoch 0:  57%|█████▋    | 3213/5616 [40:52<28:53,  1.39it/s, gpt_loss=0.0389, lr=2.34e-6]
+
Train step of epoch 0:  57%|█████▋    | 3214/5616 [40:52<29:44,  1.35it/s, gpt_loss=0.0389, lr=2.34e-6]
+
Train step of epoch 0:  57%|█████▋    | 3214/5616 [40:53<29:44,  1.35it/s, gpt_loss=0.0329, lr=2.34e-6]
+
Train step of epoch 0:  57%|█████▋    | 3215/5616 [40:53<29:31,  1.36it/s, gpt_loss=0.0329, lr=2.34e-6]
+
Train step of epoch 0:  57%|█████▋    | 3215/5616 [40:54<29:31,  1.36it/s, gpt_loss=0.1, lr=2.33e-6]   
+
Train step of epoch 0:  57%|█████▋    | 3216/5616 [40:54<28:54,  1.38it/s, gpt_loss=0.1, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3216/5616 [40:54<28:54,  1.38it/s, gpt_loss=0.0536, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3217/5616 [40:54<28:44,  1.39it/s, gpt_loss=0.0536, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3217/5616 [40:55<28:44,  1.39it/s, gpt_loss=0.0745, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3218/5616 [40:55<28:00,  1.43it/s, gpt_loss=0.0745, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3218/5616 [40:56<28:00,  1.43it/s, gpt_loss=0.0308, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3219/5616 [40:56<28:14,  1.41it/s, gpt_loss=0.0308, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3219/5616 [40:57<28:14,  1.41it/s, gpt_loss=0.00254, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3220/5616 [40:57<29:03,  1.37it/s, gpt_loss=0.00254, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3220/5616 [40:58<29:03,  1.37it/s, gpt_loss=0.00548, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3221/5616 [40:58<33:37,  1.19it/s, gpt_loss=0.00548, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3221/5616 [40:58<33:37,  1.19it/s, gpt_loss=0.00236, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3222/5616 [40:58<31:44,  1.26it/s, gpt_loss=0.00236, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3222/5616 [40:59<31:44,  1.26it/s, gpt_loss=0.0533, lr=2.33e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3223/5616 [40:59<31:11,  1.28it/s, gpt_loss=0.0533, lr=2.33e-6]
+
Train step of epoch 0:  57%|█████▋    | 3223/5616 [41:00<31:11,  1.28it/s, gpt_loss=0.051, lr=2.32e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3224/5616 [41:00<30:20,  1.31it/s, gpt_loss=0.051, lr=2.32e-6]
+
Train step of epoch 0:  57%|█████▋    | 3224/5616 [41:00<30:20,  1.31it/s, gpt_loss=0.0214, lr=2.32e-6]
+
Train step of epoch 0:  57%|█████▋    | 3225/5616 [41:00<29:13,  1.36it/s, gpt_loss=0.0214, lr=2.32e-6]
+
Train step of epoch 0:  57%|█████▋    | 3225/5616 [41:01<29:13,  1.36it/s, gpt_loss=0.057, lr=2.32e-6] 
+
Train step of epoch 0:  57%|█████▋    | 3226/5616 [41:01<28:50,  1.38it/s, gpt_loss=0.057, lr=2.32e-6]
+
Train step of epoch 0:  57%|█████▋    | 3226/5616 [41:02<28:50,  1.38it/s, gpt_loss=0.119, lr=2.32e-6]
+
Train step of epoch 0:  57%|█████▋    | 3227/5616 [41:02<30:00,  1.33it/s, gpt_loss=0.119, lr=2.32e-6]
+
Train step of epoch 0:  57%|█████▋    | 3227/5616 [41:03<30:00,  1.33it/s, gpt_loss=0.0487, lr=2.32e-6]
+
Train step of epoch 0:  57%|█████▋    | 3228/5616 [41:03<29:22,  1.35it/s, gpt_loss=0.0487, lr=2.32e-6]
+
Train step of epoch 0:  57%|█████▋    | 3228/5616 [41:03<29:22,  1.35it/s, gpt_loss=0.0178, lr=2.32e-6]
+
Train step of epoch 0:  57%|█████▋    | 3229/5616 [41:03<28:17,  1.41it/s, gpt_loss=0.0178, lr=2.32e-6]
+
Train step of epoch 0:  57%|█████▋    | 3229/5616 [41:04<28:17,  1.41it/s, gpt_loss=0.0116, lr=2.32e-6]
+
Train step of epoch 0:  58%|█████▊    | 3230/5616 [41:04<28:06,  1.42it/s, gpt_loss=0.0116, lr=2.32e-6]
+
Train step of epoch 0:  58%|█████▊    | 3230/5616 [41:05<28:06,  1.42it/s, gpt_loss=0.0271, lr=2.32e-6]
+
Train step of epoch 0:  58%|█████▊    | 3231/5616 [41:05<27:30,  1.45it/s, gpt_loss=0.0271, lr=2.32e-6]
+
Train step of epoch 0:  58%|█████▊    | 3231/5616 [41:06<27:30,  1.45it/s, gpt_loss=0.00898, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3232/5616 [41:06<29:38,  1.34it/s, gpt_loss=0.00898, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3232/5616 [41:06<29:38,  1.34it/s, gpt_loss=0.0587, lr=2.31e-6] 
+
Train step of epoch 0:  58%|█████▊    | 3233/5616 [41:06<29:27,  1.35it/s, gpt_loss=0.0587, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3233/5616 [41:07<29:27,  1.35it/s, gpt_loss=0.0156, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3234/5616 [41:07<33:15,  1.19it/s, gpt_loss=0.0156, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3234/5616 [41:08<33:15,  1.19it/s, gpt_loss=0.0378, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3235/5616 [41:08<30:59,  1.28it/s, gpt_loss=0.0378, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3235/5616 [41:09<30:59,  1.28it/s, gpt_loss=0.0219, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3236/5616 [41:09<30:51,  1.29it/s, gpt_loss=0.0219, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3236/5616 [41:09<30:51,  1.29it/s, gpt_loss=0.0264, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3237/5616 [41:09<29:48,  1.33it/s, gpt_loss=0.0264, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3237/5616 [41:10<29:48,  1.33it/s, gpt_loss=0.00808, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3238/5616 [41:10<29:46,  1.33it/s, gpt_loss=0.00808, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3238/5616 [41:11<29:46,  1.33it/s, gpt_loss=0.0121, lr=2.31e-6] 
+
Train step of epoch 0:  58%|█████▊    | 3239/5616 [41:11<30:14,  1.31it/s, gpt_loss=0.0121, lr=2.31e-6]
+
Train step of epoch 0:  58%|█████▊    | 3239/5616 [41:12<30:14,  1.31it/s, gpt_loss=0.0199, lr=2.3e-6] 
+
Train step of epoch 0:  58%|█████▊    | 3240/5616 [41:12<30:50,  1.28it/s, gpt_loss=0.0199, lr=2.3e-6]
+
Train step of epoch 0:  58%|█████▊    | 3240/5616 [41:13<30:50,  1.28it/s, gpt_loss=0.115, lr=2.3e-6] 
+
Train step of epoch 0:  58%|█████▊    | 3241/5616 [41:13<31:15,  1.27it/s, gpt_loss=0.115, lr=2.3e-6]
+
Train step of epoch 0:  58%|█████▊    | 3241/5616 [41:13<31:15,  1.27it/s, gpt_loss=0.127, lr=2.3e-6]
+
Train step of epoch 0:  58%|█████▊    | 3242/5616 [41:13<31:41,  1.25it/s, gpt_loss=0.127, lr=2.3e-6]
+
Train step of epoch 0:  58%|█████▊    | 3242/5616 [41:14<31:41,  1.25it/s, gpt_loss=0.12, lr=2.3e-6] 
+
Train step of epoch 0:  58%|█████▊    | 3243/5616 [41:14<31:38,  1.25it/s, gpt_loss=0.12, lr=2.3e-6]
+
Train step of epoch 0:  58%|█████▊    | 3243/5616 [41:15<31:38,  1.25it/s, gpt_loss=0.0255, lr=2.3e-6]
+
Train step of epoch 0:  58%|█████▊    | 3244/5616 [41:15<31:09,  1.27it/s, gpt_loss=0.0255, lr=2.3e-6]
+
Train step of epoch 0:  58%|█████▊    | 3244/5616 [41:16<31:09,  1.27it/s, gpt_loss=0.0826, lr=2.3e-6]
+
Train step of epoch 0:  58%|█████▊    | 3245/5616 [41:16<29:56,  1.32it/s, gpt_loss=0.0826, lr=2.3e-6]
+
Train step of epoch 0:  58%|█████▊    | 3245/5616 [41:16<29:56,  1.32it/s, gpt_loss=0.0705, lr=2.3e-6]
+
Train step of epoch 0:  58%|█████▊    | 3246/5616 [41:16<29:34,  1.34it/s, gpt_loss=0.0705, lr=2.3e-6]
+
Train step of epoch 0:  58%|█████▊    | 3246/5616 [41:17<29:34,  1.34it/s, gpt_loss=0.00809, lr=2.3e-6]
+
Train step of epoch 0:  58%|█████▊    | 3247/5616 [41:17<28:38,  1.38it/s, gpt_loss=0.00809, lr=2.3e-6]
+
Train step of epoch 0:  58%|█████▊    | 3247/5616 [41:18<28:38,  1.38it/s, gpt_loss=0.0652, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3248/5616 [41:18<30:58,  1.27it/s, gpt_loss=0.0652, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3248/5616 [41:19<30:58,  1.27it/s, gpt_loss=0.0225, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3249/5616 [41:19<29:47,  1.32it/s, gpt_loss=0.0225, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3249/5616 [41:19<29:47,  1.32it/s, gpt_loss=0.0956, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3250/5616 [41:19<28:27,  1.39it/s, gpt_loss=0.0956, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3250/5616 [41:20<28:27,  1.39it/s, gpt_loss=0.0737, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3251/5616 [41:20<28:09,  1.40it/s, gpt_loss=0.0737, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3251/5616 [41:21<28:09,  1.40it/s, gpt_loss=0.0223, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3252/5616 [41:21<28:51,  1.37it/s, gpt_loss=0.0223, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3252/5616 [41:22<28:51,  1.37it/s, gpt_loss=0.0313, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3253/5616 [41:22<32:42,  1.20it/s, gpt_loss=0.0313, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3253/5616 [41:23<32:42,  1.20it/s, gpt_loss=0.0353, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3254/5616 [41:23<30:42,  1.28it/s, gpt_loss=0.0353, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3254/5616 [41:23<30:42,  1.28it/s, gpt_loss=0.0491, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3255/5616 [41:23<29:20,  1.34it/s, gpt_loss=0.0491, lr=2.29e-6]
+
Train step of epoch 0:  58%|█████▊    | 3255/5616 [41:24<29:20,  1.34it/s, gpt_loss=0.0343, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3256/5616 [41:24<29:20,  1.34it/s, gpt_loss=0.0343, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3256/5616 [41:25<29:20,  1.34it/s, gpt_loss=0.0162, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3257/5616 [41:25<30:40,  1.28it/s, gpt_loss=0.0162, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3257/5616 [41:26<30:40,  1.28it/s, gpt_loss=0.0108, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3258/5616 [41:26<34:35,  1.14it/s, gpt_loss=0.0108, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3258/5616 [41:27<34:35,  1.14it/s, gpt_loss=0.0163, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3259/5616 [41:27<32:19,  1.22it/s, gpt_loss=0.0163, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3259/5616 [41:27<32:19,  1.22it/s, gpt_loss=0.00945, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3260/5616 [41:27<32:06,  1.22it/s, gpt_loss=0.00945, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3260/5616 [41:28<32:06,  1.22it/s, gpt_loss=0.0489, lr=2.28e-6] 
+
Train step of epoch 0:  58%|█████▊    | 3261/5616 [41:28<34:33,  1.14it/s, gpt_loss=0.0489, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3261/5616 [41:29<34:33,  1.14it/s, gpt_loss=0.0468, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3262/5616 [41:29<34:45,  1.13it/s, gpt_loss=0.0468, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3262/5616 [41:30<34:45,  1.13it/s, gpt_loss=0.0407, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3263/5616 [41:30<32:09,  1.22it/s, gpt_loss=0.0407, lr=2.28e-6]
+
Train step of epoch 0:  58%|█████▊    | 3263/5616 [41:31<32:09,  1.22it/s, gpt_loss=0.00289, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3264/5616 [41:31<31:38,  1.24it/s, gpt_loss=0.00289, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3264/5616 [41:31<31:38,  1.24it/s, gpt_loss=0.0269, lr=2.27e-6] 
+
Train step of epoch 0:  58%|█████▊    | 3265/5616 [41:31<29:35,  1.32it/s, gpt_loss=0.0269, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3265/5616 [41:32<29:35,  1.32it/s, gpt_loss=0.0727, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3266/5616 [41:32<29:17,  1.34it/s, gpt_loss=0.0727, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3266/5616 [41:33<29:17,  1.34it/s, gpt_loss=0.0315, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3267/5616 [41:33<28:04,  1.39it/s, gpt_loss=0.0315, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3267/5616 [41:33<28:04,  1.39it/s, gpt_loss=0.0362, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3268/5616 [41:33<27:58,  1.40it/s, gpt_loss=0.0362, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3268/5616 [41:34<27:58,  1.40it/s, gpt_loss=0.0237, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3269/5616 [41:34<30:26,  1.29it/s, gpt_loss=0.0237, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3269/5616 [41:35<30:26,  1.29it/s, gpt_loss=0.0318, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3270/5616 [41:35<29:13,  1.34it/s, gpt_loss=0.0318, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3270/5616 [41:36<29:13,  1.34it/s, gpt_loss=0.0536, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3271/5616 [41:36<29:33,  1.32it/s, gpt_loss=0.0536, lr=2.27e-6]
+
Train step of epoch 0:  58%|█████▊    | 3271/5616 [41:37<29:33,  1.32it/s, gpt_loss=0.0432, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3272/5616 [41:37<30:30,  1.28it/s, gpt_loss=0.0432, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3272/5616 [41:37<30:30,  1.28it/s, gpt_loss=0.0203, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3273/5616 [41:37<29:51,  1.31it/s, gpt_loss=0.0203, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3273/5616 [41:38<29:51,  1.31it/s, gpt_loss=0.0425, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3274/5616 [41:38<29:49,  1.31it/s, gpt_loss=0.0425, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3274/5616 [41:39<29:49,  1.31it/s, gpt_loss=0.00818, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3275/5616 [41:39<29:06,  1.34it/s, gpt_loss=0.00818, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3275/5616 [41:40<29:06,  1.34it/s, gpt_loss=0.0671, lr=2.26e-6] 
+
Train step of epoch 0:  58%|█████▊    | 3276/5616 [41:40<29:03,  1.34it/s, gpt_loss=0.0671, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3276/5616 [41:40<29:03,  1.34it/s, gpt_loss=0.00254, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3277/5616 [41:40<28:21,  1.37it/s, gpt_loss=0.00254, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3277/5616 [41:41<28:21,  1.37it/s, gpt_loss=0.0138, lr=2.26e-6] 
+
Train step of epoch 0:  58%|█████▊    | 3278/5616 [41:41<27:34,  1.41it/s, gpt_loss=0.0138, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3278/5616 [41:42<27:34,  1.41it/s, gpt_loss=0.00477, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3279/5616 [41:42<27:59,  1.39it/s, gpt_loss=0.00477, lr=2.26e-6]
+
Train step of epoch 0:  58%|█████▊    | 3279/5616 [41:43<27:59,  1.39it/s, gpt_loss=0.00757, lr=2.25e-6]
+
Train step of epoch 0:  58%|█████▊    | 3280/5616 [41:43<30:41,  1.27it/s, gpt_loss=0.00757, lr=2.25e-6]
+
Train step of epoch 0:  58%|█████▊    | 3280/5616 [41:43<30:41,  1.27it/s, gpt_loss=0.0928, lr=2.25e-6] 
+
Train step of epoch 0:  58%|█████▊    | 3281/5616 [41:43<29:28,  1.32it/s, gpt_loss=0.0928, lr=2.25e-6]
+
Train step of epoch 0:  58%|█████▊    | 3281/5616 [41:44<29:28,  1.32it/s, gpt_loss=0.00855, lr=2.25e-6]
+
Train step of epoch 0:  58%|█████▊    | 3282/5616 [41:44<28:45,  1.35it/s, gpt_loss=0.00855, lr=2.25e-6]
+
Train step of epoch 0:  58%|█████▊    | 3282/5616 [41:45<28:45,  1.35it/s, gpt_loss=0.0371, lr=2.25e-6] 
+
Train step of epoch 0:  58%|█████▊    | 3283/5616 [41:45<30:01,  1.30it/s, gpt_loss=0.0371, lr=2.25e-6]
+
Train step of epoch 0:  58%|█████▊    | 3283/5616 [41:46<30:01,  1.30it/s, gpt_loss=0.00404, lr=2.25e-6]
+
Train step of epoch 0:  58%|█████▊    | 3284/5616 [41:46<30:50,  1.26it/s, gpt_loss=0.00404, lr=2.25e-6]
+
Train step of epoch 0:  58%|█████▊    | 3284/5616 [41:46<30:50,  1.26it/s, gpt_loss=0.00267, lr=2.25e-6]
+
Train step of epoch 0:  58%|█████▊    | 3285/5616 [41:46<29:01,  1.34it/s, gpt_loss=0.00267, lr=2.25e-6]
+
Train step of epoch 0:  58%|█████▊    | 3285/5616 [41:47<29:01,  1.34it/s, gpt_loss=0.0124, lr=2.25e-6] 
+
Train step of epoch 0:  59%|█████▊    | 3286/5616 [41:47<28:41,  1.35it/s, gpt_loss=0.0124, lr=2.25e-6]
+
Train step of epoch 0:  59%|█████▊    | 3286/5616 [41:48<28:41,  1.35it/s, gpt_loss=0.0231, lr=2.25e-6]
+
Train step of epoch 0:  59%|█████▊    | 3287/5616 [41:48<27:51,  1.39it/s, gpt_loss=0.0231, lr=2.25e-6]
+
Train step of epoch 0:  59%|█████▊    | 3287/5616 [41:49<27:51,  1.39it/s, gpt_loss=0.0634, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3288/5616 [41:49<28:43,  1.35it/s, gpt_loss=0.0634, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3288/5616 [41:49<28:43,  1.35it/s, gpt_loss=0.0219, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3289/5616 [41:49<28:26,  1.36it/s, gpt_loss=0.0219, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3289/5616 [41:50<28:26,  1.36it/s, gpt_loss=0.0172, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3290/5616 [41:50<27:15,  1.42it/s, gpt_loss=0.0172, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3290/5616 [41:51<27:15,  1.42it/s, gpt_loss=0.0279, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3291/5616 [41:51<26:53,  1.44it/s, gpt_loss=0.0279, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3291/5616 [41:52<26:53,  1.44it/s, gpt_loss=0.0658, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3292/5616 [41:52<29:50,  1.30it/s, gpt_loss=0.0658, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3292/5616 [41:52<29:50,  1.30it/s, gpt_loss=0.00555, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3293/5616 [41:52<28:52,  1.34it/s, gpt_loss=0.00555, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3293/5616 [41:53<28:52,  1.34it/s, gpt_loss=0.00526, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3294/5616 [41:53<27:44,  1.40it/s, gpt_loss=0.00526, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3294/5616 [41:54<27:44,  1.40it/s, gpt_loss=0.124, lr=2.24e-6]  
+
Train step of epoch 0:  59%|█████▊    | 3295/5616 [41:54<27:30,  1.41it/s, gpt_loss=0.124, lr=2.24e-6]
+
Train step of epoch 0:  59%|█████▊    | 3295/5616 [41:54<27:30,  1.41it/s, gpt_loss=0.0361, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▊    | 3296/5616 [41:54<28:36,  1.35it/s, gpt_loss=0.0361, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▊    | 3296/5616 [41:55<28:36,  1.35it/s, gpt_loss=0.0716, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▊    | 3297/5616 [41:55<28:32,  1.35it/s, gpt_loss=0.0716, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▊    | 3297/5616 [41:56<28:32,  1.35it/s, gpt_loss=0.0726, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▊    | 3298/5616 [41:56<27:19,  1.41it/s, gpt_loss=0.0726, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▊    | 3298/5616 [41:56<27:19,  1.41it/s, gpt_loss=0.055, lr=2.23e-6] 
+
Train step of epoch 0:  59%|█████▊    | 3299/5616 [41:56<27:10,  1.42it/s, gpt_loss=0.055, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▊    | 3299/5616 [41:57<27:10,  1.42it/s, gpt_loss=0.0412, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▉    | 3300/5616 [41:57<27:29,  1.40it/s, gpt_loss=0.0412, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▉    | 3300/5616 [41:58<27:29,  1.40it/s, gpt_loss=0.119, lr=2.23e-6] 
+
Train step of epoch 0:  59%|█████▉    | 3301/5616 [41:58<26:46,  1.44it/s, gpt_loss=0.119, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▉    | 3301/5616 [41:59<26:46,  1.44it/s, gpt_loss=0.0737, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▉    | 3302/5616 [41:59<26:54,  1.43it/s, gpt_loss=0.0737, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▉    | 3302/5616 [41:59<26:54,  1.43it/s, gpt_loss=0.0315, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▉    | 3303/5616 [41:59<27:29,  1.40it/s, gpt_loss=0.0315, lr=2.23e-6]
+
Train step of epoch 0:  59%|█████▉    | 3303/5616 [42:00<27:29,  1.40it/s, gpt_loss=0.0469, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3304/5616 [42:00<28:14,  1.36it/s, gpt_loss=0.0469, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3304/5616 [42:01<28:14,  1.36it/s, gpt_loss=0.0384, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3305/5616 [42:01<29:42,  1.30it/s, gpt_loss=0.0384, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3305/5616 [42:02<29:42,  1.30it/s, gpt_loss=0.00353, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3306/5616 [42:02<28:04,  1.37it/s, gpt_loss=0.00353, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3306/5616 [42:02<28:04,  1.37it/s, gpt_loss=0.0227, lr=2.22e-6] 
+
Train step of epoch 0:  59%|█████▉    | 3307/5616 [42:02<27:33,  1.40it/s, gpt_loss=0.0227, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3307/5616 [42:03<27:33,  1.40it/s, gpt_loss=0.12, lr=2.22e-6]  
+
Train step of epoch 0:  59%|█████▉    | 3308/5616 [42:03<29:45,  1.29it/s, gpt_loss=0.12, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3308/5616 [42:04<29:45,  1.29it/s, gpt_loss=0.155, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3309/5616 [42:04<30:22,  1.27it/s, gpt_loss=0.155, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3309/5616 [42:05<30:22,  1.27it/s, gpt_loss=0.0754, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3310/5616 [42:05<28:28,  1.35it/s, gpt_loss=0.0754, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3310/5616 [42:05<28:28,  1.35it/s, gpt_loss=0.0159, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3311/5616 [42:05<28:50,  1.33it/s, gpt_loss=0.0159, lr=2.22e-6]
+
Train step of epoch 0:  59%|█████▉    | 3311/5616 [42:06<28:50,  1.33it/s, gpt_loss=0.00515, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3312/5616 [42:06<30:21,  1.26it/s, gpt_loss=0.00515, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3312/5616 [42:07<30:21,  1.26it/s, gpt_loss=0.0132, lr=2.21e-6] 
+
Train step of epoch 0:  59%|█████▉    | 3313/5616 [42:07<29:05,  1.32it/s, gpt_loss=0.0132, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3313/5616 [42:08<29:05,  1.32it/s, gpt_loss=0.0104, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3314/5616 [42:08<29:02,  1.32it/s, gpt_loss=0.0104, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3314/5616 [42:08<29:02,  1.32it/s, gpt_loss=0.0178, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3315/5616 [42:08<28:42,  1.34it/s, gpt_loss=0.0178, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3315/5616 [42:09<28:42,  1.34it/s, gpt_loss=0.0344, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3316/5616 [42:09<29:06,  1.32it/s, gpt_loss=0.0344, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3316/5616 [42:10<29:06,  1.32it/s, gpt_loss=0.0599, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3317/5616 [42:10<30:42,  1.25it/s, gpt_loss=0.0599, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3317/5616 [42:11<30:42,  1.25it/s, gpt_loss=0.0206, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3318/5616 [42:11<30:28,  1.26it/s, gpt_loss=0.0206, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3318/5616 [42:12<30:28,  1.26it/s, gpt_loss=0.0599, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3319/5616 [42:12<29:43,  1.29it/s, gpt_loss=0.0599, lr=2.21e-6]
+
Train step of epoch 0:  59%|█████▉    | 3319/5616 [42:12<29:43,  1.29it/s, gpt_loss=0.0102, lr=2.2e-6] 
+
Train step of epoch 0:  59%|█████▉    | 3320/5616 [42:12<28:59,  1.32it/s, gpt_loss=0.0102, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3320/5616 [42:13<28:59,  1.32it/s, gpt_loss=0.0436, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3321/5616 [42:13<28:42,  1.33it/s, gpt_loss=0.0436, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3321/5616 [42:14<28:42,  1.33it/s, gpt_loss=0.0194, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3322/5616 [42:14<29:51,  1.28it/s, gpt_loss=0.0194, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3322/5616 [42:15<29:51,  1.28it/s, gpt_loss=0.00407, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3323/5616 [42:15<28:49,  1.33it/s, gpt_loss=0.00407, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3323/5616 [42:15<28:49,  1.33it/s, gpt_loss=0.00555, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3324/5616 [42:15<29:34,  1.29it/s, gpt_loss=0.00555, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3324/5616 [42:16<29:34,  1.29it/s, gpt_loss=0.0225, lr=2.2e-6] 
+
Train step of epoch 0:  59%|█████▉    | 3325/5616 [42:16<30:09,  1.27it/s, gpt_loss=0.0225, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3325/5616 [42:17<30:09,  1.27it/s, gpt_loss=0.0328, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3326/5616 [42:17<28:36,  1.33it/s, gpt_loss=0.0328, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3326/5616 [42:18<28:36,  1.33it/s, gpt_loss=0.00811, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3327/5616 [42:18<28:33,  1.34it/s, gpt_loss=0.00811, lr=2.2e-6]
+
Train step of epoch 0:  59%|█████▉    | 3327/5616 [42:18<28:33,  1.34it/s, gpt_loss=0.0331, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3328/5616 [42:18<29:21,  1.30it/s, gpt_loss=0.0331, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3328/5616 [42:19<29:21,  1.30it/s, gpt_loss=0.00412, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3329/5616 [42:19<28:14,  1.35it/s, gpt_loss=0.00412, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3329/5616 [42:20<28:14,  1.35it/s, gpt_loss=0.00205, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3330/5616 [42:20<27:24,  1.39it/s, gpt_loss=0.00205, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3330/5616 [42:21<27:24,  1.39it/s, gpt_loss=0.00571, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3331/5616 [42:21<27:12,  1.40it/s, gpt_loss=0.00571, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3331/5616 [42:21<27:12,  1.40it/s, gpt_loss=0.0291, lr=2.19e-6] 
+
Train step of epoch 0:  59%|█████▉    | 3332/5616 [42:21<28:13,  1.35it/s, gpt_loss=0.0291, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3332/5616 [42:22<28:13,  1.35it/s, gpt_loss=0.0232, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3333/5616 [42:22<28:32,  1.33it/s, gpt_loss=0.0232, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3333/5616 [42:23<28:32,  1.33it/s, gpt_loss=0.022, lr=2.19e-6] 
+
Train step of epoch 0:  59%|█████▉    | 3334/5616 [42:23<27:00,  1.41it/s, gpt_loss=0.022, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3334/5616 [42:23<27:00,  1.41it/s, gpt_loss=0.00271, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3335/5616 [42:23<26:25,  1.44it/s, gpt_loss=0.00271, lr=2.19e-6]
+
Train step of epoch 0:  59%|█████▉    | 3335/5616 [42:24<26:25,  1.44it/s, gpt_loss=0.0139, lr=2.18e-6] 
+
Train step of epoch 0:  59%|█████▉    | 3336/5616 [42:24<26:50,  1.42it/s, gpt_loss=0.0139, lr=2.18e-6]
+
Train step of epoch 0:  59%|█████▉    | 3336/5616 [42:25<26:50,  1.42it/s, gpt_loss=0.0019, lr=2.18e-6]
+
Train step of epoch 0:  59%|█████▉    | 3337/5616 [42:25<27:53,  1.36it/s, gpt_loss=0.0019, lr=2.18e-6]
+
Train step of epoch 0:  59%|█████▉    | 3337/5616 [42:26<27:53,  1.36it/s, gpt_loss=0.029, lr=2.18e-6] 
+
Train step of epoch 0:  59%|█████▉    | 3338/5616 [42:26<28:15,  1.34it/s, gpt_loss=0.029, lr=2.18e-6]
+
Train step of epoch 0:  59%|█████▉    | 3338/5616 [42:26<28:15,  1.34it/s, gpt_loss=0.0584, lr=2.18e-6]
+
Train step of epoch 0:  59%|█████▉    | 3339/5616 [42:26<28:48,  1.32it/s, gpt_loss=0.0584, lr=2.18e-6]
+
Train step of epoch 0:  59%|█████▉    | 3339/5616 [42:27<28:48,  1.32it/s, gpt_loss=0.029, lr=2.18e-6] 
+
Train step of epoch 0:  59%|█████▉    | 3340/5616 [42:27<28:12,  1.35it/s, gpt_loss=0.029, lr=2.18e-6]
+
Train step of epoch 0:  59%|█████▉    | 3340/5616 [42:28<28:12,  1.35it/s, gpt_loss=0.0318, lr=2.18e-6]
+
Train step of epoch 0:  59%|█████▉    | 3341/5616 [42:28<27:00,  1.40it/s, gpt_loss=0.0318, lr=2.18e-6]
+
Train step of epoch 0:  59%|█████▉    | 3341/5616 [42:29<27:00,  1.40it/s, gpt_loss=0.108, lr=2.18e-6] 
+
Train step of epoch 0:  60%|█████▉    | 3342/5616 [42:29<26:47,  1.41it/s, gpt_loss=0.108, lr=2.18e-6]
+
Train step of epoch 0:  60%|█████▉    | 3342/5616 [42:29<26:47,  1.41it/s, gpt_loss=0.0186, lr=2.18e-6]
+
Train step of epoch 0:  60%|█████▉    | 3343/5616 [42:29<26:39,  1.42it/s, gpt_loss=0.0186, lr=2.18e-6]
+
Train step of epoch 0:  60%|█████▉    | 3343/5616 [42:30<26:39,  1.42it/s, gpt_loss=0.0184, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3344/5616 [42:30<27:59,  1.35it/s, gpt_loss=0.0184, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3344/5616 [42:31<27:59,  1.35it/s, gpt_loss=0.0419, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3345/5616 [42:31<27:19,  1.39it/s, gpt_loss=0.0419, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3345/5616 [42:31<27:19,  1.39it/s, gpt_loss=0.00342, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3346/5616 [42:31<27:32,  1.37it/s, gpt_loss=0.00342, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3346/5616 [42:33<27:32,  1.37it/s, gpt_loss=0.0247, lr=2.17e-6] 
+
Train step of epoch 0:  60%|█████▉    | 3347/5616 [42:33<32:02,  1.18it/s, gpt_loss=0.0247, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3347/5616 [42:33<32:02,  1.18it/s, gpt_loss=0.0262, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3348/5616 [42:33<31:34,  1.20it/s, gpt_loss=0.0262, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3348/5616 [42:34<31:34,  1.20it/s, gpt_loss=0.0689, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3349/5616 [42:34<30:22,  1.24it/s, gpt_loss=0.0689, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3349/5616 [42:35<30:22,  1.24it/s, gpt_loss=0.00898, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3350/5616 [42:35<28:51,  1.31it/s, gpt_loss=0.00898, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3350/5616 [42:36<28:51,  1.31it/s, gpt_loss=0.00707, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3351/5616 [42:36<29:23,  1.28it/s, gpt_loss=0.00707, lr=2.17e-6]
+
Train step of epoch 0:  60%|█████▉    | 3351/5616 [42:36<29:23,  1.28it/s, gpt_loss=0.0178, lr=2.16e-6] 
+
Train step of epoch 0:  60%|█████▉    | 3352/5616 [42:36<29:37,  1.27it/s, gpt_loss=0.0178, lr=2.16e-6]
+
Train step of epoch 0:  60%|█████▉    | 3352/5616 [42:37<29:37,  1.27it/s, gpt_loss=0.0662, lr=2.16e-6]
+
Train step of epoch 0:  60%|████��▉    | 3353/5616 [42:37<29:42,  1.27it/s, gpt_loss=0.0662, lr=2.16e-6]
+
Train step of epoch 0:  60%|█████▉    | 3353/5616 [42:38<29:42,  1.27it/s, gpt_loss=0.0208, lr=2.16e-6]
+
Train step of epoch 0:  60%|█████▉    | 3354/5616 [42:38<29:26,  1.28it/s, gpt_loss=0.0208, lr=2.16e-6]
+
Train step of epoch 0:  60%|█████▉    | 3354/5616 [42:39<29:26,  1.28it/s, gpt_loss=0.0438, lr=2.16e-6]
+
Train step of epoch 0:  60%|█████▉    | 3355/5616 [42:39<29:15,  1.29it/s, gpt_loss=0.0438, lr=2.16e-6]
+
Train step of epoch 0:  60%|█████▉    | 3355/5616 [42:40<29:15,  1.29it/s, gpt_loss=0.0106, lr=2.16e-6]
+
Train step of epoch 0:  60%|█████▉    | 3356/5616 [42:40<29:25,  1.28it/s, gpt_loss=0.0106, lr=2.16e-6]
+
Train step of epoch 0:  60%|█████▉    | 3356/5616 [42:41<29:25,  1.28it/s, gpt_loss=0.00847, lr=2.16e-6]
+
Train step of epoch 0:  60%|█████▉    | 3357/5616 [42:41<34:17,  1.10it/s, gpt_loss=0.00847, lr=2.16e-6]
+
Train step of epoch 0:  60%|█████▉    | 3357/5616 [42:42<34:17,  1.10it/s, gpt_loss=0.0169, lr=2.16e-6] 
+
Train step of epoch 0:  60%|█████▉    | 3358/5616 [42:42<32:32,  1.16it/s, gpt_loss=0.0169, lr=2.16e-6]
+
Train step of epoch 0:  60%|█████▉    | 3358/5616 [42:42<32:32,  1.16it/s, gpt_loss=0.0547, lr=2.16e-6]
+
Train step of epoch 0:  60%|█████▉    | 3359/5616 [42:42<31:27,  1.20it/s, gpt_loss=0.0547, lr=2.16e-6]
+
Train step of epoch 0:  60%|█████▉    | 3359/5616 [42:43<31:27,  1.20it/s, gpt_loss=0.0034, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3360/5616 [42:43<31:00,  1.21it/s, gpt_loss=0.0034, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3360/5616 [42:44<31:00,  1.21it/s, gpt_loss=0.0277, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3361/5616 [42:44<31:07,  1.21it/s, gpt_loss=0.0277, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3361/5616 [42:45<31:07,  1.21it/s, gpt_loss=0.0235, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3362/5616 [42:45<30:46,  1.22it/s, gpt_loss=0.0235, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3362/5616 [42:45<30:46,  1.22it/s, gpt_loss=0.17, lr=2.15e-6]  
+
Train step of epoch 0:  60%|█████▉    | 3363/5616 [42:45<29:20,  1.28it/s, gpt_loss=0.17, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3363/5616 [42:46<29:20,  1.28it/s, gpt_loss=0.00343, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3364/5616 [42:46<28:36,  1.31it/s, gpt_loss=0.00343, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3364/5616 [42:47<28:36,  1.31it/s, gpt_loss=0.0119, lr=2.15e-6] 
+
Train step of epoch 0:  60%|█████▉    | 3365/5616 [42:47<27:34,  1.36it/s, gpt_loss=0.0119, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3365/5616 [42:48<27:34,  1.36it/s, gpt_loss=0.0503, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3366/5616 [42:48<32:05,  1.17it/s, gpt_loss=0.0503, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3366/5616 [42:49<32:05,  1.17it/s, gpt_loss=0.00194, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3367/5616 [42:49<29:55,  1.25it/s, gpt_loss=0.00194, lr=2.15e-6]
+
Train step of epoch 0:  60%|█████▉    | 3367/5616 [42:49<29:55,  1.25it/s, gpt_loss=0.0306, lr=2.14e-6] 
+
Train step of epoch 0:  60%|█████▉    | 3368/5616 [42:49<29:26,  1.27it/s, gpt_loss=0.0306, lr=2.14e-6]
+
Train step of epoch 0:  60%|█████▉    | 3368/5616 [42:50<29:26,  1.27it/s, gpt_loss=0.0231, lr=2.14e-6]
+
Train step of epoch 0:  60%|█████▉    | 3369/5616 [42:50<30:05,  1.24it/s, gpt_loss=0.0231, lr=2.14e-6]
+
Train step of epoch 0:  60%|█████▉    | 3369/5616 [42:51<30:05,  1.24it/s, gpt_loss=0.0126, lr=2.14e-6]
+
Train step of epoch 0:  60%|██████    | 3370/5616 [42:51<28:34,  1.31it/s, gpt_loss=0.0126, lr=2.14e-6]
+
Train step of epoch 0:  60%|██████    | 3370/5616 [42:52<28:34,  1.31it/s, gpt_loss=0.104, lr=2.14e-6] 
+
Train step of epoch 0:  60%|██████    | 3371/5616 [42:52<31:03,  1.20it/s, gpt_loss=0.104, lr=2.14e-6]
+
Train step of epoch 0:  60%|██████    | 3371/5616 [42:53<31:03,  1.20it/s, gpt_loss=0.0889, lr=2.14e-6]
+
Train step of epoch 0:  60%|██████    | 3372/5616 [42:53<29:38,  1.26it/s, gpt_loss=0.0889, lr=2.14e-6]
+
Train step of epoch 0:  60%|██████    | 3372/5616 [42:53<29:38,  1.26it/s, gpt_loss=0.0092, lr=2.14e-6]
+
Train step of epoch 0:  60%|██████    | 3373/5616 [42:53<29:05,  1.29it/s, gpt_loss=0.0092, lr=2.14e-6]
+
Train step of epoch 0:  60%|██████    | 3373/5616 [42:54<29:05,  1.29it/s, gpt_loss=0.0193, lr=2.14e-6]
+
Train step of epoch 0:  60%|██████    | 3374/5616 [42:54<28:17,  1.32it/s, gpt_loss=0.0193, lr=2.14e-6]
+
Train step of epoch 0:  60%|██████    | 3374/5616 [42:55<28:17,  1.32it/s, gpt_loss=0.0446, lr=2.14e-6]
+
Train step of epoch 0:  60%|██████    | 3375/5616 [42:55<27:03,  1.38it/s, gpt_loss=0.0446, lr=2.14e-6]
+
Train step of epoch 0:  60%|██████    | 3375/5616 [42:55<27:03,  1.38it/s, gpt_loss=0.0266, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3376/5616 [42:55<28:12,  1.32it/s, gpt_loss=0.0266, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3376/5616 [42:56<28:12,  1.32it/s, gpt_loss=0.0218, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3377/5616 [42:56<27:59,  1.33it/s, gpt_loss=0.0218, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3377/5616 [42:57<27:59,  1.33it/s, gpt_loss=0.0151, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3378/5616 [42:57<27:24,  1.36it/s, gpt_loss=0.0151, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3378/5616 [42:58<27:24,  1.36it/s, gpt_loss=0.0113, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3379/5616 [42:58<27:16,  1.37it/s, gpt_loss=0.0113, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3379/5616 [42:59<27:16,  1.37it/s, gpt_loss=0.0125, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3380/5616 [42:59<29:46,  1.25it/s, gpt_loss=0.0125, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3380/5616 [42:59<29:46,  1.25it/s, gpt_loss=0.0242, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3381/5616 [42:59<28:43,  1.30it/s, gpt_loss=0.0242, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3381/5616 [43:00<28:43,  1.30it/s, gpt_loss=0.0048, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3382/5616 [43:00<28:27,  1.31it/s, gpt_loss=0.0048, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3382/5616 [43:01<28:27,  1.31it/s, gpt_loss=0.0492, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3383/5616 [43:01<26:55,  1.38it/s, gpt_loss=0.0492, lr=2.13e-6]
+
Train step of epoch 0:  60%|██████    | 3383/5616 [43:02<26:55,  1.38it/s, gpt_loss=0.0692, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3384/5616 [43:02<29:43,  1.25it/s, gpt_loss=0.0692, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3384/5616 [43:02<29:43,  1.25it/s, gpt_loss=0.0365, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3385/5616 [43:02<28:21,  1.31it/s, gpt_loss=0.0365, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3385/5616 [43:03<28:21,  1.31it/s, gpt_loss=0.0359, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3386/5616 [43:03<27:35,  1.35it/s, gpt_loss=0.0359, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3386/5616 [43:04<27:35,  1.35it/s, gpt_loss=0.0226, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3387/5616 [43:04<26:57,  1.38it/s, gpt_loss=0.0226, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3387/5616 [43:04<26:57,  1.38it/s, gpt_loss=0.0149, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3388/5616 [43:04<27:21,  1.36it/s, gpt_loss=0.0149, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3388/5616 [43:05<27:21,  1.36it/s, gpt_loss=0.0263, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3389/5616 [43:05<27:37,  1.34it/s, gpt_loss=0.0263, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3389/5616 [43:06<27:37,  1.34it/s, gpt_loss=0.0159, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3390/5616 [43:06<27:26,  1.35it/s, gpt_loss=0.0159, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3390/5616 [43:07<27:26,  1.35it/s, gpt_loss=0.00835, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3391/5616 [43:07<27:17,  1.36it/s, gpt_loss=0.00835, lr=2.12e-6]
+
Train step of epoch 0:  60%|██████    | 3391/5616 [43:07<27:17,  1.36it/s, gpt_loss=0.00987, lr=2.11e-6]
+
Train step of epoch 0:  60%|██████    | 3392/5616 [43:07<27:33,  1.34it/s, gpt_loss=0.00987, lr=2.11e-6]
+
Train step of epoch 0:  60%|██████    | 3392/5616 [43:08<27:33,  1.34it/s, gpt_loss=0.0429, lr=2.11e-6] 
+
Train step of epoch 0:  60%|██████    | 3393/5616 [43:08<28:06,  1.32it/s, gpt_loss=0.0429, lr=2.11e-6]
+
Train step of epoch 0:  60%|██████    | 3393/5616 [43:09<28:06,  1.32it/s, gpt_loss=0.00335, lr=2.11e-6]
+
Train step of epoch 0:  60%|██████    | 3394/5616 [43:09<27:05,  1.37it/s, gpt_loss=0.00335, lr=2.11e-6]
+
Train step of epoch 0:  60%|██████    | 3394/5616 [43:10<27:05,  1.37it/s, gpt_loss=0.0354, lr=2.11e-6] 
+
Train step of epoch 0:  60%|██████    | 3395/5616 [43:10<27:43,  1.34it/s, gpt_loss=0.0354, lr=2.11e-6]
+
Train step of epoch 0:  60%|██████    | 3395/5616 [43:11<27:43,  1.34it/s, gpt_loss=0.0943, lr=2.11e-6]
+
Train step of epoch 0:  60%|██████    | 3396/5616 [43:11<28:47,  1.29it/s, gpt_loss=0.0943, lr=2.11e-6]
+
Train step of epoch 0:  60%|██████    | 3396/5616 [43:11<28:47,  1.29it/s, gpt_loss=0.00846, lr=2.11e-6]
+
Train step of epoch 0:  60%|██████    | 3397/5616 [43:11<27:28,  1.35it/s, gpt_loss=0.00846, lr=2.11e-6]
+
Train step of epoch 0:  60%|██████    | 3397/5616 [43:12<27:28,  1.35it/s, gpt_loss=0.095, lr=2.11e-6]  
+
Train step of epoch 0:  61%|██████    | 3398/5616 [43:12<27:32,  1.34it/s, gpt_loss=0.095, lr=2.11e-6]
+
Train step of epoch 0:  61%|██████    | 3398/5616 [43:13<27:32,  1.34it/s, gpt_loss=0.0176, lr=2.11e-6]
+
Train step of epoch 0:  61%|██████    | 3399/5616 [43:13<26:40,  1.39it/s, gpt_loss=0.0176, lr=2.11e-6]
+
Train step of epoch 0:  61%|██████    | 3399/5616 [43:14<26:40,  1.39it/s, gpt_loss=0.0138, lr=2.1e-6] 
+
Train step of epoch 0:  61%|██████    | 3400/5616 [43:14<32:41,  1.13it/s, gpt_loss=0.0138, lr=2.1e-6]
+
Train step of epoch 0:  61%|██████    | 3400/5616 [43:15<32:41,  1.13it/s, gpt_loss=0.00311, lr=2.1e-6]
+
Train step of epoch 0:  61%|██████    | 3401/5616 [43:15<31:04,  1.19it/s, gpt_loss=0.00311, lr=2.1e-6]
+
Train step of epoch 0:  61%|██████    | 3401/5616 [43:15<31:04,  1.19it/s, gpt_loss=0.0897, lr=2.1e-6] 
+
Train step of epoch 0:  61%|██████    | 3402/5616 [43:15<29:10,  1.26it/s, gpt_loss=0.0897, lr=2.1e-6]
+
Train step of epoch 0:  61%|██████    | 3402/5616 [43:16<29:10,  1.26it/s, gpt_loss=0.0107, lr=2.1e-6]
+
Train step of epoch 0:  61%|██████    | 3403/5616 [43:16<28:11,  1.31it/s, gpt_loss=0.0107, lr=2.1e-6]
+
Train step of epoch 0:  61%|██████    | 3403/5616 [43:17<28:11,  1.31it/s, gpt_loss=0.00266, lr=2.1e-6]
+
Train step of epoch 0:  61%|██████    | 3404/5616 [43:17<28:17,  1.30it/s, gpt_loss=0.00266, lr=2.1e-6]
+
Train step of epoch 0:  61%|██████    | 3404/5616 [43:17<28:17,  1.30it/s, gpt_loss=0.0131, lr=2.1e-6] 
+
Train step of epoch 0:  61%|██████    | 3405/5616 [43:17<27:29,  1.34it/s, gpt_loss=0.0131, lr=2.1e-6]
+
Train step of epoch 0:  61%|██████    | 3405/5616 [43:18<27:29,  1.34it/s, gpt_loss=0.0016, lr=2.1e-6]
+
Train step of epoch 0:  61%|██████    | 3406/5616 [43:18<26:35,  1.39it/s, gpt_loss=0.0016, lr=2.1e-6]
+
Train step of epoch 0:  61%|██████    | 3406/5616 [43:19<26:35,  1.39it/s, gpt_loss=0.00369, lr=2.1e-6]
+
Train step of epoch 0:  61%|██████    | 3407/5616 [43:19<27:21,  1.35it/s, gpt_loss=0.00369, lr=2.1e-6]
+
Train step of epoch 0:  61%|██████    | 3407/5616 [43:20<27:21,  1.35it/s, gpt_loss=0.0367, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3408/5616 [43:20<27:20,  1.35it/s, gpt_loss=0.0367, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3408/5616 [43:21<27:20,  1.35it/s, gpt_loss=0.119, lr=2.09e-6] 
+
Train step of epoch 0:  61%|██████    | 3409/5616 [43:21<28:22,  1.30it/s, gpt_loss=0.119, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3409/5616 [43:21<28:22,  1.30it/s, gpt_loss=0.0127, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3410/5616 [43:21<29:30,  1.25it/s, gpt_loss=0.0127, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3410/5616 [43:22<29:30,  1.25it/s, gpt_loss=0.0657, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3411/5616 [43:22<28:20,  1.30it/s, gpt_loss=0.0657, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3411/5616 [43:23<28:20,  1.30it/s, gpt_loss=0.121, lr=2.09e-6] 
+
Train step of epoch 0:  61%|██████    | 3412/5616 [43:23<27:59,  1.31it/s, gpt_loss=0.121, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3412/5616 [43:24<27:59,  1.31it/s, gpt_loss=0.0137, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3413/5616 [43:24<27:15,  1.35it/s, gpt_loss=0.0137, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3413/5616 [43:24<27:15,  1.35it/s, gpt_loss=0.0374, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3414/5616 [43:24<27:32,  1.33it/s, gpt_loss=0.0374, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3414/5616 [43:25<27:32,  1.33it/s, gpt_loss=0.0453, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3415/5616 [43:25<26:37,  1.38it/s, gpt_loss=0.0453, lr=2.09e-6]
+
Train step of epoch 0:  61%|██████    | 3415/5616 [43:26<26:37,  1.38it/s, gpt_loss=0.0311, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3416/5616 [43:26<26:59,  1.36it/s, gpt_loss=0.0311, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3416/5616 [43:26<26:59,  1.36it/s, gpt_loss=0.0522, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3417/5616 [43:26<27:09,  1.35it/s, gpt_loss=0.0522, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3417/5616 [43:27<27:09,  1.35it/s, gpt_loss=0.000861, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3418/5616 [43:27<28:14,  1.30it/s, gpt_loss=0.000861, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3418/5616 [43:28<28:14,  1.30it/s, gpt_loss=0.0792, lr=2.08e-6]  
+
Train step of epoch 0:  61%|██████    | 3419/5616 [43:28<27:05,  1.35it/s, gpt_loss=0.0792, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3419/5616 [43:29<27:05,  1.35it/s, gpt_loss=0.0807, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3420/5616 [43:29<27:32,  1.33it/s, gpt_loss=0.0807, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3420/5616 [43:29<27:32,  1.33it/s, gpt_loss=0.00706, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3421/5616 [43:29<26:58,  1.36it/s, gpt_loss=0.00706, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3421/5616 [43:30<26:58,  1.36it/s, gpt_loss=0.032, lr=2.08e-6]  
+
Train step of epoch 0:  61%|██████    | 3422/5616 [43:30<26:16,  1.39it/s, gpt_loss=0.032, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3422/5616 [43:31<26:16,  1.39it/s, gpt_loss=0.089, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3423/5616 [43:31<25:53,  1.41it/s, gpt_loss=0.089, lr=2.08e-6]
+
Train step of epoch 0:  61%|██████    | 3423/5616 [43:32<25:53,  1.41it/s, gpt_loss=0.0259, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3424/5616 [43:32<27:06,  1.35it/s, gpt_loss=0.0259, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3424/5616 [43:32<27:06,  1.35it/s, gpt_loss=0.0265, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3425/5616 [43:32<27:07,  1.35it/s, gpt_loss=0.0265, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3425/5616 [43:33<27:07,  1.35it/s, gpt_loss=0.0149, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3426/5616 [43:33<27:23,  1.33it/s, gpt_loss=0.0149, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3426/5616 [43:34<27:23,  1.33it/s, gpt_loss=0.0925, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3427/5616 [43:34<26:12,  1.39it/s, gpt_loss=0.0925, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3427/5616 [43:35<26:12,  1.39it/s, gpt_loss=0.0153, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3428/5616 [43:35<26:50,  1.36it/s, gpt_loss=0.0153, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3428/5616 [43:35<26:50,  1.36it/s, gpt_loss=0.0993, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3429/5616 [43:35<27:04,  1.35it/s, gpt_loss=0.0993, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3429/5616 [43:36<27:04,  1.35it/s, gpt_loss=0.0239, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3430/5616 [43:36<27:30,  1.32it/s, gpt_loss=0.0239, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3430/5616 [43:37<27:30,  1.32it/s, gpt_loss=0.0753, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3431/5616 [43:37<26:44,  1.36it/s, gpt_loss=0.0753, lr=2.07e-6]
+
Train step of epoch 0:  61%|██████    | 3431/5616 [43:38<26:44,  1.36it/s, gpt_loss=0.0041, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3432/5616 [43:38<27:23,  1.33it/s, gpt_loss=0.0041, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3432/5616 [43:38<27:23,  1.33it/s, gpt_loss=0.0284, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3433/5616 [43:38<27:09,  1.34it/s, gpt_loss=0.0284, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3433/5616 [43:39<27:09,  1.34it/s, gpt_loss=0.0678, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3434/5616 [43:39<27:57,  1.30it/s, gpt_loss=0.0678, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3434/5616 [43:40<27:57,  1.30it/s, gpt_loss=0.117, lr=2.06e-6] 
+
Train step of epoch 0:  61%|██████    | 3435/5616 [43:40<28:13,  1.29it/s, gpt_loss=0.117, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3435/5616 [43:41<28:13,  1.29it/s, gpt_loss=0.00811, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3436/5616 [43:41<28:40,  1.27it/s, gpt_loss=0.00811, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3436/5616 [43:41<28:40,  1.27it/s, gpt_loss=0.0243, lr=2.06e-6] 
+
Train step of epoch 0:  61%|██████    | 3437/5616 [43:41<27:52,  1.30it/s, gpt_loss=0.0243, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3437/5616 [43:42<27:52,  1.30it/s, gpt_loss=0.0192, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3438/5616 [43:42<27:25,  1.32it/s, gpt_loss=0.0192, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3438/5616 [43:43<27:25,  1.32it/s, gpt_loss=0.00505, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3439/5616 [43:43<28:21,  1.28it/s, gpt_loss=0.00505, lr=2.06e-6]
+
Train step of epoch 0:  61%|██████    | 3439/5616 [43:44<28:21,  1.28it/s, gpt_loss=0.0874, lr=2.05e-6] 
+
Train step of epoch 0:  61%|██████▏   | 3440/5616 [43:44<28:03,  1.29it/s, gpt_loss=0.0874, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3440/5616 [43:45<28:03,  1.29it/s, gpt_loss=0.0534, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3441/5616 [43:45<28:14,  1.28it/s, gpt_loss=0.0534, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3441/5616 [43:45<28:14,  1.28it/s, gpt_loss=0.0314, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3442/5616 [43:45<28:55,  1.25it/s, gpt_loss=0.0314, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3442/5616 [43:46<28:55,  1.25it/s, gpt_loss=0.00616, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3443/5616 [43:46<29:13,  1.24it/s, gpt_loss=0.00616, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3443/5616 [43:47<29:13,  1.24it/s, gpt_loss=0.0374, lr=2.05e-6] 
+
Train step of epoch 0:  61%|██████▏   | 3444/5616 [43:47<28:39,  1.26it/s, gpt_loss=0.0374, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3444/5616 [43:48<28:39,  1.26it/s, gpt_loss=0.0088, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3445/5616 [43:48<27:56,  1.30it/s, gpt_loss=0.0088, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3445/5616 [43:49<27:56,  1.30it/s, gpt_loss=0.0175, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3446/5616 [43:49<27:43,  1.30it/s, gpt_loss=0.0175, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3446/5616 [43:49<27:43,  1.30it/s, gpt_loss=0.0227, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3447/5616 [43:49<26:38,  1.36it/s, gpt_loss=0.0227, lr=2.05e-6]
+
Train step of epoch 0:  61%|██████▏   | 3447/5616 [43:50<26:38,  1.36it/s, gpt_loss=0.0438, lr=2.04e-6]
+
Train step of epoch 0:  61%|██████▏   | 3448/5616 [43:50<27:43,  1.30it/s, gpt_loss=0.0438, lr=2.04e-6]
+
Train step of epoch 0:  61%|██████▏   | 3448/5616 [43:51<27:43,  1.30it/s, gpt_loss=0.0436, lr=2.04e-6]
+
Train step of epoch 0:  61%|██████▏   | 3449/5616 [43:51<27:50,  1.30it/s, gpt_loss=0.0436, lr=2.04e-6]
+
Train step of epoch 0:  61%|██████▏   | 3449/5616 [43:52<27:50,  1.30it/s, gpt_loss=0.0506, lr=2.04e-6]
+
Train step of epoch 0:  61%|██████▏   | 3450/5616 [43:52<27:08,  1.33it/s, gpt_loss=0.0506, lr=2.04e-6]
+
Train step of epoch 0:  61%|██████▏   | 3450/5616 [43:52<27:08,  1.33it/s, gpt_loss=0.094, lr=2.04e-6] 
+
Train step of epoch 0:  61%|██████▏   | 3451/5616 [43:52<27:59,  1.29it/s, gpt_loss=0.094, lr=2.04e-6]
+
Train step of epoch 0:  61%|██████▏   | 3451/5616 [43:53<27:59,  1.29it/s, gpt_loss=0.0994, lr=2.04e-6]
+
Train step of epoch 0:  61%|██████▏   | 3452/5616 [43:53<29:04,  1.24it/s, gpt_loss=0.0994, lr=2.04e-6]
+
Train step of epoch 0:  61%|██████▏   | 3452/5616 [43:54<29:04,  1.24it/s, gpt_loss=0.0586, lr=2.04e-6]
+
Train step of epoch 0:  61%|██████▏   | 3453/5616 [43:54<29:17,  1.23it/s, gpt_loss=0.0586, lr=2.04e-6]
+
Train step of epoch 0:  61%|██████▏   | 3453/5616 [43:55<29:17,  1.23it/s, gpt_loss=0.036, lr=2.04e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3454/5616 [43:55<28:17,  1.27it/s, gpt_loss=0.036, lr=2.04e-6]
+
Train step of epoch 0:  62%|██████▏   | 3454/5616 [43:56<28:17,  1.27it/s, gpt_loss=0.0406, lr=2.04e-6]
+
Train step of epoch 0:  62%|██████▏   | 3455/5616 [43:56<28:42,  1.25it/s, gpt_loss=0.0406, lr=2.04e-6]
+
Train step of epoch 0:  62%|██████▏   | 3455/5616 [43:56<28:42,  1.25it/s, gpt_loss=0.035, lr=2.03e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3456/5616 [43:56<28:52,  1.25it/s, gpt_loss=0.035, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3456/5616 [43:57<28:52,  1.25it/s, gpt_loss=0.0833, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3457/5616 [43:57<28:06,  1.28it/s, gpt_loss=0.0833, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3457/5616 [43:58<28:06,  1.28it/s, gpt_loss=0.0659, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3458/5616 [43:58<27:27,  1.31it/s, gpt_loss=0.0659, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3458/5616 [43:59<27:27,  1.31it/s, gpt_loss=0.0364, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3459/5616 [43:59<26:22,  1.36it/s, gpt_loss=0.0364, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3459/5616 [43:59<26:22,  1.36it/s, gpt_loss=0.0202, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3460/5616 [43:59<26:30,  1.36it/s, gpt_loss=0.0202, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3460/5616 [44:00<26:30,  1.36it/s, gpt_loss=0.00373, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3461/5616 [44:00<25:57,  1.38it/s, gpt_loss=0.00373, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3461/5616 [44:01<25:57,  1.38it/s, gpt_loss=0.0142, lr=2.03e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3462/5616 [44:01<25:07,  1.43it/s, gpt_loss=0.0142, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3462/5616 [44:01<25:07,  1.43it/s, gpt_loss=0.0467, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3463/5616 [44:01<25:01,  1.43it/s, gpt_loss=0.0467, lr=2.03e-6]
+
Train step of epoch 0:  62%|██████▏   | 3463/5616 [44:02<25:01,  1.43it/s, gpt_loss=0.0131, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3464/5616 [44:02<26:03,  1.38it/s, gpt_loss=0.0131, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3464/5616 [44:03<26:03,  1.38it/s, gpt_loss=0.052, lr=2.02e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3465/5616 [44:03<26:00,  1.38it/s, gpt_loss=0.052, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3465/5616 [44:04<26:00,  1.38it/s, gpt_loss=0.0271, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3466/5616 [44:04<26:08,  1.37it/s, gpt_loss=0.0271, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3466/5616 [44:04<26:08,  1.37it/s, gpt_loss=0.0117, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3467/5616 [44:04<26:12,  1.37it/s, gpt_loss=0.0117, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3467/5616 [44:05<26:12,  1.37it/s, gpt_loss=0.006, lr=2.02e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3468/5616 [44:05<27:27,  1.30it/s, gpt_loss=0.006, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3468/5616 [44:06<27:27,  1.30it/s, gpt_loss=0.0556, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3469/5616 [44:06<27:00,  1.32it/s, gpt_loss=0.0556, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3469/5616 [44:07<27:00,  1.32it/s, gpt_loss=0.0144, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3470/5616 [44:07<25:43,  1.39it/s, gpt_loss=0.0144, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3470/5616 [44:07<25:43,  1.39it/s, gpt_loss=0.0532, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3471/5616 [44:07<26:27,  1.35it/s, gpt_loss=0.0532, lr=2.02e-6]
+
Train step of epoch 0:  62%|██████▏   | 3471/5616 [44:08<26:27,  1.35it/s, gpt_loss=0.0199, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3472/5616 [44:08<26:54,  1.33it/s, gpt_loss=0.0199, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3472/5616 [44:09<26:54,  1.33it/s, gpt_loss=0.054, lr=2.01e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3473/5616 [44:09<27:54,  1.28it/s, gpt_loss=0.054, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3473/5616 [44:10<27:54,  1.28it/s, gpt_loss=0.0367, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3474/5616 [44:10<26:51,  1.33it/s, gpt_loss=0.0367, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3474/5616 [44:10<26:51,  1.33it/s, gpt_loss=0.0285, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3475/5616 [44:10<28:10,  1.27it/s, gpt_loss=0.0285, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3475/5616 [44:11<28:10,  1.27it/s, gpt_loss=0.0424, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3476/5616 [44:11<29:02,  1.23it/s, gpt_loss=0.0424, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3476/5616 [44:12<29:02,  1.23it/s, gpt_loss=0.0874, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3477/5616 [44:12<27:47,  1.28it/s, gpt_loss=0.0874, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3477/5616 [44:13<27:47,  1.28it/s, gpt_loss=0.0304, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3478/5616 [44:13<26:01,  1.37it/s, gpt_loss=0.0304, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3478/5616 [44:14<26:01,  1.37it/s, gpt_loss=0.0104, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3479/5616 [44:14<29:34,  1.20it/s, gpt_loss=0.0104, lr=2.01e-6]
+
Train step of epoch 0:  62%|██████▏   | 3479/5616 [44:15<29:34,  1.20it/s, gpt_loss=0.0533, lr=2e-6]   
+
Train step of epoch 0:  62%|██████▏   | 3480/5616 [44:15<33:03,  1.08it/s, gpt_loss=0.0533, lr=2e-6]
+
Train step of epoch 0:  62%|██████▏   | 3480/5616 [44:16<33:03,  1.08it/s, gpt_loss=0.0315, lr=2e-6]
+
Train step of epoch 0:  62%|██████▏   | 3481/5616 [44:16<31:48,  1.12it/s, gpt_loss=0.0315, lr=2e-6]
+
Train step of epoch 0:  62%|██████▏   | 3481/5616 [44:16<31:48,  1.12it/s, gpt_loss=0.057, lr=2e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3482/5616 [44:16<29:23,  1.21it/s, gpt_loss=0.057, lr=2e-6]
+
Train step of epoch 0:  62%|██████▏   | 3482/5616 [44:17<29:23,  1.21it/s, gpt_loss=0.0333, lr=2e-6]
+
Train step of epoch 0:  62%|██████▏   | 3483/5616 [44:17<27:44,  1.28it/s, gpt_loss=0.0333, lr=2e-6]
+
Train step of epoch 0:  62%|██████▏   | 3483/5616 [44:18<27:44,  1.28it/s, gpt_loss=0.123, lr=2e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3484/5616 [44:18<31:55,  1.11it/s, gpt_loss=0.123, lr=2e-6]
+
Train step of epoch 0:  62%|██████▏   | 3484/5616 [44:19<31:55,  1.11it/s, gpt_loss=0.0271, lr=2e-6]
+
Train step of epoch 0:  62%|██████▏   | 3485/5616 [44:19<29:25,  1.21it/s, gpt_loss=0.0271, lr=2e-6]
+
Train step of epoch 0:  62%|██████▏   | 3485/5616 [44:20<29:25,  1.21it/s, gpt_loss=0.0317, lr=2e-6]
+
Train step of epoch 0:  62%|██████▏   | 3486/5616 [44:20<29:04,  1.22it/s, gpt_loss=0.0317, lr=2e-6]
+
Train step of epoch 0:  62%|██████▏   | 3486/5616 [44:20<29:04,  1.22it/s, gpt_loss=0.142, lr=2e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3487/5616 [44:20<26:57,  1.32it/s, gpt_loss=0.142, lr=2e-6]
+
Train step of epoch 0:  62%|██████▏   | 3487/5616 [44:21<26:57,  1.32it/s, gpt_loss=0.0997, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3488/5616 [44:21<27:45,  1.28it/s, gpt_loss=0.0997, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3488/5616 [44:22<27:45,  1.28it/s, gpt_loss=0.0452, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3489/5616 [44:22<26:05,  1.36it/s, gpt_loss=0.0452, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3489/5616 [44:22<26:05,  1.36it/s, gpt_loss=0.0268, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3490/5616 [44:22<25:02,  1.42it/s, gpt_loss=0.0268, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3490/5616 [44:23<25:02,  1.42it/s, gpt_loss=0.0276, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3491/5616 [44:23<24:45,  1.43it/s, gpt_loss=0.0276, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3491/5616 [44:24<24:45,  1.43it/s, gpt_loss=0.0347, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3492/5616 [44:24<26:15,  1.35it/s, gpt_loss=0.0347, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3492/5616 [44:25<26:15,  1.35it/s, gpt_loss=0.137, lr=1.99e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3493/5616 [44:25<26:09,  1.35it/s, gpt_loss=0.137, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3493/5616 [44:25<26:09,  1.35it/s, gpt_loss=0.0502, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3494/5616 [44:25<25:45,  1.37it/s, gpt_loss=0.0502, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3494/5616 [44:26<25:45,  1.37it/s, gpt_loss=0.0944, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3495/5616 [44:26<25:19,  1.40it/s, gpt_loss=0.0944, lr=1.99e-6]
+
Train step of epoch 0:  62%|██████▏   | 3495/5616 [44:27<25:19,  1.40it/s, gpt_loss=0.057, lr=1.98e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3496/5616 [44:27<26:27,  1.34it/s, gpt_loss=0.057, lr=1.98e-6]
+
Train step of epoch 0:  62%|██████▏   | 3496/5616 [44:28<26:27,  1.34it/s, gpt_loss=0.0649, lr=1.98e-6]
+
Train step of epoch 0:  62%|██████▏   | 3497/5616 [44:28<26:36,  1.33it/s, gpt_loss=0.0649, lr=1.98e-6]
+
Train step of epoch 0:  62%|██████▏   | 3497/5616 [44:28<26:36,  1.33it/s, gpt_loss=0.041, lr=1.98e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3498/5616 [44:28<26:09,  1.35it/s, gpt_loss=0.041, lr=1.98e-6]
+
Train step of epoch 0:  62%|██████▏   | 3498/5616 [44:29<26:09,  1.35it/s, gpt_loss=0.00801, lr=1.98e-6]
+
Train step of epoch 0:  62%|██████▏   | 3499/5616 [44:29<24:50,  1.42it/s, gpt_loss=0.00801, lr=1.98e-6]
+
Train step of epoch 0:  62%|██████▏   | 3499/5616 [44:30<24:50,  1.42it/s, gpt_loss=0.0142, lr=1.98e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3500/5616 [44:30<25:53,  1.36it/s, gpt_loss=0.0142, lr=1.98e-6]
+
Train step of epoch 0:  62%|██████▏   | 3500/5616 [44:31<25:53,  1.36it/s, gpt_loss=0.0138, lr=1.98e-6]
+
Train step of epoch 0:  62%|██████▏   | 3501/5616 [44:31<26:07,  1.35it/s, gpt_loss=0.0138, lr=1.98e-6]
+
Train step of epoch 0:  62%|██████▏   | 3501/5616 [44:31<26:07,  1.35it/s, gpt_loss=0.0322, lr=1.98e-6]
+
Train step of epoch 0:  62%|██████▏   | 3502/5616 [44:31<26:07,  1.35it/s, gpt_loss=0.0322, lr=1.98e-6]
+
Train step of epoch 0:  62%|██████▏   | 3502/5616 [44:32<26:07,  1.35it/s, gpt_loss=0.0133, lr=1.98e-6]
+
Train step of epoch 0:  62%|██████▏   | 3503/5616 [44:32<26:50,  1.31it/s, gpt_loss=0.0133, lr=1.98e-6]
+
Train step of epoch 0:  62%|██████▏   | 3503/5616 [44:33<26:50,  1.31it/s, gpt_loss=0.0559, lr=1.97e-6]
+
Train step of epoch 0:  62%|██████▏   | 3504/5616 [44:33<27:24,  1.28it/s, gpt_loss=0.0559, lr=1.97e-6]
+
Train step of epoch 0:  62%|██████▏   | 3504/5616 [44:34<27:24,  1.28it/s, gpt_loss=0.121, lr=1.97e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3505/5616 [44:34<27:17,  1.29it/s, gpt_loss=0.121, lr=1.97e-6]
+
Train step of epoch 0:  62%|██████▏   | 3505/5616 [44:35<27:17,  1.29it/s, gpt_loss=0.00876, lr=1.97e-6]
+
Train step of epoch 0:  62%|██████▏   | 3506/5616 [44:35<29:00,  1.21it/s, gpt_loss=0.00876, lr=1.97e-6]
+
Train step of epoch 0:  62%|██████▏   | 3506/5616 [44:35<29:00,  1.21it/s, gpt_loss=0.0336, lr=1.97e-6] 
+
Train step of epoch 0:  62%|██████▏   | 3507/5616 [44:35<27:53,  1.26it/s, gpt_loss=0.0336, lr=1.97e-6]
+
Train step of epoch 0:  62%|██████▏   | 3507/5616 [44:36<27:53,  1.26it/s, gpt_loss=0.0715, lr=1.97e-6]
+
Train step of epoch 0:  62%|██████▏   | 3508/5616 [44:36<28:50,  1.22it/s, gpt_loss=0.0715, lr=1.97e-6]
+
Train step of epoch 0:  62%|██████▏   | 3508/5616 [44:37<28:50,  1.22it/s, gpt_loss=0.0441, lr=1.97e-6]
+
Train step of epoch 0:  62%|██████▏   | 3509/5616 [44:37<28:45,  1.22it/s, gpt_loss=0.0441, lr=1.97e-6]
+
Train step of epoch 0:  62%|██████▏   | 3509/5616 [44:38<28:45,  1.22it/s, gpt_loss=0.00718, lr=1.97e-6]
+
Train step of epoch 0:  62%|██████▎   | 3510/5616 [44:38<28:00,  1.25it/s, gpt_loss=0.00718, lr=1.97e-6]
+
Train step of epoch 0:  62%|██████▎   | 3510/5616 [44:38<28:00,  1.25it/s, gpt_loss=0.0057, lr=1.97e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3511/5616 [44:38<26:19,  1.33it/s, gpt_loss=0.0057, lr=1.97e-6]
+
Train step of epoch 0:  63%|██████▎   | 3511/5616 [44:39<26:19,  1.33it/s, gpt_loss=0.0498, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3512/5616 [44:39<26:53,  1.30it/s, gpt_loss=0.0498, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3512/5616 [44:40<26:53,  1.30it/s, gpt_loss=0.00388, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3513/5616 [44:40<25:37,  1.37it/s, gpt_loss=0.00388, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3513/5616 [44:41<25:37,  1.37it/s, gpt_loss=0.0201, lr=1.96e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3514/5616 [44:41<25:30,  1.37it/s, gpt_loss=0.0201, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3514/5616 [44:42<25:30,  1.37it/s, gpt_loss=0.00927, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3515/5616 [44:42<28:12,  1.24it/s, gpt_loss=0.00927, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3515/5616 [44:42<28:12,  1.24it/s, gpt_loss=0.0414, lr=1.96e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3516/5616 [44:42<29:11,  1.20it/s, gpt_loss=0.0414, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3516/5616 [44:43<29:11,  1.20it/s, gpt_loss=0.0203, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3517/5616 [44:43<27:59,  1.25it/s, gpt_loss=0.0203, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3517/5616 [44:44<27:59,  1.25it/s, gpt_loss=0.0397, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3518/5616 [44:44<29:22,  1.19it/s, gpt_loss=0.0397, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3518/5616 [44:45<29:22,  1.19it/s, gpt_loss=0.0112, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3519/5616 [44:45<28:13,  1.24it/s, gpt_loss=0.0112, lr=1.96e-6]
+
Train step of epoch 0:  63%|██████▎   | 3519/5616 [44:46<28:13,  1.24it/s, gpt_loss=0.0113, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3520/5616 [44:46<27:32,  1.27it/s, gpt_loss=0.0113, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3520/5616 [44:46<27:32,  1.27it/s, gpt_loss=0.0379, lr=1.95e-6]
+
Train step of epoch 0:  63%|████���█▎   | 3521/5616 [44:46<28:13,  1.24it/s, gpt_loss=0.0379, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3521/5616 [44:48<28:13,  1.24it/s, gpt_loss=0.0324, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3522/5616 [44:48<32:24,  1.08it/s, gpt_loss=0.0324, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3522/5616 [44:48<32:24,  1.08it/s, gpt_loss=0.038, lr=1.95e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3523/5616 [44:48<29:49,  1.17it/s, gpt_loss=0.038, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3523/5616 [44:49<29:49,  1.17it/s, gpt_loss=0.0139, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3524/5616 [44:49<28:45,  1.21it/s, gpt_loss=0.0139, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3524/5616 [44:50<28:45,  1.21it/s, gpt_loss=0.0186, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3525/5616 [44:50<28:11,  1.24it/s, gpt_loss=0.0186, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3525/5616 [44:51<28:11,  1.24it/s, gpt_loss=0.15, lr=1.95e-6]  
+
Train step of epoch 0:  63%|██████▎   | 3526/5616 [44:51<27:26,  1.27it/s, gpt_loss=0.15, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3526/5616 [44:51<27:26,  1.27it/s, gpt_loss=0.139, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3527/5616 [44:51<26:35,  1.31it/s, gpt_loss=0.139, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3527/5616 [44:52<26:35,  1.31it/s, gpt_loss=0.0263, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3528/5616 [44:52<26:31,  1.31it/s, gpt_loss=0.0263, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3528/5616 [44:53<26:31,  1.31it/s, gpt_loss=0.0663, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3529/5616 [44:53<26:01,  1.34it/s, gpt_loss=0.0663, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3529/5616 [44:54<26:01,  1.34it/s, gpt_loss=0.00398, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3530/5616 [44:54<25:29,  1.36it/s, gpt_loss=0.00398, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3530/5616 [44:54<25:29,  1.36it/s, gpt_loss=0.0319, lr=1.95e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3531/5616 [44:54<26:46,  1.30it/s, gpt_loss=0.0319, lr=1.95e-6]
+
Train step of epoch 0:  63%|██████▎   | 3531/5616 [44:55<26:46,  1.30it/s, gpt_loss=0.0605, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3532/5616 [44:55<28:11,  1.23it/s, gpt_loss=0.0605, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3532/5616 [44:56<28:11,  1.23it/s, gpt_loss=0.0625, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3533/5616 [44:56<26:49,  1.29it/s, gpt_loss=0.0625, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3533/5616 [44:57<26:49,  1.29it/s, gpt_loss=0.00767, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3534/5616 [44:57<25:50,  1.34it/s, gpt_loss=0.00767, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3534/5616 [44:57<25:50,  1.34it/s, gpt_loss=0.0406, lr=1.94e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3535/5616 [44:57<24:57,  1.39it/s, gpt_loss=0.0406, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3535/5616 [44:58<24:57,  1.39it/s, gpt_loss=0.0229, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3536/5616 [44:58<26:04,  1.33it/s, gpt_loss=0.0229, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3536/5616 [44:59<26:04,  1.33it/s, gpt_loss=0.00296, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3537/5616 [44:59<24:59,  1.39it/s, gpt_loss=0.00296, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3537/5616 [44:59<24:59,  1.39it/s, gpt_loss=0.0938, lr=1.94e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3538/5616 [44:59<24:04,  1.44it/s, gpt_loss=0.0938, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3538/5616 [45:00<24:04,  1.44it/s, gpt_loss=0.0392, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3539/5616 [45:00<23:16,  1.49it/s, gpt_loss=0.0392, lr=1.94e-6]
+
Train step of epoch 0:  63%|██████▎   | 3539/5616 [45:01<23:16,  1.49it/s, gpt_loss=0.0089, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3540/5616 [45:01<24:50,  1.39it/s, gpt_loss=0.0089, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3540/5616 [45:02<24:50,  1.39it/s, gpt_loss=0.0945, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3541/5616 [45:02<25:58,  1.33it/s, gpt_loss=0.0945, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3541/5616 [45:02<25:58,  1.33it/s, gpt_loss=0.0372, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3542/5616 [45:02<26:16,  1.32it/s, gpt_loss=0.0372, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3542/5616 [45:03<26:16,  1.32it/s, gpt_loss=0.0261, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3543/5616 [45:03<24:55,  1.39it/s, gpt_loss=0.0261, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3543/5616 [45:04<24:55,  1.39it/s, gpt_loss=0.0195, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3544/5616 [45:04<26:27,  1.30it/s, gpt_loss=0.0195, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3544/5616 [45:05<26:27,  1.30it/s, gpt_loss=0.025, lr=1.93e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3545/5616 [45:05<25:58,  1.33it/s, gpt_loss=0.025, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3545/5616 [45:06<25:58,  1.33it/s, gpt_loss=0.0434, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3546/5616 [45:06<27:43,  1.24it/s, gpt_loss=0.0434, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3546/5616 [45:06<27:43,  1.24it/s, gpt_loss=0.00341, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3547/5616 [45:06<28:07,  1.23it/s, gpt_loss=0.00341, lr=1.93e-6]
+
Train step of epoch 0:  63%|██████▎   | 3547/5616 [45:07<28:07,  1.23it/s, gpt_loss=0.0383, lr=1.92e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3548/5616 [45:07<28:57,  1.19it/s, gpt_loss=0.0383, lr=1.92e-6]
+
Train step of epoch 0:  63%|██████▎   | 3548/5616 [45:08<28:57,  1.19it/s, gpt_loss=0.0372, lr=1.92e-6]
+
Train step of epoch 0:  63%|██████▎   | 3549/5616 [45:08<28:00,  1.23it/s, gpt_loss=0.0372, lr=1.92e-6]
+
Train step of epoch 0:  63%|██████▎   | 3549/5616 [45:09<28:00,  1.23it/s, gpt_loss=0.00149, lr=1.92e-6]
+
Train step of epoch 0:  63%|██████▎   | 3550/5616 [45:09<26:14,  1.31it/s, gpt_loss=0.00149, lr=1.92e-6]
+
Train step of epoch 0:  63%|██████▎   | 3550/5616 [45:10<26:14,  1.31it/s, gpt_loss=0.0137, lr=1.92e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3551/5616 [45:10<26:19,  1.31it/s, gpt_loss=0.0137, lr=1.92e-6]
+
Train step of epoch 0:  63%|██████▎   | 3551/5616 [45:10<26:19,  1.31it/s, gpt_loss=0.0104, lr=1.92e-6]
+
Train step of epoch 0:  63%|██████▎   | 3552/5616 [45:10<26:25,  1.30it/s, gpt_loss=0.0104, lr=1.92e-6]
+
Train step of epoch 0:  63%|██████▎   | 3552/5616 [45:11<26:25,  1.30it/s, gpt_loss=0.0292, lr=1.92e-6]
+
Train step of epoch 0:  63%|██████▎   | 3553/5616 [45:11<25:31,  1.35it/s, gpt_loss=0.0292, lr=1.92e-6]
+
Train step of epoch 0:  63%|██████▎   | 3553/5616 [45:12<25:31,  1.35it/s, gpt_loss=0.148, lr=1.92e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3554/5616 [45:12<26:31,  1.30it/s, gpt_loss=0.148, lr=1.92e-6]
+
Train step of epoch 0:  63%|██████▎   | 3554/5616 [45:13<26:31,  1.30it/s, gpt_loss=0.0233, lr=1.92e-6]
+
Train step of epoch 0:  63%|██████▎   | 3555/5616 [45:13<25:47,  1.33it/s, gpt_loss=0.0233, lr=1.92e-6]
+
Train step of epoch 0:  63%|██████▎   | 3555/5616 [45:13<25:47,  1.33it/s, gpt_loss=0.043, lr=1.91e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3556/5616 [45:13<25:42,  1.34it/s, gpt_loss=0.043, lr=1.91e-6]
+
Train step of epoch 0:  63%|██████▎   | 3556/5616 [45:14<25:42,  1.34it/s, gpt_loss=0.0592, lr=1.91e-6]
+
Train step of epoch 0:  63%|██████▎   | 3557/5616 [45:14<26:09,  1.31it/s, gpt_loss=0.0592, lr=1.91e-6]
+
Train step of epoch 0:  63%|██████▎   | 3557/5616 [45:15<26:09,  1.31it/s, gpt_loss=0.0302, lr=1.91e-6]
+
Train step of epoch 0:  63%|██████▎   | 3558/5616 [45:15<25:17,  1.36it/s, gpt_loss=0.0302, lr=1.91e-6]
+
Train step of epoch 0:  63%|██████▎   | 3558/5616 [45:15<25:17,  1.36it/s, gpt_loss=0.0476, lr=1.91e-6]
+
Train step of epoch 0:  63%|██████▎   | 3559/5616 [45:15<25:10,  1.36it/s, gpt_loss=0.0476, lr=1.91e-6]
+
Train step of epoch 0:  63%|██████▎   | 3559/5616 [45:16<25:10,  1.36it/s, gpt_loss=0.032, lr=1.91e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3560/5616 [45:16<26:12,  1.31it/s, gpt_loss=0.032, lr=1.91e-6]
+
Train step of epoch 0:  63%|██████▎   | 3560/5616 [45:17<26:12,  1.31it/s, gpt_loss=0.0112, lr=1.91e-6]
+
Train step of epoch 0:  63%|██████▎   | 3561/5616 [45:17<25:47,  1.33it/s, gpt_loss=0.0112, lr=1.91e-6]
+
Train step of epoch 0:  63%|██████▎   | 3561/5616 [45:18<25:47,  1.33it/s, gpt_loss=0.11, lr=1.91e-6]  
+
Train step of epoch 0:  63%|██████▎   | 3562/5616 [45:18<25:04,  1.36it/s, gpt_loss=0.11, lr=1.91e-6]
+
Train step of epoch 0:  63%|██████▎   | 3562/5616 [45:18<25:04,  1.36it/s, gpt_loss=0.0107, lr=1.91e-6]
+
Train step of epoch 0:  63%|██████▎   | 3563/5616 [45:18<24:32,  1.39it/s, gpt_loss=0.0107, lr=1.91e-6]
+
Train step of epoch 0:  63%|██████▎   | 3563/5616 [45:19<24:32,  1.39it/s, gpt_loss=0.0406, lr=1.9e-6] 
+
Train step of epoch 0:  63%|██████▎   | 3564/5616 [45:19<26:11,  1.31it/s, gpt_loss=0.0406, lr=1.9e-6]
+
Train step of epoch 0:  63%|██████▎   | 3564/5616 [45:20<26:11,  1.31it/s, gpt_loss=0.00881, lr=1.9e-6]
+
Train step of epoch 0:  63%|██████▎   | 3565/5616 [45:20<26:49,  1.27it/s, gpt_loss=0.00881, lr=1.9e-6]
+
Train step of epoch 0:  63%|██████▎   | 3565/5616 [45:21<26:49,  1.27it/s, gpt_loss=0.01, lr=1.9e-6]   
+
Train step of epoch 0:  63%|██████▎   | 3566/5616 [45:21<25:45,  1.33it/s, gpt_loss=0.01, lr=1.9e-6]
+
Train step of epoch 0:  63%|██████▎   | 3566/5616 [45:21<25:45,  1.33it/s, gpt_loss=0.00568, lr=1.9e-6]
+
Train step of epoch 0:  64%|██████▎   | 3567/5616 [45:21<25:16,  1.35it/s, gpt_loss=0.00568, lr=1.9e-6]
+
Train step of epoch 0:  64%|██████▎   | 3567/5616 [45:22<25:16,  1.35it/s, gpt_loss=0.16, lr=1.9e-6]   
+
Train step of epoch 0:  64%|██████▎   | 3568/5616 [45:22<25:57,  1.31it/s, gpt_loss=0.16, lr=1.9e-6]
+
Train step of epoch 0:  64%|██████▎   | 3568/5616 [45:23<25:57,  1.31it/s, gpt_loss=0.0737, lr=1.9e-6]
+
Train step of epoch 0:  64%|██████▎   | 3569/5616 [45:23<25:30,  1.34it/s, gpt_loss=0.0737, lr=1.9e-6]
+
Train step of epoch 0:  64%|██████▎   | 3569/5616 [45:24<25:30,  1.34it/s, gpt_loss=0.0176, lr=1.9e-6]
+
Train step of epoch 0:  64%|██████▎   | 3570/5616 [45:24<24:52,  1.37it/s, gpt_loss=0.0176, lr=1.9e-6]
+
Train step of epoch 0:  64%|██████▎   | 3570/5616 [45:24<24:52,  1.37it/s, gpt_loss=0.0149, lr=1.9e-6]
+
Train step of epoch 0:  64%|██████▎   | 3571/5616 [45:24<24:47,  1.37it/s, gpt_loss=0.0149, lr=1.9e-6]
+
Train step of epoch 0:  64%|██████▎   | 3571/5616 [45:25<24:47,  1.37it/s, gpt_loss=0.0084, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3572/5616 [45:25<25:33,  1.33it/s, gpt_loss=0.0084, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3572/5616 [45:26<25:33,  1.33it/s, gpt_loss=0.0394, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3573/5616 [45:26<25:35,  1.33it/s, gpt_loss=0.0394, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3573/5616 [45:27<25:35,  1.33it/s, gpt_loss=0.0228, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3574/5616 [45:27<26:04,  1.31it/s, gpt_loss=0.0228, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3574/5616 [45:28<26:04,  1.31it/s, gpt_loss=0.0562, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3575/5616 [45:28<26:04,  1.30it/s, gpt_loss=0.0562, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3575/5616 [45:28<26:04,  1.30it/s, gpt_loss=0.00695, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3576/5616 [45:28<25:47,  1.32it/s, gpt_loss=0.00695, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3576/5616 [45:29<25:47,  1.32it/s, gpt_loss=0.0246, lr=1.89e-6] 
+
Train step of epoch 0:  64%|██████▎   | 3577/5616 [45:29<26:01,  1.31it/s, gpt_loss=0.0246, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3577/5616 [45:30<26:01,  1.31it/s, gpt_loss=0.0459, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3578/5616 [45:30<25:09,  1.35it/s, gpt_loss=0.0459, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3578/5616 [45:31<25:09,  1.35it/s, gpt_loss=0.0484, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3579/5616 [45:31<25:26,  1.33it/s, gpt_loss=0.0484, lr=1.89e-6]
+
Train step of epoch 0:  64%|██████▎   | 3579/5616 [45:31<25:26,  1.33it/s, gpt_loss=0.0266, lr=1.88e-6]
+
Train step of epoch 0:  64%|██████▎   | 3580/5616 [45:31<25:04,  1.35it/s, gpt_loss=0.0266, lr=1.88e-6]
+
Train step of epoch 0:  64%|██████▎   | 3580/5616 [45:32<25:04,  1.35it/s, gpt_loss=0.00572, lr=1.88e-6]
+
Train step of epoch 0:  64%|██████▍   | 3581/5616 [45:32<24:58,  1.36it/s, gpt_loss=0.00572, lr=1.88e-6]
+
Train step of epoch 0:  64%|██████▍   | 3581/5616 [45:33<24:58,  1.36it/s, gpt_loss=0.0366, lr=1.88e-6] 
+
Train step of epoch 0:  64%|██████▍   | 3582/5616 [45:33<26:02,  1.30it/s, gpt_loss=0.0366, lr=1.88e-6]
+
Train step of epoch 0:  64%|██████▍   | 3582/5616 [45:33<26:02,  1.30it/s, gpt_loss=0.014, lr=1.88e-6] 
+
Train step of epoch 0:  64%|██████▍   | 3583/5616 [45:33<25:15,  1.34it/s, gpt_loss=0.014, lr=1.88e-6]
+
Train step of epoch 0:  64%|��█████▍   | 3583/5616 [45:34<25:15,  1.34it/s, gpt_loss=0.045, lr=1.88e-6]
+
Train step of epoch 0:  64%|██████▍   | 3584/5616 [45:34<26:26,  1.28it/s, gpt_loss=0.045, lr=1.88e-6]
+
Train step of epoch 0:  64%|██████▍   | 3584/5616 [45:35<26:26,  1.28it/s, gpt_loss=0.0299, lr=1.88e-6]
+
Train step of epoch 0:  64%|██████▍   | 3585/5616 [45:35<26:37,  1.27it/s, gpt_loss=0.0299, lr=1.88e-6]
+
Train step of epoch 0:  64%|██████▍   | 3585/5616 [45:36<26:37,  1.27it/s, gpt_loss=0.00367, lr=1.88e-6]
+
Train step of epoch 0:  64%|██████▍   | 3586/5616 [45:36<25:59,  1.30it/s, gpt_loss=0.00367, lr=1.88e-6]
+
Train step of epoch 0:  64%|██████▍   | 3586/5616 [45:37<25:59,  1.30it/s, gpt_loss=0.0503, lr=1.88e-6] 
+
Train step of epoch 0:  64%|██████▍   | 3587/5616 [45:37<25:08,  1.35it/s, gpt_loss=0.0503, lr=1.88e-6]
+
Train step of epoch 0:  64%|██████▍   | 3587/5616 [45:37<25:08,  1.35it/s, gpt_loss=0.0239, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3588/5616 [45:37<25:14,  1.34it/s, gpt_loss=0.0239, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3588/5616 [45:38<25:14,  1.34it/s, gpt_loss=0.0248, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3589/5616 [45:38<24:48,  1.36it/s, gpt_loss=0.0248, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3589/5616 [45:39<24:48,  1.36it/s, gpt_loss=0.0471, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3590/5616 [45:39<24:52,  1.36it/s, gpt_loss=0.0471, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3590/5616 [45:39<24:52,  1.36it/s, gpt_loss=0.0407, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3591/5616 [45:39<24:31,  1.38it/s, gpt_loss=0.0407, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3591/5616 [45:41<24:31,  1.38it/s, gpt_loss=0.00747, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3592/5616 [45:41<27:44,  1.22it/s, gpt_loss=0.00747, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3592/5616 [45:41<27:44,  1.22it/s, gpt_loss=0.00739, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3593/5616 [45:41<26:11,  1.29it/s, gpt_loss=0.00739, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3593/5616 [45:42<26:11,  1.29it/s, gpt_loss=0.011, lr=1.87e-6]  
+
Train step of epoch 0:  64%|██████▍   | 3594/5616 [45:42<25:51,  1.30it/s, gpt_loss=0.011, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3594/5616 [45:43<25:51,  1.30it/s, gpt_loss=0.0564, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3595/5616 [45:43<25:36,  1.32it/s, gpt_loss=0.0564, lr=1.87e-6]
+
Train step of epoch 0:  64%|██████▍   | 3595/5616 [45:43<25:36,  1.32it/s, gpt_loss=0.0757, lr=1.86e-6]
+
Train step of epoch 0:  64%|██████▍   | 3596/5616 [45:43<25:46,  1.31it/s, gpt_loss=0.0757, lr=1.86e-6]
+
Train step of epoch 0:  64%|██████▍   | 3596/5616 [45:44<25:46,  1.31it/s, gpt_loss=0.197, lr=1.86e-6] 
+
Train step of epoch 0:  64%|██████▍   | 3597/5616 [45:44<28:23,  1.19it/s, gpt_loss=0.197, lr=1.86e-6]
+
Train step of epoch 0:  64%|██████▍   | 3597/5616 [45:45<28:23,  1.19it/s, gpt_loss=0.0432, lr=1.86e-6]
+
Train step of epoch 0:  64%|██████▍   | 3598/5616 [45:45<27:37,  1.22it/s, gpt_loss=0.0432, lr=1.86e-6]
+
Train step of epoch 0:  64%|██████▍   | 3598/5616 [45:46<27:37,  1.22it/s, gpt_loss=0.0357, lr=1.86e-6]
+
Train step of epoch 0:  64%|██████▍   | 3599/5616 [45:46<25:53,  1.30it/s, gpt_loss=0.0357, lr=1.86e-6][2025-06-30 07:23:55,654] [INFO] [logging.py:128:log_dist] [Rank 0] step=900, skipped=0, lr=[1.8584672868109182e-06, 1.8584672868109182e-06], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 07:23:55,655] [INFO] [timer.py:264:stop] epoch=0/micro_step=3600/global_step=900, RunningAvgSamplesPerSec=42.47955357973689, CurrSamplesPerSec=41.001528955967586, MemAllocated=16.0GB, MaxMemAllocated=54.86GB
+
+
Train step of epoch 0:  64%|██████▍   | 3599/5616 [45:47<25:53,  1.30it/s, gpt_loss=0.00124, lr=1.86e-6]
+
Train step of epoch 0:  64%|██████▍   | 3600/5616 [45:47<25:16,  1.33it/s, gpt_loss=0.00124, lr=1.86e-6]
+
Train step of epoch 0:  64%|██████▍   | 3600/5616 [45:47<25:16,  1.33it/s, gpt_loss=0.0329, lr=1.86e-6] 
+
Train step of epoch 0:  64%|██████▍   | 3601/5616 [45:47<24:46,  1.36it/s, gpt_loss=0.0329, lr=1.86e-6]
+
Train step of epoch 0:  64%|██████▍   | 3601/5616 [45:48<24:46,  1.36it/s, gpt_loss=0.00966, lr=1.86e-6]
+
Train step of epoch 0:  64%|██████▍   | 3602/5616 [45:48<27:40,  1.21it/s, gpt_loss=0.00966, lr=1.86e-6]
+
Train step of epoch 0:  64%|██████▍   | 3602/5616 [45:49<27:40,  1.21it/s, gpt_loss=0.04, lr=1.86e-6]   
+
Train step of epoch 0:  64%|██████▍   | 3603/5616 [45:49<26:13,  1.28it/s, gpt_loss=0.04, lr=1.86e-6]
+
Train step of epoch 0:  64%|██████▍   | 3603/5616 [45:50<26:13,  1.28it/s, gpt_loss=0.00857, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3604/5616 [45:50<26:07,  1.28it/s, gpt_loss=0.00857, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3604/5616 [45:51<26:07,  1.28it/s, gpt_loss=0.0259, lr=1.85e-6] 
+
Train step of epoch 0:  64%|██████▍   | 3605/5616 [45:51<27:32,  1.22it/s, gpt_loss=0.0259, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3605/5616 [45:51<27:32,  1.22it/s, gpt_loss=0.0118, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3606/5616 [45:51<26:16,  1.28it/s, gpt_loss=0.0118, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3606/5616 [45:52<26:16,  1.28it/s, gpt_loss=0.021, lr=1.85e-6] 
+
Train step of epoch 0:  64%|██████▍   | 3607/5616 [45:52<25:08,  1.33it/s, gpt_loss=0.021, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3607/5616 [45:53<25:08,  1.33it/s, gpt_loss=0.0335, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3608/5616 [45:53<25:18,  1.32it/s, gpt_loss=0.0335, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3608/5616 [45:54<25:18,  1.32it/s, gpt_loss=0.0287, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3609/5616 [45:54<25:13,  1.33it/s, gpt_loss=0.0287, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3609/5616 [45:54<25:13,  1.33it/s, gpt_loss=0.0853, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3610/5616 [45:54<25:03,  1.33it/s, gpt_loss=0.0853, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3610/5616 [45:55<25:03,  1.33it/s, gpt_loss=0.0701, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3611/5616 [45:55<24:15,  1.38it/s, gpt_loss=0.0701, lr=1.85e-6]
+
Train step of epoch 0:  64%|██████▍   | 3611/5616 [45:56<24:15,  1.38it/s, gpt_loss=0.0336, lr=1.84e-6]
+
Train step of epoch 0:  64%|██████▍   | 3612/5616 [45:56<26:10,  1.28it/s, gpt_loss=0.0336, lr=1.84e-6]
+
Train step of epoch 0:  64%|██████▍   | 3612/5616 [45:57<26:10,  1.28it/s, gpt_loss=0.0142, lr=1.84e-6]
+
Train step of epoch 0:  64%|██████▍   | 3613/5616 [45:57<25:19,  1.32it/s, gpt_loss=0.0142, lr=1.84e-6]
+
Train step of epoch 0:  64%|██████▍   | 3613/5616 [45:57<25:19,  1.32it/s, gpt_loss=0.147, lr=1.84e-6] 
+
Train step of epoch 0:  64%|██████▍   | 3614/5616 [45:57<24:59,  1.34it/s, gpt_loss=0.147, lr=1.84e-6]
+
Train step of epoch 0:  64%|██████▍   | 3614/5616 [45:58<24:59,  1.34it/s, gpt_loss=0.0204, lr=1.84e-6]
+
Train step of epoch 0:  64%|██████▍   | 3615/5616 [45:58<23:53,  1.40it/s, gpt_loss=0.0204, lr=1.84e-6]
+
Train step of epoch 0:  64%|██████▍   | 3615/5616 [45:59<23:53,  1.40it/s, gpt_loss=0.00997, lr=1.84e-6]
+
Train step of epoch 0:  64%|██████▍   | 3616/5616 [45:59<24:42,  1.35it/s, gpt_loss=0.00997, lr=1.84e-6]
+
Train step of epoch 0:  64%|██████▍   | 3616/5616 [45:59<24:42,  1.35it/s, gpt_loss=0.0272, lr=1.84e-6] 
+
Train step of epoch 0:  64%|██████▍   | 3617/5616 [45:59<24:09,  1.38it/s, gpt_loss=0.0272, lr=1.84e-6]
+
Train step of epoch 0:  64%|██████▍   | 3617/5616 [46:00<24:09,  1.38it/s, gpt_loss=0.00622, lr=1.84e-6]
+
Train step of epoch 0:  64%|██████▍   | 3618/5616 [46:00<23:42,  1.40it/s, gpt_loss=0.00622, lr=1.84e-6]
+
Train step of epoch 0:  64%|██████▍   | 3618/5616 [46:01<23:42,  1.40it/s, gpt_loss=0.0222, lr=1.84e-6] 
+
Train step of epoch 0:  64%|██████▍   | 3619/5616 [46:01<23:51,  1.40it/s, gpt_loss=0.0222, lr=1.84e-6]
+
Train step of epoch 0:  64%|██████▍   | 3619/5616 [46:02<23:51,  1.40it/s, gpt_loss=0.0968, lr=1.83e-6]
+
Train step of epoch 0:  64%|██████▍   | 3620/5616 [46:02<25:23,  1.31it/s, gpt_loss=0.0968, lr=1.83e-6]
+
Train step of epoch 0:  64%|██████▍   | 3620/5616 [46:02<25:23,  1.31it/s, gpt_loss=0.207, lr=1.83e-6] 
+
Train step of epoch 0:  64%|██████▍   | 3621/5616 [46:02<25:00,  1.33it/s, gpt_loss=0.207, lr=1.83e-6]
+
Train step of epoch 0:  64%|██████▍   | 3621/5616 [46:03<25:00,  1.33it/s, gpt_loss=0.0746, lr=1.83e-6]
+
Train step of epoch 0:  64%|██████▍   | 3622/5616 [46:03<24:09,  1.38it/s, gpt_loss=0.0746, lr=1.83e-6]
+
Train step of epoch 0:  64%|██████▍   | 3622/5616 [46:04<24:09,  1.38it/s, gpt_loss=0.0675, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3623/5616 [46:04<23:36,  1.41it/s, gpt_loss=0.0675, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3623/5616 [46:05<23:36,  1.41it/s, gpt_loss=0.0137, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3624/5616 [46:05<25:07,  1.32it/s, gpt_loss=0.0137, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3624/5616 [46:05<25:07,  1.32it/s, gpt_loss=0.0555, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3625/5616 [46:05<24:33,  1.35it/s, gpt_loss=0.0555, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3625/5616 [46:06<24:33,  1.35it/s, gpt_loss=0.00338, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3626/5616 [46:06<24:29,  1.35it/s, gpt_loss=0.00338, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3626/5616 [46:07<24:29,  1.35it/s, gpt_loss=0.0346, lr=1.83e-6] 
+
Train step of epoch 0:  65%|██████▍   | 3627/5616 [46:07<25:39,  1.29it/s, gpt_loss=0.0346, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3627/5616 [46:08<25:39,  1.29it/s, gpt_loss=0.0981, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3628/5616 [46:08<25:36,  1.29it/s, gpt_loss=0.0981, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3628/5616 [46:08<25:36,  1.29it/s, gpt_loss=0.00847, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3629/5616 [46:08<24:52,  1.33it/s, gpt_loss=0.00847, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3629/5616 [46:09<24:52,  1.33it/s, gpt_loss=0.158, lr=1.83e-6]  
+
Train step of epoch 0:  65%|██████▍   | 3630/5616 [46:09<24:09,  1.37it/s, gpt_loss=0.158, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3630/5616 [46:10<24:09,  1.37it/s, gpt_loss=0.0102, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3631/5616 [46:10<24:30,  1.35it/s, gpt_loss=0.0102, lr=1.83e-6]
+
Train step of epoch 0:  65%|██████▍   | 3631/5616 [46:11<24:30,  1.35it/s, gpt_loss=0.0638, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3632/5616 [46:11<24:40,  1.34it/s, gpt_loss=0.0638, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3632/5616 [46:11<24:40,  1.34it/s, gpt_loss=0.0354, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3633/5616 [46:11<25:18,  1.31it/s, gpt_loss=0.0354, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3633/5616 [46:12<25:18,  1.31it/s, gpt_loss=0.0364, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3634/5616 [46:12<24:28,  1.35it/s, gpt_loss=0.0364, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3634/5616 [46:13<24:28,  1.35it/s, gpt_loss=0.0609, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3635/5616 [46:13<25:10,  1.31it/s, gpt_loss=0.0609, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3635/5616 [46:14<25:10,  1.31it/s, gpt_loss=0.02, lr=1.82e-6]  
+
Train step of epoch 0:  65%|██████▍   | 3636/5616 [46:14<25:04,  1.32it/s, gpt_loss=0.02, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3636/5616 [46:14<25:04,  1.32it/s, gpt_loss=0.0163, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3637/5616 [46:14<24:35,  1.34it/s, gpt_loss=0.0163, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3637/5616 [46:15<24:35,  1.34it/s, gpt_loss=0.0615, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3638/5616 [46:15<24:17,  1.36it/s, gpt_loss=0.0615, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3638/5616 [46:16<24:17,  1.36it/s, gpt_loss=0.00559, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3639/5616 [46:16<23:34,  1.40it/s, gpt_loss=0.00559, lr=1.82e-6]
+
Train step of epoch 0:  65%|██████▍   | 3639/5616 [46:17<23:34,  1.40it/s, gpt_loss=0.0597, lr=1.81e-6] 
+
Train step of epoch 0:  65%|██████▍   | 3640/5616 [46:17<24:48,  1.33it/s, gpt_loss=0.0597, lr=1.81e-6]
+
Train step of epoch 0:  65%|██████▍   | 3640/5616 [46:17<24:48,  1.33it/s, gpt_loss=0.0341, lr=1.81e-6]
+
Train step of epoch 0:  65%|██████▍   | 3641/5616 [46:17<24:27,  1.35it/s, gpt_loss=0.0341, lr=1.81e-6]
+
Train step of epoch 0:  65%|██████▍   | 3641/5616 [46:18<24:27,  1.35it/s, gpt_loss=0.036, lr=1.81e-6] 
+
Train step of epoch 0:  65%|██████▍   | 3642/5616 [46:18<24:27,  1.34it/s, gpt_loss=0.036, lr=1.81e-6]
+
Train step of epoch 0:  65%|██████▍   | 3642/5616 [46:19<24:27,  1.34it/s, gpt_loss=0.0285, lr=1.81e-6]
+
Train step of epoch 0:  65%|██████▍   | 3643/5616 [46:19<23:58,  1.37it/s, gpt_loss=0.0285, lr=1.81e-6]
+
Train step of epoch 0:  65%|██████▍   | 3643/5616 [46:20<23:58,  1.37it/s, gpt_loss=0.0147, lr=1.81e-6]
+
Train step of epoch 0:  65%|██████▍   | 3644/5616 [46:20<24:03,  1.37it/s, gpt_loss=0.0147, lr=1.81e-6]
+
Train step of epoch 0:  65%|██████▍   | 3644/5616 [46:20<24:03,  1.37it/s, gpt_loss=0.0469, lr=1.81e-6]
+
Train step of epoch 0:  65%|██████▍   | 3645/5616 [46:20<24:11,  1.36it/s, gpt_loss=0.0469, lr=1.81e-6]
+
Train step of epoch 0:  65%|██████▍   | 3645/5616 [46:21<24:11,  1.36it/s, gpt_loss=0.0272, lr=1.81e-6]
+
Train step of epoch 0:  65%|██████▍   | 3646/5616 [46:21<23:35,  1.39it/s, gpt_loss=0.0272, lr=1.81e-6]
+
Train step of epoch 0:  65%|██████▍   | 3646/5616 [46:22<23:35,  1.39it/s, gpt_loss=0.018, lr=1.81e-6] 
+
Train step of epoch 0:  65%|██████▍   | 3647/5616 [46:22<23:06,  1.42it/s, gpt_loss=0.018, lr=1.81e-6]
+
Train step of epoch 0:  65%|██████▍   | 3647/5616 [46:23<23:06,  1.42it/s, gpt_loss=0.098, lr=1.8e-6] 
+
Train step of epoch 0:  65%|██████▍   | 3648/5616 [46:23<24:31,  1.34it/s, gpt_loss=0.098, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▍   | 3648/5616 [46:23<24:31,  1.34it/s, gpt_loss=0.0194, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▍   | 3649/5616 [46:23<24:18,  1.35it/s, gpt_loss=0.0194, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▍   | 3649/5616 [46:24<24:18,  1.35it/s, gpt_loss=0.011, lr=1.8e-6] 
+
Train step of epoch 0:  65%|██████▍   | 3650/5616 [46:24<26:10,  1.25it/s, gpt_loss=0.011, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▍   | 3650/5616 [46:25<26:10,  1.25it/s, gpt_loss=0.0114, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▌   | 3651/5616 [46:25<25:22,  1.29it/s, gpt_loss=0.0114, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▌   | 3651/5616 [46:26<25:22,  1.29it/s, gpt_loss=0.0369, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▌   | 3652/5616 [46:26<24:54,  1.31it/s, gpt_loss=0.0369, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▌   | 3652/5616 [46:27<24:54,  1.31it/s, gpt_loss=0.0854, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▌   | 3653/5616 [46:27<26:18,  1.24it/s, gpt_loss=0.0854, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▌   | 3653/5616 [46:27<26:18,  1.24it/s, gpt_loss=0.0175, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▌   | 3654/5616 [46:27<24:58,  1.31it/s, gpt_loss=0.0175, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▌   | 3654/5616 [46:28<24:58,  1.31it/s, gpt_loss=0.0487, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▌   | 3655/5616 [46:28<25:01,  1.31it/s, gpt_loss=0.0487, lr=1.8e-6]
+
Train step of epoch 0:  65%|██████▌   | 3655/5616 [46:29<25:01,  1.31it/s, gpt_loss=0.0389, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3656/5616 [46:29<25:37,  1.27it/s, gpt_loss=0.0389, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3656/5616 [46:29<25:37,  1.27it/s, gpt_loss=0.0199, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3657/5616 [46:29<24:47,  1.32it/s, gpt_loss=0.0199, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3657/5616 [46:30<24:47,  1.32it/s, gpt_loss=0.0601, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3658/5616 [46:30<24:19,  1.34it/s, gpt_loss=0.0601, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3658/5616 [46:31<24:19,  1.34it/s, gpt_loss=0.0195, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3659/5616 [46:31<25:05,  1.30it/s, gpt_loss=0.0195, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3659/5616 [46:32<25:05,  1.30it/s, gpt_loss=0.0324, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3660/5616 [46:32<26:18,  1.24it/s, gpt_loss=0.0324, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3660/5616 [46:33<26:18,  1.24it/s, gpt_loss=0.0326, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3661/5616 [46:33<25:55,  1.26it/s, gpt_loss=0.0326, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3661/5616 [46:33<25:55,  1.26it/s, gpt_loss=0.0474, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3662/5616 [46:33<25:21,  1.28it/s, gpt_loss=0.0474, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3662/5616 [46:34<25:21,  1.28it/s, gpt_loss=0.01, lr=1.79e-6]  
+
Train step of epoch 0:  65%|██████▌   | 3663/5616 [46:34<24:39,  1.32it/s, gpt_loss=0.01, lr=1.79e-6]
+
Train step of epoch 0:  65%|██████▌   | 3663/5616 [46:35<24:39,  1.32it/s, gpt_loss=0.00667, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3664/5616 [46:35<24:34,  1.32it/s, gpt_loss=0.00667, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3664/5616 [46:36<24:34,  1.32it/s, gpt_loss=0.0959, lr=1.78e-6] 
+
Train step of epoch 0:  65%|██████▌   | 3665/5616 [46:36<23:57,  1.36it/s, gpt_loss=0.0959, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3665/5616 [46:37<23:57,  1.36it/s, gpt_loss=0.044, lr=1.78e-6] 
+
Train step of epoch 0:  65%|██████▌   | 3666/5616 [46:37<25:51,  1.26it/s, gpt_loss=0.044, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3666/5616 [46:37<25:51,  1.26it/s, gpt_loss=0.0384, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3667/5616 [46:37<26:04,  1.25it/s, gpt_loss=0.0384, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3667/5616 [46:38<26:04,  1.25it/s, gpt_loss=0.0812, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3668/5616 [46:38<25:42,  1.26it/s, gpt_loss=0.0812, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3668/5616 [46:39<25:42,  1.26it/s, gpt_loss=0.0155, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3669/5616 [46:39<24:13,  1.34it/s, gpt_loss=0.0155, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3669/5616 [46:40<24:13,  1.34it/s, gpt_loss=0.0443, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3670/5616 [46:40<24:37,  1.32it/s, gpt_loss=0.0443, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3670/5616 [46:40<24:37,  1.32it/s, gpt_loss=0.0249, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3671/5616 [46:40<23:55,  1.35it/s, gpt_loss=0.0249, lr=1.78e-6]
+
Train step of epoch 0:  65%|██████▌   | 3671/5616 [46:41<23:55,  1.35it/s, gpt_loss=0.00567, lr=1.77e-6]
+
Train step of epoch 0:  65%|██████▌   | 3672/5616 [46:41<24:49,  1.31it/s, gpt_loss=0.00567, lr=1.77e-6]
+
Train step of epoch 0:  65%|██████▌   | 3672/5616 [46:42<24:49,  1.31it/s, gpt_loss=0.043, lr=1.77e-6]  
+
Train step of epoch 0:  65%|██████▌   | 3673/5616 [46:42<25:06,  1.29it/s, gpt_loss=0.043, lr=1.77e-6]
+
Train step of epoch 0:  65%|██████▌   | 3673/5616 [46:43<25:06,  1.29it/s, gpt_loss=0.0323, lr=1.77e-6]
+
Train step of epoch 0:  65%|██████▌   | 3674/5616 [46:43<24:14,  1.33it/s, gpt_loss=0.0323, lr=1.77e-6]
+
Train step of epoch 0:  65%|██████▌   | 3674/5616 [46:43<24:14,  1.33it/s, gpt_loss=0.0223, lr=1.77e-6]
+
Train step of epoch 0:  65%|██████▌   | 3675/5616 [46:43<22:58,  1.41it/s, gpt_loss=0.0223, lr=1.77e-6]
+
Train step of epoch 0:  65%|██████▌   | 3675/5616 [46:44<22:58,  1.41it/s, gpt_loss=0.0492, lr=1.77e-6]
+
Train step of epoch 0:  65%|██████▌   | 3676/5616 [46:44<23:38,  1.37it/s, gpt_loss=0.0492, lr=1.77e-6]
+
Train step of epoch 0:  65%|██████▌   | 3676/5616 [46:45<23:38,  1.37it/s, gpt_loss=0.0999, lr=1.77e-6]
+
Train step of epoch 0:  65%|██████▌   | 3677/5616 [46:45<23:09,  1.40it/s, gpt_loss=0.0999, lr=1.77e-6]
+
Train step of epoch 0:  65%|██████▌   | 3677/5616 [46:45<23:09,  1.40it/s, gpt_loss=0.00508, lr=1.77e-6]
+
Train step of epoch 0:  65%|██████▌   | 3678/5616 [46:45<23:57,  1.35it/s, gpt_loss=0.00508, lr=1.77e-6]
+
Train step of epoch 0:  65%|██████▌   | 3678/5616 [46:46<23:57,  1.35it/s, gpt_loss=0.0672, lr=1.77e-6] 
+
Train step of epoch 0:  66%|██████▌   | 3679/5616 [46:46<23:16,  1.39it/s, gpt_loss=0.0672, lr=1.77e-6]
+
Train step of epoch 0:  66%|██████▌   | 3679/5616 [46:47<23:16,  1.39it/s, gpt_loss=0.00795, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3680/5616 [46:47<24:13,  1.33it/s, gpt_loss=0.00795, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3680/5616 [46:48<24:13,  1.33it/s, gpt_loss=0.0671, lr=1.76e-6] 
+
Train step of epoch 0:  66%|██████▌   | 3681/5616 [46:48<25:29,  1.27it/s, gpt_loss=0.0671, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3681/5616 [46:48<25:29,  1.27it/s, gpt_loss=0.0156, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3682/5616 [46:48<24:22,  1.32it/s, gpt_loss=0.0156, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3682/5616 [46:49<24:22,  1.32it/s, gpt_loss=0.0117, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3683/5616 [46:49<23:35,  1.37it/s, gpt_loss=0.0117, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3683/5616 [46:50<23:35,  1.37it/s, gpt_loss=0.00308, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3684/5616 [46:50<23:37,  1.36it/s, gpt_loss=0.00308, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3684/5616 [46:51<23:37,  1.36it/s, gpt_loss=0.0215, lr=1.76e-6] 
+
Train step of epoch 0:  66%|██████▌   | 3685/5616 [46:51<22:58,  1.40it/s, gpt_loss=0.0215, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3685/5616 [46:51<22:58,  1.40it/s, gpt_loss=0.0237, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3686/5616 [46:51<23:54,  1.35it/s, gpt_loss=0.0237, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3686/5616 [46:52<23:54,  1.35it/s, gpt_loss=0.0207, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3687/5616 [46:52<23:22,  1.38it/s, gpt_loss=0.0207, lr=1.76e-6]
+
Train step of epoch 0:  66%|██████▌   | 3687/5616 [46:53<23:22,  1.38it/s, gpt_loss=0.0962, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3688/5616 [46:53<24:12,  1.33it/s, gpt_loss=0.0962, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3688/5616 [46:54<24:12,  1.33it/s, gpt_loss=0.0577, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3689/5616 [46:54<25:25,  1.26it/s, gpt_loss=0.0577, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3689/5616 [46:54<25:25,  1.26it/s, gpt_loss=0.0501, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3690/5616 [46:54<23:48,  1.35it/s, gpt_loss=0.0501, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3690/5616 [46:55<23:48,  1.35it/s, gpt_loss=0.104, lr=1.75e-6] 
+
Train step of epoch 0:  66%|██████▌   | 3691/5616 [46:55<24:08,  1.33it/s, gpt_loss=0.104, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3691/5616 [46:56<24:08,  1.33it/s, gpt_loss=0.0435, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3692/5616 [46:56<24:37,  1.30it/s, gpt_loss=0.0435, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3692/5616 [46:57<24:37,  1.30it/s, gpt_loss=0.0411, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3693/5616 [46:57<24:03,  1.33it/s, gpt_loss=0.0411, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3693/5616 [46:57<24:03,  1.33it/s, gpt_loss=0.102, lr=1.75e-6] 
+
Train step of epoch 0:  66%|██████▌   | 3694/5616 [46:57<24:44,  1.29it/s, gpt_loss=0.102, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3694/5616 [46:58<24:44,  1.29it/s, gpt_loss=0.0374, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3695/5616 [46:58<23:51,  1.34it/s, gpt_loss=0.0374, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3695/5616 [46:59<23:51,  1.34it/s, gpt_loss=0.0159, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3696/5616 [46:59<25:21,  1.26it/s, gpt_loss=0.0159, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3696/5616 [47:00<25:21,  1.26it/s, gpt_loss=0.0303, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3697/5616 [47:00<25:14,  1.27it/s, gpt_loss=0.0303, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3697/5616 [47:01<25:14,  1.27it/s, gpt_loss=0.0171, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3698/5616 [47:01<24:03,  1.33it/s, gpt_loss=0.0171, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3698/5616 [47:01<24:03,  1.33it/s, gpt_loss=0.0446, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3699/5616 [47:01<23:56,  1.33it/s, gpt_loss=0.0446, lr=1.75e-6]
+
Train step of epoch 0:  66%|██████▌   | 3699/5616 [47:02<23:56,  1.33it/s, gpt_loss=0.00741, lr=1.74e-6]
+
Train step of epoch 0:  66%|██████▌   | 3700/5616 [47:02<24:07,  1.32it/s, gpt_loss=0.00741, lr=1.74e-6]
+
Train step of epoch 0:  66%|██████▌   | 3700/5616 [47:03<24:07,  1.32it/s, gpt_loss=0.2, lr=1.74e-6]    
+
Train step of epoch 0:  66%|██████▌   | 3701/5616 [47:03<25:13,  1.27it/s, gpt_loss=0.2, lr=1.74e-6]
+
Train step of epoch 0:  66%|██████▌   | 3701/5616 [47:04<25:13,  1.27it/s, gpt_loss=0.0136, lr=1.74e-6]
+
Train step of epoch 0:  66%|██████▌   | 3702/5616 [47:04<23:52,  1.34it/s, gpt_loss=0.0136, lr=1.74e-6]
+
Train step of epoch 0:  66%|██████▌   | 3702/5616 [47:04<23:52,  1.34it/s, gpt_loss=0.099, lr=1.74e-6] 
+
Train step of epoch 0:  66%|██████▌   | 3703/5616 [47:04<23:43,  1.34it/s, gpt_loss=0.099, lr=1.74e-6]
+
Train step of epoch 0:  66%|██████▌   | 3703/5616 [47:05<23:43,  1.34it/s, gpt_loss=0.148, lr=1.74e-6]
+
Train step of epoch 0:  66%|██████▌   | 3704/5616 [47:05<25:06,  1.27it/s, gpt_loss=0.148, lr=1.74e-6]
+
Train step of epoch 0:  66%|██████▌   | 3704/5616 [47:06<25:06,  1.27it/s, gpt_loss=0.00886, lr=1.74e-6]
+
Train step of epoch 0:  66%|██████▌   | 3705/5616 [47:06<28:50,  1.10it/s, gpt_loss=0.00886, lr=1.74e-6]
+
Train step of epoch 0:  66%|██████▌   | 3705/5616 [47:07<28:50,  1.10it/s, gpt_loss=0.0213, lr=1.74e-6] 
+
Train step of epoch 0:  66%|██████▌   | 3706/5616 [47:07<28:19,  1.12it/s, gpt_loss=0.0213, lr=1.74e-6]
+
Train step of epoch 0:  66%|██████▌   | 3706/5616 [47:08<28:19,  1.12it/s, gpt_loss=0.00379, lr=1.74e-6]
+
Train step of epoch 0:  66%|██████▌   | 3707/5616 [47:08<26:33,  1.20it/s, gpt_loss=0.00379, lr=1.74e-6]
+
Train step of epoch 0:  66%|██████▌   | 3707/5616 [47:09<26:33,  1.20it/s, gpt_loss=0.0943, lr=1.73e-6] 
+
Train step of epoch 0:  66%|██████▌   | 3708/5616 [47:09<26:14,  1.21it/s, gpt_loss=0.0943, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3708/5616 [47:09<26:14,  1.21it/s, gpt_loss=0.0221, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3709/5616 [47:09<25:02,  1.27it/s, gpt_loss=0.0221, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3709/5616 [47:10<25:02,  1.27it/s, gpt_loss=0.0162, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3710/5616 [47:10<27:04,  1.17it/s, gpt_loss=0.0162, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3710/5616 [47:11<27:04,  1.17it/s, gpt_loss=0.0369, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3711/5616 [47:11<25:23,  1.25it/s, gpt_loss=0.0369, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3711/5616 [47:12<25:23,  1.25it/s, gpt_loss=0.00421, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3712/5616 [47:12<25:20,  1.25it/s, gpt_loss=0.00421, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3712/5616 [47:13<25:20,  1.25it/s, gpt_loss=0.0613, lr=1.73e-6] 
+
Train step of epoch 0:  66%|██████▌   | 3713/5616 [47:13<24:36,  1.29it/s, gpt_loss=0.0613, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3713/5616 [47:13<24:36,  1.29it/s, gpt_loss=0.0879, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3714/5616 [47:13<23:40,  1.34it/s, gpt_loss=0.0879, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3714/5616 [47:14<23:40,  1.34it/s, gpt_loss=0.0721, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3715/5616 [47:14<23:14,  1.36it/s, gpt_loss=0.0721, lr=1.73e-6]
+
Train step of epoch 0:  66%|██████▌   | 3715/5616 [47:15<23:14,  1.36it/s, gpt_loss=0.037, lr=1.72e-6] 
+
Train step of epoch 0:  66%|██████▌   | 3716/5616 [47:15<23:21,  1.36it/s, gpt_loss=0.037, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▌   | 3716/5616 [47:15<23:21,  1.36it/s, gpt_loss=0.0381, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▌   | 3717/5616 [47:15<22:58,  1.38it/s, gpt_loss=0.0381, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▌   | 3717/5616 [47:16<22:58,  1.38it/s, gpt_loss=0.00711, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▌   | 3718/5616 [47:16<22:33,  1.40it/s, gpt_loss=0.00711, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▌   | 3718/5616 [47:17<22:33,  1.40it/s, gpt_loss=0.0434, lr=1.72e-6] 
+
Train step of epoch 0:  66%|██████▌   | 3719/5616 [47:17<22:12,  1.42it/s, gpt_loss=0.0434, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▌   | 3719/5616 [47:18<22:12,  1.42it/s, gpt_loss=0.0311, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▌   | 3720/5616 [47:18<22:20,  1.41it/s, gpt_loss=0.0311, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▌   | 3720/5616 [47:18<22:20,  1.41it/s, gpt_loss=0.0247, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▋   | 3721/5616 [47:18<24:14,  1.30it/s, gpt_loss=0.0247, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▋   | 3721/5616 [47:19<24:14,  1.30it/s, gpt_loss=0.0131, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▋   | 3722/5616 [47:19<23:23,  1.35it/s, gpt_loss=0.0131, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▋   | 3722/5616 [47:20<23:23,  1.35it/s, gpt_loss=0.00608, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▋   | 3723/5616 [47:20<22:49,  1.38it/s, gpt_loss=0.00608, lr=1.72e-6]
+
Train step of epoch 0:  66%|██████▋   | 3723/5616 [47:21<22:49,  1.38it/s, gpt_loss=0.0694, lr=1.71e-6] 
+
Train step of epoch 0:  66%|██████▋   | 3724/5616 [47:21<25:50,  1.22it/s, gpt_loss=0.0694, lr=1.71e-6]
+
Train step of epoch 0:  66%|██████▋   | 3724/5616 [47:22<25:50,  1.22it/s, gpt_loss=0.011, lr=1.71e-6] 
+
Train step of epoch 0:  66%|██████▋   | 3725/5616 [47:22<29:04,  1.08it/s, gpt_loss=0.011, lr=1.71e-6]
+
Train step of epoch 0:  66%|██████▋   | 3725/5616 [47:23<29:04,  1.08it/s, gpt_loss=0.0738, lr=1.71e-6]
+
Train step of epoch 0:  66%|██████▋   | 3726/5616 [47:23<27:14,  1.16it/s, gpt_loss=0.0738, lr=1.71e-6]
+
Train step of epoch 0:  66%|██████▋   | 3726/5616 [47:23<27:14,  1.16it/s, gpt_loss=0.0254, lr=1.71e-6]
+
Train step of epoch 0:  66%|██████▋   | 3727/5616 [47:23<26:08,  1.20it/s, gpt_loss=0.0254, lr=1.71e-6]
+
Train step of epoch 0:  66%|█���████▋   | 3727/5616 [47:24<26:08,  1.20it/s, gpt_loss=0.0368, lr=1.71e-6]
+
Train step of epoch 0:  66%|██████▋   | 3728/5616 [47:24<25:21,  1.24it/s, gpt_loss=0.0368, lr=1.71e-6]
+
Train step of epoch 0:  66%|██████▋   | 3728/5616 [47:25<25:21,  1.24it/s, gpt_loss=0.0255, lr=1.71e-6]
+
Train step of epoch 0:  66%|██████▋   | 3729/5616 [47:25<23:51,  1.32it/s, gpt_loss=0.0255, lr=1.71e-6]
+
Train step of epoch 0:  66%|██████▋   | 3729/5616 [47:26<23:51,  1.32it/s, gpt_loss=0.0187, lr=1.71e-6]
+
Train step of epoch 0:  66%|██████▋   | 3730/5616 [47:26<23:42,  1.33it/s, gpt_loss=0.0187, lr=1.71e-6]
+
Train step of epoch 0:  66%|██████▋   | 3730/5616 [47:26<23:42,  1.33it/s, gpt_loss=0.0463, lr=1.71e-6]
+
Train step of epoch 0:  66%|██████▋   | 3731/5616 [47:26<23:07,  1.36it/s, gpt_loss=0.0463, lr=1.71e-6]
+
Train step of epoch 0:  66%|██████▋   | 3731/5616 [47:27<23:07,  1.36it/s, gpt_loss=0.0653, lr=1.7e-6] 
+
Train step of epoch 0:  66%|██████▋   | 3732/5616 [47:27<23:19,  1.35it/s, gpt_loss=0.0653, lr=1.7e-6]
+
Train step of epoch 0:  66%|██████▋   | 3732/5616 [47:28<23:19,  1.35it/s, gpt_loss=0.0497, lr=1.7e-6]
+
Train step of epoch 0:  66%|██████▋   | 3733/5616 [47:28<23:18,  1.35it/s, gpt_loss=0.0497, lr=1.7e-6]
+
Train step of epoch 0:  66%|██████▋   | 3733/5616 [47:28<23:18,  1.35it/s, gpt_loss=0.0247, lr=1.7e-6]
+
Train step of epoch 0:  66%|██████▋   | 3734/5616 [47:28<22:40,  1.38it/s, gpt_loss=0.0247, lr=1.7e-6]
+
Train step of epoch 0:  66%|██████▋   | 3734/5616 [47:29<22:40,  1.38it/s, gpt_loss=0.028, lr=1.7e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3735/5616 [47:29<22:40,  1.38it/s, gpt_loss=0.028, lr=1.7e-6]
+
Train step of epoch 0:  67%|██████▋   | 3735/5616 [47:30<22:40,  1.38it/s, gpt_loss=0.00423, lr=1.7e-6]
+
Train step of epoch 0:  67%|██████▋   | 3736/5616 [47:30<22:54,  1.37it/s, gpt_loss=0.00423, lr=1.7e-6]
+
Train step of epoch 0:  67%|██████▋   | 3736/5616 [47:31<22:54,  1.37it/s, gpt_loss=0.00475, lr=1.7e-6]
+
Train step of epoch 0:  67%|██████▋   | 3737/5616 [47:31<23:16,  1.35it/s, gpt_loss=0.00475, lr=1.7e-6]
+
Train step of epoch 0:  67%|██████▋   | 3737/5616 [47:31<23:16,  1.35it/s, gpt_loss=0.0264, lr=1.7e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3738/5616 [47:31<22:34,  1.39it/s, gpt_loss=0.0264, lr=1.7e-6]
+
Train step of epoch 0:  67%|██████▋   | 3738/5616 [47:32<22:34,  1.39it/s, gpt_loss=0.00793, lr=1.7e-6]
+
Train step of epoch 0:  67%|██████▋   | 3739/5616 [47:32<22:55,  1.36it/s, gpt_loss=0.00793, lr=1.7e-6]
+
Train step of epoch 0:  67%|██████▋   | 3739/5616 [47:33<22:55,  1.36it/s, gpt_loss=0.0201, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3740/5616 [47:33<23:34,  1.33it/s, gpt_loss=0.0201, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3740/5616 [47:34<23:34,  1.33it/s, gpt_loss=0.0238, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3741/5616 [47:34<23:30,  1.33it/s, gpt_loss=0.0238, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3741/5616 [47:35<23:30,  1.33it/s, gpt_loss=0.0259, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3742/5616 [47:35<23:55,  1.31it/s, gpt_loss=0.0259, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3742/5616 [47:35<23:55,  1.31it/s, gpt_loss=0.00452, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3743/5616 [47:35<22:59,  1.36it/s, gpt_loss=0.00452, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3743/5616 [47:36<22:59,  1.36it/s, gpt_loss=0.0503, lr=1.69e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3744/5616 [47:36<24:04,  1.30it/s, gpt_loss=0.0503, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3744/5616 [47:37<24:04,  1.30it/s, gpt_loss=0.107, lr=1.69e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3745/5616 [47:37<25:03,  1.24it/s, gpt_loss=0.107, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3745/5616 [47:38<25:03,  1.24it/s, gpt_loss=0.0147, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3746/5616 [47:38<23:40,  1.32it/s, gpt_loss=0.0147, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3746/5616 [47:39<23:40,  1.32it/s, gpt_loss=0.021, lr=1.69e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3747/5616 [47:39<26:05,  1.19it/s, gpt_loss=0.021, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3747/5616 [47:39<26:05,  1.19it/s, gpt_loss=0.232, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3748/5616 [47:39<25:24,  1.22it/s, gpt_loss=0.232, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3748/5616 [47:40<25:24,  1.22it/s, gpt_loss=0.0767, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3749/5616 [47:40<25:33,  1.22it/s, gpt_loss=0.0767, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3749/5616 [47:41<25:33,  1.22it/s, gpt_loss=0.092, lr=1.69e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3750/5616 [47:41<24:21,  1.28it/s, gpt_loss=0.092, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3750/5616 [47:42<24:21,  1.28it/s, gpt_loss=0.0947, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3751/5616 [47:42<23:51,  1.30it/s, gpt_loss=0.0947, lr=1.69e-6]
+
Train step of epoch 0:  67%|██████▋   | 3751/5616 [47:43<23:51,  1.30it/s, gpt_loss=0.00345, lr=1.68e-6]
+
Train step of epoch 0:  67%|██████▋   | 3752/5616 [47:43<26:12,  1.19it/s, gpt_loss=0.00345, lr=1.68e-6]
+
Train step of epoch 0:  67%|██████▋   | 3752/5616 [47:43<26:12,  1.19it/s, gpt_loss=0.15, lr=1.68e-6]   
+
Train step of epoch 0:  67%|██████▋   | 3753/5616 [47:43<24:45,  1.25it/s, gpt_loss=0.15, lr=1.68e-6]
+
Train step of epoch 0:  67%|██████▋   | 3753/5616 [47:44<24:45,  1.25it/s, gpt_loss=0.0491, lr=1.68e-6]
+
Train step of epoch 0:  67%|██████▋   | 3754/5616 [47:44<23:51,  1.30it/s, gpt_loss=0.0491, lr=1.68e-6]
+
Train step of epoch 0:  67%|██████▋   | 3754/5616 [47:45<23:51,  1.30it/s, gpt_loss=0.029, lr=1.68e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3755/5616 [47:45<23:07,  1.34it/s, gpt_loss=0.029, lr=1.68e-6]
+
Train step of epoch 0:  67%|██████▋   | 3755/5616 [47:45<23:07,  1.34it/s, gpt_loss=0.0102, lr=1.68e-6]
+
Train step of epoch 0:  67%|██████▋   | 3756/5616 [47:45<23:05,  1.34it/s, gpt_loss=0.0102, lr=1.68e-6]
+
Train step of epoch 0:  67%|██████▋   | 3756/5616 [47:46<23:05,  1.34it/s, gpt_loss=0.103, lr=1.68e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3757/5616 [47:46<22:10,  1.40it/s, gpt_loss=0.103, lr=1.68e-6]
+
Train step of epoch 0:  67%|██████▋   | 3757/5616 [47:47<22:10,  1.40it/s, gpt_loss=0.0718, lr=1.68e-6]
+
Train step of epoch 0:  67%|██████▋   | 3758/5616 [47:47<22:52,  1.35it/s, gpt_loss=0.0718, lr=1.68e-6]
+
Train step of epoch 0:  67%|██████▋   | 3758/5616 [47:48<22:52,  1.35it/s, gpt_loss=0.0162, lr=1.68e-6]
+
Train step of epoch 0:  67%|██████▋   | 3759/5616 [47:48<26:04,  1.19it/s, gpt_loss=0.0162, lr=1.68e-6]
+
Train step of epoch 0:  67%|██████▋   | 3759/5616 [47:49<26:04,  1.19it/s, gpt_loss=0.033, lr=1.67e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3760/5616 [47:49<25:49,  1.20it/s, gpt_loss=0.033, lr=1.67e-6]
+
Train step of epoch 0:  67%|██████▋   | 3760/5616 [47:50<25:49,  1.20it/s, gpt_loss=0.044, lr=1.67e-6]
+
Train step of epoch 0:  67%|██████▋   | 3761/5616 [47:50<25:27,  1.21it/s, gpt_loss=0.044, lr=1.67e-6]
+
Train step of epoch 0:  67%|██████▋   | 3761/5616 [47:50<25:27,  1.21it/s, gpt_loss=0.0146, lr=1.67e-6]
+
Train step of epoch 0:  67%|██████▋   | 3762/5616 [47:50<24:06,  1.28it/s, gpt_loss=0.0146, lr=1.67e-6]
+
Train step of epoch 0:  67%|██████▋   | 3762/5616 [47:51<24:06,  1.28it/s, gpt_loss=0.0289, lr=1.67e-6]
+
Train step of epoch 0:  67%|██████▋   | 3763/5616 [47:51<25:26,  1.21it/s, gpt_loss=0.0289, lr=1.67e-6]
+
Train step of epoch 0:  67%|██████▋   | 3763/5616 [47:52<25:26,  1.21it/s, gpt_loss=0.00765, lr=1.67e-6]
+
Train step of epoch 0:  67%|██████▋   | 3764/5616 [47:52<25:36,  1.21it/s, gpt_loss=0.00765, lr=1.67e-6]
+
Train step of epoch 0:  67%|██████▋   | 3764/5616 [47:53<25:36,  1.21it/s, gpt_loss=0.0187, lr=1.67e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3765/5616 [47:53<24:32,  1.26it/s, gpt_loss=0.0187, lr=1.67e-6]
+
Train step of epoch 0:  67%|██████▋   | 3765/5616 [47:54<24:32,  1.26it/s, gpt_loss=0.00685, lr=1.67e-6]
+
Train step of epoch 0:  67%|██████▋   | 3766/5616 [47:54<24:16,  1.27it/s, gpt_loss=0.00685, lr=1.67e-6]
+
Train step of epoch 0:  67%|██████▋   | 3766/5616 [47:54<24:16,  1.27it/s, gpt_loss=0.0157, lr=1.67e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3767/5616 [47:54<23:47,  1.30it/s, gpt_loss=0.0157, lr=1.67e-6]
+
Train step of epoch 0:  67%|██████▋   | 3767/5616 [47:55<23:47,  1.30it/s, gpt_loss=0.00533, lr=1.66e-6]
+
Train step of epoch 0:  67%|██████▋   | 3768/5616 [47:55<24:17,  1.27it/s, gpt_loss=0.00533, lr=1.66e-6]
+
Train step of epoch 0:  67%|██████▋   | 3768/5616 [47:56<24:17,  1.27it/s, gpt_loss=0.0989, lr=1.66e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3769/5616 [47:56<24:10,  1.27it/s, gpt_loss=0.0989, lr=1.66e-6]
+
Train step of epoch 0:  67%|██████▋   | 3769/5616 [47:57<24:10,  1.27it/s, gpt_loss=0.00655, lr=1.66e-6]
+
Train step of epoch 0:  67%|██████▋   | 3770/5616 [47:57<22:54,  1.34it/s, gpt_loss=0.00655, lr=1.66e-6]
+
Train step of epoch 0:  67%|██████▋   | 3770/5616 [47:57<22:54,  1.34it/s, gpt_loss=0.0143, lr=1.66e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3771/5616 [47:57<21:54,  1.40it/s, gpt_loss=0.0143, lr=1.66e-6]
+
Train step of epoch 0:  67%|██████▋   | 3771/5616 [47:58<21:54,  1.40it/s, gpt_loss=0.0156, lr=1.66e-6]
+
Train step of epoch 0:  67%|██████▋   | 3772/5616 [47:58<21:55,  1.40it/s, gpt_loss=0.0156, lr=1.66e-6]
+
Train step of epoch 0:  67%|██████▋   | 3772/5616 [47:59<21:55,  1.40it/s, gpt_loss=0.0227, lr=1.66e-6]
+
Train step of epoch 0:  67%|██████▋   | 3773/5616 [47:59<21:52,  1.40it/s, gpt_loss=0.0227, lr=1.66e-6]
+
Train step of epoch 0:  67%|██████▋   | 3773/5616 [47:59<21:52,  1.40it/s, gpt_loss=0.032, lr=1.66e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3774/5616 [47:59<22:14,  1.38it/s, gpt_loss=0.032, lr=1.66e-6]
+
Train step of epoch 0:  67%|██████▋   | 3774/5616 [48:00<22:14,  1.38it/s, gpt_loss=0.136, lr=1.66e-6]
+
Train step of epoch 0:  67%|██████▋   | 3775/5616 [48:00<22:00,  1.39it/s, gpt_loss=0.136, lr=1.66e-6]
+
Train step of epoch 0:  67%|██████▋   | 3775/5616 [48:01<22:00,  1.39it/s, gpt_loss=0.0474, lr=1.65e-6]
+
Train step of epoch 0:  67%|██████▋   | 3776/5616 [48:01<22:22,  1.37it/s, gpt_loss=0.0474, lr=1.65e-6]
+
Train step of epoch 0:  67%|██████▋   | 3776/5616 [48:02<22:22,  1.37it/s, gpt_loss=0.0802, lr=1.65e-6]
+
Train step of epoch 0:  67%|██████▋   | 3777/5616 [48:02<22:01,  1.39it/s, gpt_loss=0.0802, lr=1.65e-6]
+
Train step of epoch 0:  67%|██████▋   | 3777/5616 [48:03<22:01,  1.39it/s, gpt_loss=0.125, lr=1.65e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3778/5616 [48:03<24:57,  1.23it/s, gpt_loss=0.125, lr=1.65e-6]
+
Train step of epoch 0:  67%|██████▋   | 3778/5616 [48:03<24:57,  1.23it/s, gpt_loss=0.014, lr=1.65e-6]
+
Train step of epoch 0:  67%|██████▋   | 3779/5616 [48:03<24:12,  1.26it/s, gpt_loss=0.014, lr=1.65e-6]
+
Train step of epoch 0:  67%|██████▋   | 3779/5616 [48:04<24:12,  1.26it/s, gpt_loss=0.00686, lr=1.65e-6]
+
Train step of epoch 0:  67%|██████▋   | 3780/5616 [48:04<25:23,  1.20it/s, gpt_loss=0.00686, lr=1.65e-6]
+
Train step of epoch 0:  67%|██████▋   | 3780/5616 [48:05<25:23,  1.20it/s, gpt_loss=0.207, lr=1.65e-6]  
+
Train step of epoch 0:  67%|██████▋   | 3781/5616 [48:05<23:45,  1.29it/s, gpt_loss=0.207, lr=1.65e-6]
+
Train step of epoch 0:  67%|██████▋   | 3781/5616 [48:06<23:45,  1.29it/s, gpt_loss=0.0578, lr=1.65e-6]
+
Train step of epoch 0:  67%|██████▋   | 3782/5616 [48:06<22:59,  1.33it/s, gpt_loss=0.0578, lr=1.65e-6]
+
Train step of epoch 0:  67%|██████▋   | 3782/5616 [48:06<22:59,  1.33it/s, gpt_loss=0.012, lr=1.65e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3783/5616 [48:06<23:14,  1.31it/s, gpt_loss=0.012, lr=1.65e-6]
+
Train step of epoch 0:  67%|██████▋   | 3783/5616 [48:07<23:14,  1.31it/s, gpt_loss=0.0375, lr=1.64e-6]
+
Train step of epoch 0:  67%|██████▋   | 3784/5616 [48:07<23:21,  1.31it/s, gpt_loss=0.0375, lr=1.64e-6]
+
Train step of epoch 0:  67%|██████▋   | 3784/5616 [48:08<23:21,  1.31it/s, gpt_loss=0.0925, lr=1.64e-6]
+
Train step of epoch 0:  67%|██████▋   | 3785/5616 [48:08<23:54,  1.28it/s, gpt_loss=0.0925, lr=1.64e-6]
+
Train step of epoch 0:  67%|██████▋   | 3785/5616 [48:09<23:54,  1.28it/s, gpt_loss=0.00502, lr=1.64e-6]
+
Train step of epoch 0:  67%|██████▋   | 3786/5616 [48:09<23:34,  1.29it/s, gpt_loss=0.00502, lr=1.64e-6]
+
Train step of epoch 0:  67%|██████▋   | 3786/5616 [48:09<23:34,  1.29it/s, gpt_loss=0.0186, lr=1.64e-6] 
+
Train step of epoch 0:  67%|██████▋   | 3787/5616 [48:09<23:46,  1.28it/s, gpt_loss=0.0186, lr=1.64e-6]
+
Train step of epoch 0:  67%|██████▋   | 3787/5616 [48:10<23:46,  1.28it/s, gpt_loss=0.00883, lr=1.64e-6]
+
Train step of epoch 0:  67%|██████▋   | 3788/5616 [48:10<24:10,  1.26it/s, gpt_loss=0.00883, lr=1.64e-6]
+
Train step of epoch 0:  67%|██████▋   | 3788/5616 [48:11<24:10,  1.26it/s, gpt_loss=0.084, lr=1.64e-6]  
+
Train step of epoch 0:  67%|██████▋   | 3789/5616 [48:11<23:03,  1.32it/s, gpt_loss=0.084, lr=1.64e-6]
+
Train step of epoch 0:  67%|██████▋   | 3789/5616 [48:12<23:03,  1.32it/s, gpt_loss=0.0319, lr=1.64e-6]
+
Train step of epoch 0:  67%|██████▋   | 3790/5616 [48:12<22:17,  1.37it/s, gpt_loss=0.0319, lr=1.64e-6]
+
Train step of epoch 0:  67%|██████▋   | 3790/5616 [48:12<22:17,  1.37it/s, gpt_loss=0.0335, lr=1.64e-6]
+
Train step of epoch 0:  68%|██████▊   | 3791/5616 [48:12<21:48,  1.39it/s, gpt_loss=0.0335, lr=1.64e-6]
+
Train step of epoch 0:  68%|██████▊   | 3791/5616 [48:13<21:48,  1.39it/s, gpt_loss=0.026, lr=1.64e-6] 
+
Train step of epoch 0:  68%|██████▊   | 3792/5616 [48:13<21:42,  1.40it/s, gpt_loss=0.026, lr=1.64e-6]
+
Train step of epoch 0:  68%|██████▊   | 3792/5616 [48:14<21:42,  1.40it/s, gpt_loss=0.0318, lr=1.64e-6]
+
Train step of epoch 0:  68%|██████▊   | 3793/5616 [48:14<21:00,  1.45it/s, gpt_loss=0.0318, lr=1.64e-6]
+
Train step of epoch 0:  68%|██████▊   | 3793/5616 [48:14<21:00,  1.45it/s, gpt_loss=0.0109, lr=1.64e-6]
+
Train step of epoch 0:  68%|██████▊   | 3794/5616 [48:14<20:50,  1.46it/s, gpt_loss=0.0109, lr=1.64e-6]
+
Train step of epoch 0:  68%|██████▊   | 3794/5616 [48:15<20:50,  1.46it/s, gpt_loss=0.01, lr=1.64e-6]  
+
Train step of epoch 0:  68%|██████▊   | 3795/5616 [48:15<21:33,  1.41it/s, gpt_loss=0.01, lr=1.64e-6]
+
Train step of epoch 0:  68%|██████▊   | 3795/5616 [48:16<21:33,  1.41it/s, gpt_loss=0.0189, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3796/5616 [48:16<22:01,  1.38it/s, gpt_loss=0.0189, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3796/5616 [48:17<22:01,  1.38it/s, gpt_loss=0.0397, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3797/5616 [48:17<21:06,  1.44it/s, gpt_loss=0.0397, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3797/5616 [48:17<21:06,  1.44it/s, gpt_loss=0.0694, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3798/5616 [48:17<20:53,  1.45it/s, gpt_loss=0.0694, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3798/5616 [48:18<20:53,  1.45it/s, gpt_loss=0.0171, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3799/5616 [48:18<21:47,  1.39it/s, gpt_loss=0.0171, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3799/5616 [48:19<21:47,  1.39it/s, gpt_loss=0.0143, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3800/5616 [48:19<21:47,  1.39it/s, gpt_loss=0.0143, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3800/5616 [48:19<21:47,  1.39it/s, gpt_loss=0.00525, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3801/5616 [48:19<21:29,  1.41it/s, gpt_loss=0.00525, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3801/5616 [48:20<21:29,  1.41it/s, gpt_loss=0.0262, lr=1.63e-6] 
+
Train step of epoch 0:  68%|██████▊   | 3802/5616 [48:20<21:46,  1.39it/s, gpt_loss=0.0262, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3802/5616 [48:21<21:46,  1.39it/s, gpt_loss=0.0579, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3803/5616 [48:21<22:54,  1.32it/s, gpt_loss=0.0579, lr=1.63e-6]
+
Train step of epoch 0:  68%|██████▊   | 3803/5616 [48:22<22:54,  1.32it/s, gpt_loss=0.0485, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3804/5616 [48:22<23:43,  1.27it/s, gpt_loss=0.0485, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3804/5616 [48:23<23:43,  1.27it/s, gpt_loss=0.00818, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3805/5616 [48:23<24:14,  1.25it/s, gpt_loss=0.00818, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3805/5616 [48:23<24:14,  1.25it/s, gpt_loss=0.0159, lr=1.62e-6] 
+
Train step of epoch 0:  68%|██████▊   | 3806/5616 [48:23<24:02,  1.25it/s, gpt_loss=0.0159, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3806/5616 [48:24<24:02,  1.25it/s, gpt_loss=0.0461, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3807/5616 [48:24<23:16,  1.30it/s, gpt_loss=0.0461, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3807/5616 [48:25<23:16,  1.30it/s, gpt_loss=0.0566, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3808/5616 [48:25<22:53,  1.32it/s, gpt_loss=0.0566, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3808/5616 [48:26<22:53,  1.32it/s, gpt_loss=0.0733, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3809/5616 [48:26<24:24,  1.23it/s, gpt_loss=0.0733, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3809/5616 [48:27<24:24,  1.23it/s, gpt_loss=0.0089, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3810/5616 [48:27<25:20,  1.19it/s, gpt_loss=0.0089, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3810/5616 [48:27<25:20,  1.19it/s, gpt_loss=0.0192, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3811/5616 [48:27<23:36,  1.27it/s, gpt_loss=0.0192, lr=1.62e-6]
+
Train step of epoch 0:  68%|██████▊   | 3811/5616 [48:28<23:36,  1.27it/s, gpt_loss=0.00468, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3812/5616 [48:28<23:52,  1.26it/s, gpt_loss=0.00468, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3812/5616 [48:29<23:52,  1.26it/s, gpt_loss=0.0217, lr=1.61e-6] 
+
Train step of epoch 0:  68%|██████▊   | 3813/5616 [48:29<23:48,  1.26it/s, gpt_loss=0.0217, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3813/5616 [48:30<23:48,  1.26it/s, gpt_loss=0.0264, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3814/5616 [48:30<23:11,  1.30it/s, gpt_loss=0.0264, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3814/5616 [48:30<23:11,  1.30it/s, gpt_loss=0.0361, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3815/5616 [48:30<22:07,  1.36it/s, gpt_loss=0.0361, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3815/5616 [48:31<22:07,  1.36it/s, gpt_loss=0.0286, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3816/5616 [48:31<22:17,  1.35it/s, gpt_loss=0.0286, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3816/5616 [48:32<22:17,  1.35it/s, gpt_loss=0.0268, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3817/5616 [48:32<22:29,  1.33it/s, gpt_loss=0.0268, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3817/5616 [48:33<22:29,  1.33it/s, gpt_loss=0.0216, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3818/5616 [48:33<26:25,  1.13it/s, gpt_loss=0.0216, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3818/5616 [48:34<26:25,  1.13it/s, gpt_loss=0.0433, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3819/5616 [48:34<24:33,  1.22it/s, gpt_loss=0.0433, lr=1.61e-6]
+
Train step of epoch 0:  68%|██████▊   | 3819/5616 [48:35<24:33,  1.22it/s, gpt_loss=0.00754, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3820/5616 [48:35<24:05,  1.24it/s, gpt_loss=0.00754, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3820/5616 [48:35<24:05,  1.24it/s, gpt_loss=0.0322, lr=1.6e-6] 
+
Train step of epoch 0:  68%|██████▊   | 3821/5616 [48:35<24:48,  1.21it/s, gpt_loss=0.0322, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3821/5616 [48:36<24:48,  1.21it/s, gpt_loss=0.0113, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3822/5616 [48:36<23:02,  1.30it/s, gpt_loss=0.0113, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3822/5616 [48:37<23:02,  1.30it/s, gpt_loss=0.0525, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3823/5616 [48:37<24:54,  1.20it/s, gpt_loss=0.0525, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3823/5616 [48:38<24:54,  1.20it/s, gpt_loss=0.0865, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3824/5616 [48:38<25:55,  1.15it/s, gpt_loss=0.0865, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3824/5616 [48:39<25:55,  1.15it/s, gpt_loss=0.0819, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3825/5616 [48:39<24:38,  1.21it/s, gpt_loss=0.0819, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3825/5616 [48:39<24:38,  1.21it/s, gpt_loss=0.0134, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3826/5616 [48:39<22:57,  1.30it/s, gpt_loss=0.0134, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3826/5616 [48:40<22:57,  1.30it/s, gpt_loss=0.00195, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3827/5616 [48:40<21:57,  1.36it/s, gpt_loss=0.00195, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3827/5616 [48:41<21:57,  1.36it/s, gpt_loss=0.0967, lr=1.6e-6] 
+
Train step of epoch 0:  68%|██████▊   | 3828/5616 [48:41<21:58,  1.36it/s, gpt_loss=0.0967, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3828/5616 [48:41<21:58,  1.36it/s, gpt_loss=0.00598, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3829/5616 [48:41<21:41,  1.37it/s, gpt_loss=0.00598, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3829/5616 [48:42<21:41,  1.37it/s, gpt_loss=0.0135, lr=1.6e-6] 
+
Train step of epoch 0:  68%|██████▊   | 3830/5616 [48:42<21:18,  1.40it/s, gpt_loss=0.0135, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3830/5616 [48:43<21:18,  1.40it/s, gpt_loss=0.0151, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3831/5616 [48:43<21:35,  1.38it/s, gpt_loss=0.0151, lr=1.6e-6]
+
Train step of epoch 0:  68%|██████▊   | 3831/5616 [48:44<21:35,  1.38it/s, gpt_loss=0.0339, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3832/5616 [48:44<21:39,  1.37it/s, gpt_loss=0.0339, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3832/5616 [48:44<21:39,  1.37it/s, gpt_loss=0.0473, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3833/5616 [48:44<21:34,  1.38it/s, gpt_loss=0.0473, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3833/5616 [48:45<21:34,  1.38it/s, gpt_loss=0.0902, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3834/5616 [48:45<21:30,  1.38it/s, gpt_loss=0.0902, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3834/5616 [48:46<21:30,  1.38it/s, gpt_loss=0.0748, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3835/5616 [48:46<21:49,  1.36it/s, gpt_loss=0.0748, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3835/5616 [48:47<21:49,  1.36it/s, gpt_loss=0.0855, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3836/5616 [48:47<22:02,  1.35it/s, gpt_loss=0.0855, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3836/5616 [48:47<22:02,  1.35it/s, gpt_loss=0.032, lr=1.59e-6] 
+
Train step of epoch 0:  68%|██████▊   | 3837/5616 [48:47<21:13,  1.40it/s, gpt_loss=0.032, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3837/5616 [48:48<21:13,  1.40it/s, gpt_loss=0.0435, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3838/5616 [48:48<20:37,  1.44it/s, gpt_loss=0.0435, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3838/5616 [48:49<20:37,  1.44it/s, gpt_loss=0.00955, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3839/5616 [48:49<22:16,  1.33it/s, gpt_loss=0.00955, lr=1.59e-6]
+
Train step of epoch 0:  68%|██████▊   | 3839/5616 [48:50<22:16,  1.33it/s, gpt_loss=0.0721, lr=1.58e-6] 
+
Train step of epoch 0:  68%|██████▊   | 3840/5616 [48:50<22:52,  1.29it/s, gpt_loss=0.0721, lr=1.58e-6]
+
Train step of epoch 0:  68%|██████▊   | 3840/5616 [48:50<22:52,  1.29it/s, gpt_loss=0.00882, lr=1.58e-6]
+
Train step of epoch 0:  68%|██████▊   | 3841/5616 [48:50<22:06,  1.34it/s, gpt_loss=0.00882, lr=1.58e-6]
+
Train step of epoch 0:  68%|██████▊   | 3841/5616 [48:51<22:06,  1.34it/s, gpt_loss=0.0749, lr=1.58e-6] 
+
Train step of epoch 0:  68%|██████▊   | 3842/5616 [48:51<22:19,  1.32it/s, gpt_loss=0.0749, lr=1.58e-6]
+
Train step of epoch 0:  68%|██████▊   | 3842/5616 [48:52<22:19,  1.32it/s, gpt_loss=0.115, lr=1.58e-6] 
+
Train step of epoch 0:  68%|██████▊   | 3843/5616 [48:52<22:54,  1.29it/s, gpt_loss=0.115, lr=1.58e-6]
+
Train step of epoch 0:  68%|██████▊   | 3843/5616 [48:53<22:54,  1.29it/s, gpt_loss=0.0439, lr=1.58e-6]
+
Train step of epoch 0:  68%|██████▊   | 3844/5616 [48:53<23:26,  1.26it/s, gpt_loss=0.0439, lr=1.58e-6]
+
Train step of epoch 0:  68%|██████▊   | 3844/5616 [48:53<23:26,  1.26it/s, gpt_loss=0.0694, lr=1.58e-6]
+
Train step of epoch 0:  68%|██████▊   | 3845/5616 [48:53<22:17,  1.32it/s, gpt_loss=0.0694, lr=1.58e-6]
+
Train step of epoch 0:  68%|██████▊   | 3845/5616 [48:54<22:17,  1.32it/s, gpt_loss=0.0262, lr=1.58e-6]
+
Train step of epoch 0:  68%|██████▊   | 3846/5616 [48:54<22:22,  1.32it/s, gpt_loss=0.0262, lr=1.58e-6]
+
Train step of epoch 0:  68%|██████▊   | 3846/5616 [48:55<22:22,  1.32it/s, gpt_loss=0.0449, lr=1.58e-6]
+
Train step of epoch 0:  69%|██████▊   | 3847/5616 [48:55<25:29,  1.16it/s, gpt_loss=0.0449, lr=1.58e-6]
+
Train step of epoch 0:  69%|██████▊   | 3847/5616 [48:56<25:29,  1.16it/s, gpt_loss=0.019, lr=1.57e-6] 
+
Train step of epoch 0:  69%|██████▊   | 3848/5616 [48:56<24:26,  1.21it/s, gpt_loss=0.019, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3848/5616 [48:57<24:26,  1.21it/s, gpt_loss=0.0869, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3849/5616 [48:57<22:48,  1.29it/s, gpt_loss=0.0869, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3849/5616 [48:57<22:48,  1.29it/s, gpt_loss=0.0686, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3850/5616 [48:57<22:59,  1.28it/s, gpt_loss=0.0686, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3850/5616 [48:58<22:59,  1.28it/s, gpt_loss=0.0565, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3851/5616 [48:58<22:16,  1.32it/s, gpt_loss=0.0565, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3851/5616 [48:59<22:16,  1.32it/s, gpt_loss=0.0958, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3852/5616 [48:59<23:12,  1.27it/s, gpt_loss=0.0958, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3852/5616 [49:00<23:12,  1.27it/s, gpt_loss=0.0512, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3853/5616 [49:00<22:05,  1.33it/s, gpt_loss=0.0512, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3853/5616 [49:01<22:05,  1.33it/s, gpt_loss=0.0246, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3854/5616 [49:01<22:47,  1.29it/s, gpt_loss=0.0246, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3854/5616 [49:01<22:47,  1.29it/s, gpt_loss=0.00568, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3855/5616 [49:01<22:34,  1.30it/s, gpt_loss=0.00568, lr=1.57e-6]
+
Train step of epoch 0:  69%|██████▊   | 3855/5616 [49:02<22:34,  1.30it/s, gpt_loss=0.0767, lr=1.56e-6] 
+
Train step of epoch 0:  69%|██████▊   | 3856/5616 [49:02<22:25,  1.31it/s, gpt_loss=0.0767, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▊   | 3856/5616 [49:03<22:25,  1.31it/s, gpt_loss=0.0465, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▊   | 3857/5616 [49:03<22:14,  1.32it/s, gpt_loss=0.0465, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▊   | 3857/5616 [49:03<22:14,  1.32it/s, gpt_loss=0.00915, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▊   | 3858/5616 [49:03<21:43,  1.35it/s, gpt_loss=0.00915, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▊   | 3858/5616 [49:04<21:43,  1.35it/s, gpt_loss=0.13, lr=1.56e-6]   
+
Train step of epoch 0:  69%|██████▊   | 3859/5616 [49:04<21:15,  1.38it/s, gpt_loss=0.13, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▊   | 3859/5616 [49:05<21:15,  1.38it/s, gpt_loss=0.0316, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▊   | 3860/5616 [49:05<21:39,  1.35it/s, gpt_loss=0.0316, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▊   | 3860/5616 [49:06<21:39,  1.35it/s, gpt_loss=0.00877, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▉   | 3861/5616 [49:06<20:48,  1.41it/s, gpt_loss=0.00877, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▉   | 3861/5616 [49:06<20:48,  1.41it/s, gpt_loss=0.0334, lr=1.56e-6] 
+
Train step of epoch 0:  69%|██████▉   | 3862/5616 [49:06<20:53,  1.40it/s, gpt_loss=0.0334, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▉   | 3862/5616 [49:07<20:53,  1.40it/s, gpt_loss=0.0662, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▉   | 3863/5616 [49:07<20:41,  1.41it/s, gpt_loss=0.0662, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▉   | 3863/5616 [49:08<20:41,  1.41it/s, gpt_loss=0.0185, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▉   | 3864/5616 [49:08<22:23,  1.30it/s, gpt_loss=0.0185, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▉   | 3864/5616 [49:09<22:23,  1.30it/s, gpt_loss=0.0263, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▉   | 3865/5616 [49:09<21:53,  1.33it/s, gpt_loss=0.0263, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▉   | 3865/5616 [49:09<21:53,  1.33it/s, gpt_loss=0.1, lr=1.56e-6]   
+
Train step of epoch 0:  69%|██████▉   | 3866/5616 [49:09<22:05,  1.32it/s, gpt_loss=0.1, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▉   | 3866/5616 [49:10<22:05,  1.32it/s, gpt_loss=0.00983, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▉   | 3867/5616 [49:10<21:14,  1.37it/s, gpt_loss=0.00983, lr=1.56e-6]
+
Train step of epoch 0:  69%|██████▉   | 3867/5616 [49:11<21:14,  1.37it/s, gpt_loss=0.044, lr=1.55e-6]  
+
Train step of epoch 0:  69%|██████▉   | 3868/5616 [49:11<22:45,  1.28it/s, gpt_loss=0.044, lr=1.55e-6]
+
Train step of epoch 0:  69%|██████▉   | 3868/5616 [49:12<22:45,  1.28it/s, gpt_loss=0.0329, lr=1.55e-6]
+
Train step of epoch 0:  69%|██████▉   | 3869/5616 [49:12<23:14,  1.25it/s, gpt_loss=0.0329, lr=1.55e-6]
+
Train step of epoch 0:  69%|██████▉   | 3869/5616 [49:12<23:14,  1.25it/s, gpt_loss=0.102, lr=1.55e-6] 
+
Train step of epoch 0:  69%|██████▉   | 3870/5616 [49:12<22:10,  1.31it/s, gpt_loss=0.102, lr=1.55e-6]
+
Train step of epoch 0:  69%|██████▉   | 3870/5616 [49:13<22:10,  1.31it/s, gpt_loss=0.0167, lr=1.55e-6]
+
Train step of epoch 0:  69%|██████▉   | 3871/5616 [49:13<23:51,  1.22it/s, gpt_loss=0.0167, lr=1.55e-6]
+
Train step of epoch 0:  69%|██████▉   | 3871/5616 [49:14<23:51,  1.22it/s, gpt_loss=0.0199, lr=1.55e-6]
+
Train step of epoch 0:  69%|██████▉   | 3872/5616 [49:14<24:02,  1.21it/s, gpt_loss=0.0199, lr=1.55e-6]
+
Train step of epoch 0:  69%|██████▉   | 3872/5616 [49:15<24:02,  1.21it/s, gpt_loss=0.0882, lr=1.55e-6]
+
Train step of epoch 0:  69%|██████▉   | 3873/5616 [49:15<23:33,  1.23it/s, gpt_loss=0.0882, lr=1.55e-6]
+
Train step of epoch 0:  69%|██████▉   | 3873/5616 [49:16<23:33,  1.23it/s, gpt_loss=0.00809, lr=1.55e-6]
+
Train step of epoch 0:  69%|██████▉   | 3874/5616 [49:16<22:44,  1.28it/s, gpt_loss=0.00809, lr=1.55e-6]
+
Train step of epoch 0:  69%|██████▉   | 3874/5616 [49:16<22:44,  1.28it/s, gpt_loss=0.0131, lr=1.55e-6] 
+
Train step of epoch 0:  69%|██████▉   | 3875/5616 [49:16<22:01,  1.32it/s, gpt_loss=0.0131, lr=1.55e-6]
+
Train step of epoch 0:  69%|██████▉   | 3875/5616 [49:17<22:01,  1.32it/s, gpt_loss=0.00669, lr=1.54e-6]
+
Train step of epoch 0:  69%|██████▉   | 3876/5616 [49:17<21:51,  1.33it/s, gpt_loss=0.00669, lr=1.54e-6]
+
Train step of epoch 0:  69%|██████▉   | 3876/5616 [49:18<21:51,  1.33it/s, gpt_loss=0.01, lr=1.54e-6]   
+
Train step of epoch 0:  69%|██████▉   | 3877/5616 [49:18<21:07,  1.37it/s, gpt_loss=0.01, lr=1.54e-6]
+
Train step of epoch 0:  69%|██████▉   | 3877/5616 [49:19<21:07,  1.37it/s, gpt_loss=0.00656, lr=1.54e-6]
+
Train step of epoch 0:  69%|██████▉   | 3878/5616 [49:19<21:13,  1.36it/s, gpt_loss=0.00656, lr=1.54e-6]
+
Train step of epoch 0:  69%|██████▉   | 3878/5616 [49:19<21:13,  1.36it/s, gpt_loss=0.0454, lr=1.54e-6] 
+
Train step of epoch 0:  69%|██████▉   | 3879/5616 [49:19<21:35,  1.34it/s, gpt_loss=0.0454, lr=1.54e-6]
+
Train step of epoch 0:  69%|██████▉   | 3879/5616 [49:20<21:35,  1.34it/s, gpt_loss=0.00893, lr=1.54e-6]
+
Train step of epoch 0:  69%|██████▉   | 3880/5616 [49:20<22:19,  1.30it/s, gpt_loss=0.00893, lr=1.54e-6]
+
Train step of epoch 0:  69%|██████▉   | 3880/5616 [49:21<22:19,  1.30it/s, gpt_loss=0.0294, lr=1.54e-6] 
+
Train step of epoch 0:  69%|██████▉   | 3881/5616 [49:21<21:34,  1.34it/s, gpt_loss=0.0294, lr=1.54e-6]
+
Train step of epoch 0:  69%|██████▉   | 3881/5616 [49:22<21:34,  1.34it/s, gpt_loss=0.0331, lr=1.54e-6]
+
Train step of epoch 0:  69%|██████▉   | 3882/5616 [49:22<21:04,  1.37it/s, gpt_loss=0.0331, lr=1.54e-6]
+
Train step of epoch 0:  69%|██████▉   | 3882/5616 [49:22<21:04,  1.37it/s, gpt_loss=0.0666, lr=1.54e-6]
+
Train step of epoch 0:  69%|██████▉   | 3883/5616 [49:22<20:14,  1.43it/s, gpt_loss=0.0666, lr=1.54e-6]
+
Train step of epoch 0:  69%|██████▉   | 3883/5616 [49:23<20:14,  1.43it/s, gpt_loss=0.00804, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3884/5616 [49:23<20:26,  1.41it/s, gpt_loss=0.00804, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3884/5616 [49:24<20:26,  1.41it/s, gpt_loss=0.00326, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3885/5616 [49:24<20:24,  1.41it/s, gpt_loss=0.00326, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3885/5616 [49:24<20:24,  1.41it/s, gpt_loss=0.0983, lr=1.53e-6] 
+
Train step of epoch 0:  69%|██████▉   | 3886/5616 [49:24<20:07,  1.43it/s, gpt_loss=0.0983, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3886/5616 [49:25<20:07,  1.43it/s, gpt_loss=0.0191, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3887/5616 [49:25<20:57,  1.37it/s, gpt_loss=0.0191, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3887/5616 [49:26<20:57,  1.37it/s, gpt_loss=0.0458, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3888/5616 [49:26<21:16,  1.35it/s, gpt_loss=0.0458, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3888/5616 [49:27<21:16,  1.35it/s, gpt_loss=0.0517, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3889/5616 [49:27<20:49,  1.38it/s, gpt_loss=0.0517, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3889/5616 [49:27<20:49,  1.38it/s, gpt_loss=0.02, lr=1.53e-6]  
+
Train step of epoch 0:  69%|██████▉   | 3890/5616 [49:27<20:32,  1.40it/s, gpt_loss=0.02, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3890/5616 [49:28<20:32,  1.40it/s, gpt_loss=0.0212, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3891/5616 [49:28<20:38,  1.39it/s, gpt_loss=0.0212, lr=1.53e-6]
+
Train step of epoch 0:  69%|██████▉   | 3891/5616 [49:29<20:38,  1.39it/s, gpt_loss=0.0326, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3892/5616 [49:29<22:10,  1.30it/s, gpt_loss=0.0326, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3892/5616 [49:30<22:10,  1.30it/s, gpt_loss=0.0147, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3893/5616 [49:30<21:25,  1.34it/s, gpt_loss=0.0147, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3893/5616 [49:30<21:25,  1.34it/s, gpt_loss=0.0532, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3894/5616 [49:30<21:43,  1.32it/s, gpt_loss=0.0532, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3894/5616 [49:31<21:43,  1.32it/s, gpt_loss=0.0205, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3895/5616 [49:31<21:31,  1.33it/s, gpt_loss=0.0205, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3895/5616 [49:32<21:31,  1.33it/s, gpt_loss=0.00608, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3896/5616 [49:32<21:32,  1.33it/s, gpt_loss=0.00608, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3896/5616 [49:32<21:32,  1.33it/s, gpt_loss=0.0182, lr=1.52e-6] 
+
Train step of epoch 0:  69%|██████▉   | 3897/5616 [49:32<20:36,  1.39it/s, gpt_loss=0.0182, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3897/5616 [49:33<20:36,  1.39it/s, gpt_loss=0.00422, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3898/5616 [49:33<19:52,  1.44it/s, gpt_loss=0.00422, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3898/5616 [49:34<19:52,  1.44it/s, gpt_loss=0.00619, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3899/5616 [49:34<21:07,  1.35it/s, gpt_loss=0.00619, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3899/5616 [49:35<21:07,  1.35it/s, gpt_loss=0.00308, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3900/5616 [49:35<21:08,  1.35it/s, gpt_loss=0.00308, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3900/5616 [49:35<21:08,  1.35it/s, gpt_loss=0.00419, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3901/5616 [49:35<21:18,  1.34it/s, gpt_loss=0.00419, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3901/5616 [49:36<21:18,  1.34it/s, gpt_loss=0.0106, lr=1.52e-6] 
+
Train step of epoch 0:  69%|██████▉   | 3902/5616 [49:36<21:18,  1.34it/s, gpt_loss=0.0106, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3902/5616 [49:37<21:18,  1.34it/s, gpt_loss=0.0231, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3903/5616 [49:37<21:20,  1.34it/s, gpt_loss=0.0231, lr=1.52e-6]
+
Train step of epoch 0:  69%|██████▉   | 3903/5616 [49:38<21:20,  1.34it/s, gpt_loss=0.0323, lr=1.51e-6]
+
Train step of epoch 0:  70%|██████▉   | 3904/5616 [49:38<21:33,  1.32it/s, gpt_loss=0.0323, lr=1.51e-6]
+
Train step of epoch 0:  70%|██████▉   | 3904/5616 [49:38<21:33,  1.32it/s, gpt_loss=0.00581, lr=1.51e-6]
+
Train step of epoch 0:  70%|██████▉   | 3905/5616 [49:38<21:29,  1.33it/s, gpt_loss=0.00581, lr=1.51e-6]
+
Train step of epoch 0:  70%|██████▉   | 3905/5616 [49:39<21:29,  1.33it/s, gpt_loss=0.00271, lr=1.51e-6]
+
Train step of epoch 0:  70%|██████▉   | 3906/5616 [49:39<20:45,  1.37it/s, gpt_loss=0.00271, lr=1.51e-6]
+
Train step of epoch 0:  70%|██████▉   | 3906/5616 [49:40<20:45,  1.37it/s, gpt_loss=0.0998, lr=1.51e-6] 
+
Train step of epoch 0:  70%|██████▉   | 3907/5616 [49:40<20:45,  1.37it/s, gpt_loss=0.0998, lr=1.51e-6]
+
Train step of epoch 0:  70%|██████▉   | 3907/5616 [49:41<20:45,  1.37it/s, gpt_loss=0.00296, lr=1.51e-6]
+
Train step of epoch 0:  70%|██████▉   | 3908/5616 [49:41<21:28,  1.33it/s, gpt_loss=0.00296, lr=1.51e-6]
+
Train step of epoch 0:  70%|██████▉   | 3908/5616 [49:41<21:28,  1.33it/s, gpt_loss=0.0436, lr=1.51e-6] 
+
Train step of epoch 0:  70%|██████▉   | 3909/5616 [49:41<21:19,  1.33it/s, gpt_loss=0.0436, lr=1.51e-6]
+
Train step of epoch 0:  70%|██████▉   | 3909/5616 [49:42<21:19,  1.33it/s, gpt_loss=0.00795, lr=1.51e-6]
+
Train step of epoch 0:  70%|██████▉   | 3910/5616 [49:42<21:45,  1.31it/s, gpt_loss=0.00795, lr=1.51e-6]
+
Train step of epoch 0:  70%|██████▉   | 3910/5616 [49:43<21:45,  1.31it/s, gpt_loss=0.0148, lr=1.51e-6] 
+
Train step of epoch 0:  70%|██████▉   | 3911/5616 [49:43<21:20,  1.33it/s, gpt_loss=0.0148, lr=1.51e-6]
+
Train step of epoch 0:  70%|██████▉   | 3911/5616 [49:44<21:20,  1.33it/s, gpt_loss=0.0663, lr=1.5e-6] 
+
Train step of epoch 0:  70%|██████▉   | 3912/5616 [49:44<21:15,  1.34it/s, gpt_loss=0.0663, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3912/5616 [49:44<21:15,  1.34it/s, gpt_loss=0.0128, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3913/5616 [49:44<20:08,  1.41it/s, gpt_loss=0.0128, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3913/5616 [49:45<20:08,  1.41it/s, gpt_loss=0.0157, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3914/5616 [49:45<19:32,  1.45it/s, gpt_loss=0.0157, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3914/5616 [49:46<19:32,  1.45it/s, gpt_loss=0.0112, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3915/5616 [49:46<19:31,  1.45it/s, gpt_loss=0.0112, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3915/5616 [49:46<19:31,  1.45it/s, gpt_loss=0.0525, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3916/5616 [49:46<20:18,  1.39it/s, gpt_loss=0.0525, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3916/5616 [49:47<20:18,  1.39it/s, gpt_loss=0.0118, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3917/5616 [49:47<20:33,  1.38it/s, gpt_loss=0.0118, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3917/5616 [49:48<20:33,  1.38it/s, gpt_loss=0.0135, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3918/5616 [49:48<20:06,  1.41it/s, gpt_loss=0.0135, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3918/5616 [49:49<20:06,  1.41it/s, gpt_loss=0.0441, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3919/5616 [49:49<19:51,  1.42it/s, gpt_loss=0.0441, lr=1.5e-6]
+
Train step of epoch 0:  70%|██████▉   | 3919/5616 [49:49<19:51,  1.42it/s, gpt_loss=0.00132, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3920/5616 [49:49<20:22,  1.39it/s, gpt_loss=0.00132, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3920/5616 [49:50<20:22,  1.39it/s, gpt_loss=0.0455, lr=1.49e-6] 
+
Train step of epoch 0:  70%|██████▉   | 3921/5616 [49:50<21:35,  1.31it/s, gpt_loss=0.0455, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3921/5616 [49:51<21:35,  1.31it/s, gpt_loss=0.0828, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3922/5616 [49:51<22:38,  1.25it/s, gpt_loss=0.0828, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3922/5616 [49:52<22:38,  1.25it/s, gpt_loss=0.087, lr=1.49e-6] 
+
Train step of epoch 0:  70%|██████▉   | 3923/5616 [49:52<22:11,  1.27it/s, gpt_loss=0.087, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3923/5616 [49:53<22:11,  1.27it/s, gpt_loss=0.0275, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3924/5616 [49:53<23:35,  1.20it/s, gpt_loss=0.0275, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3924/5616 [49:54<23:35,  1.20it/s, gpt_loss=0.112, lr=1.49e-6] 
+
Train step of epoch 0:  70%|██████▉   | 3925/5616 [49:54<22:46,  1.24it/s, gpt_loss=0.112, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3925/5616 [49:55<22:46,  1.24it/s, gpt_loss=0.00257, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3926/5616 [49:55<25:00,  1.13it/s, gpt_loss=0.00257, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3926/5616 [49:55<25:00,  1.13it/s, gpt_loss=0.0111, lr=1.49e-6] 
+
Train step of epoch 0:  70%|██████▉   | 3927/5616 [49:55<23:15,  1.21it/s, gpt_loss=0.0111, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3927/5616 [49:56<23:15,  1.21it/s, gpt_loss=0.0196, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3928/5616 [49:56<23:24,  1.20it/s, gpt_loss=0.0196, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3928/5616 [49:57<23:24,  1.20it/s, gpt_loss=0.0844, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3929/5616 [49:57<22:21,  1.26it/s, gpt_loss=0.0844, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3929/5616 [49:58<22:21,  1.26it/s, gpt_loss=0.00576, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3930/5616 [49:58<23:23,  1.20it/s, gpt_loss=0.00576, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3930/5616 [49:59<23:23,  1.20it/s, gpt_loss=0.0063, lr=1.49e-6] 
+
Train step of epoch 0:  70%|██████▉   | 3931/5616 [49:59<24:55,  1.13it/s, gpt_loss=0.0063, lr=1.49e-6]
+
Train step of epoch 0:  70%|██████▉   | 3931/5616 [50:00<24:55,  1.13it/s, gpt_loss=0.00382, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3932/5616 [50:00<23:59,  1.17it/s, gpt_loss=0.00382, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3932/5616 [50:00<23:59,  1.17it/s, gpt_loss=0.0122, lr=1.48e-6] 
+
Train step of epoch 0:  70%|███████   | 3933/5616 [50:00<22:42,  1.24it/s, gpt_loss=0.0122, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3933/5616 [50:01<22:42,  1.24it/s, gpt_loss=0.0185, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3934/5616 [50:01<21:32,  1.30it/s, gpt_loss=0.0185, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3934/5616 [50:02<21:32,  1.30it/s, gpt_loss=0.0509, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3935/5616 [50:02<21:12,  1.32it/s, gpt_loss=0.0509, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3935/5616 [50:03<21:12,  1.32it/s, gpt_loss=0.0952, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3936/5616 [50:03<23:54,  1.17it/s, gpt_loss=0.0952, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3936/5616 [50:03<23:54,  1.17it/s, gpt_loss=0.095, lr=1.48e-6] 
+
Train step of epoch 0:  70%|███████   | 3937/5616 [50:03<22:32,  1.24it/s, gpt_loss=0.095, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3937/5616 [50:04<22:32,  1.24it/s, gpt_loss=0.0385, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3938/5616 [50:04<21:30,  1.30it/s, gpt_loss=0.0385, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3938/5616 [50:05<21:30,  1.30it/s, gpt_loss=0.0386, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3939/5616 [50:05<22:34,  1.24it/s, gpt_loss=0.0386, lr=1.48e-6]
+
Train step of epoch 0:  70%|███████   | 3939/5616 [50:06<22:34,  1.24it/s, gpt_loss=0.0242, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3940/5616 [50:06<24:38,  1.13it/s, gpt_loss=0.0242, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3940/5616 [50:07<24:38,  1.13it/s, gpt_loss=0.159, lr=1.47e-6] 
+
Train step of epoch 0:  70%|███████   | 3941/5616 [50:07<22:57,  1.22it/s, gpt_loss=0.159, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3941/5616 [50:08<22:57,  1.22it/s, gpt_loss=0.0919, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3942/5616 [50:08<22:40,  1.23it/s, gpt_loss=0.0919, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3942/5616 [50:08<22:40,  1.23it/s, gpt_loss=0.0765, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3943/5616 [50:08<22:13,  1.25it/s, gpt_loss=0.0765, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3943/5616 [50:09<22:13,  1.25it/s, gpt_loss=0.0274, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3944/5616 [50:09<22:00,  1.27it/s, gpt_loss=0.0274, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3944/5616 [50:10<22:00,  1.27it/s, gpt_loss=0.0691, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3945/5616 [50:10<21:39,  1.29it/s, gpt_loss=0.0691, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3945/5616 [50:10<21:39,  1.29it/s, gpt_loss=0.0104, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3946/5616 [50:10<20:55,  1.33it/s, gpt_loss=0.0104, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3946/5616 [50:11<20:55,  1.33it/s, gpt_loss=0.0342, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3947/5616 [50:11<22:45,  1.22it/s, gpt_loss=0.0342, lr=1.47e-6]
+
Train step of epoch 0:  70%|███████   | 3947/5616 [50:12<22:45,  1.22it/s, gpt_loss=0.0192, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3948/5616 [50:12<22:49,  1.22it/s, gpt_loss=0.0192, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3948/5616 [50:13<22:49,  1.22it/s, gpt_loss=0.0604, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3949/5616 [50:13<21:50,  1.27it/s, gpt_loss=0.0604, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3949/5616 [50:14<21:50,  1.27it/s, gpt_loss=0.0326, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3950/5616 [50:14<20:54,  1.33it/s, gpt_loss=0.0326, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3950/5616 [50:14<20:54,  1.33it/s, gpt_loss=0.0592, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3951/5616 [50:14<20:43,  1.34it/s, gpt_loss=0.0592, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3951/5616 [50:15<20:43,  1.34it/s, gpt_loss=0.0273, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3952/5616 [50:15<23:31,  1.18it/s, gpt_loss=0.0273, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3952/5616 [50:16<23:31,  1.18it/s, gpt_loss=0.0592, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3953/5616 [50:16<22:38,  1.22it/s, gpt_loss=0.0592, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3953/5616 [50:17<22:38,  1.22it/s, gpt_loss=0.0969, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3954/5616 [50:17<23:03,  1.20it/s, gpt_loss=0.0969, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3954/5616 [50:18<23:03,  1.20it/s, gpt_loss=0.0409, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3955/5616 [50:18<21:25,  1.29it/s, gpt_loss=0.0409, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3955/5616 [50:19<21:25,  1.29it/s, gpt_loss=0.0456, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3956/5616 [50:19<21:33,  1.28it/s, gpt_loss=0.0456, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3956/5616 [50:19<21:33,  1.28it/s, gpt_loss=0.00282, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3957/5616 [50:19<20:46,  1.33it/s, gpt_loss=0.00282, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3957/5616 [50:20<20:46,  1.33it/s, gpt_loss=0.0138, lr=1.46e-6] 
+
Train step of epoch 0:  70%|███████   | 3958/5616 [50:20<20:08,  1.37it/s, gpt_loss=0.0138, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3958/5616 [50:21<20:08,  1.37it/s, gpt_loss=0.0291, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3959/5616 [50:21<19:49,  1.39it/s, gpt_loss=0.0291, lr=1.46e-6]
+
Train step of epoch 0:  70%|███████   | 3959/5616 [50:21<19:49,  1.39it/s, gpt_loss=0.0678, lr=1.45e-6]
+
Train step of epoch 0:  71%|███████   | 3960/5616 [50:21<21:26,  1.29it/s, gpt_loss=0.0678, lr=1.45e-6]
+
Train step of epoch 0:  71%|███████   | 3960/5616 [50:22<21:26,  1.29it/s, gpt_loss=0.13, lr=1.45e-6]  
+
Train step of epoch 0:  71%|███████   | 3961/5616 [50:22<21:47,  1.27it/s, gpt_loss=0.13, lr=1.45e-6]
+
Train step of epoch 0:  71%|███████   | 3961/5616 [50:23<21:47,  1.27it/s, gpt_loss=0.0444, lr=1.45e-6]
+
Train step of epoch 0:  71%|███████   | 3962/5616 [50:23<22:34,  1.22it/s, gpt_loss=0.0444, lr=1.45e-6]
+
Train step of epoch 0:  71%|███████   | 3962/5616 [50:24<22:34,  1.22it/s, gpt_loss=0.0601, lr=1.45e-6]
+
Train step of epoch 0:  71%|███████   | 3963/5616 [50:24<21:35,  1.28it/s, gpt_loss=0.0601, lr=1.45e-6]
+
Train step of epoch 0:  71%|███████   | 3963/5616 [50:25<21:35,  1.28it/s, gpt_loss=0.0504, lr=1.45e-6]
+
Train step of epoch 0:  71%|███████   | 3964/5616 [50:25<22:43,  1.21it/s, gpt_loss=0.0504, lr=1.45e-6]
+
Train step of epoch 0:  71%|███████   | 3964/5616 [50:26<22:43,  1.21it/s, gpt_loss=0.0207, lr=1.45e-6]
+
Train step of epoch 0:  71%|███████   | 3965/5616 [50:26<21:41,  1.27it/s, gpt_loss=0.0207, lr=1.45e-6]
+
Train step of epoch 0:  71%|███████   | 3965/5616 [50:27<21:41,  1.27it/s, gpt_loss=0.116, lr=1.45e-6] 
+
Train step of epoch 0:  71%|███████   | 3966/5616 [50:27<27:04,  1.02it/s, gpt_loss=0.116, lr=1.45e-6]
+
Train step of epoch 0:  71%|███████   | 3966/5616 [50:28<27:04,  1.02it/s, gpt_loss=0.03, lr=1.45e-6] 
+
Train step of epoch 0:  71%|███████   | 3967/5616 [50:28<24:35,  1.12it/s, gpt_loss=0.03, lr=1.45e-6]
+
Train step of epoch 0:  71%|███████   | 3967/5616 [50:28<24:35,  1.12it/s, gpt_loss=0.053, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3968/5616 [50:28<23:48,  1.15it/s, gpt_loss=0.053, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3968/5616 [50:29<23:48,  1.15it/s, gpt_loss=0.0131, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3969/5616 [50:29<22:21,  1.23it/s, gpt_loss=0.0131, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3969/5616 [50:30<22:21,  1.23it/s, gpt_loss=0.0618, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3970/5616 [50:30<24:32,  1.12it/s, gpt_loss=0.0618, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3970/5616 [50:31<24:32,  1.12it/s, gpt_loss=0.0289, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3971/5616 [50:31<22:54,  1.20it/s, gpt_loss=0.0289, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3971/5616 [50:32<22:54,  1.20it/s, gpt_loss=0.0609, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3972/5616 [50:32<21:53,  1.25it/s, gpt_loss=0.0609, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3972/5616 [50:32<21:53,  1.25it/s, gpt_loss=0.013, lr=1.44e-6] 
+
Train step of epoch 0:  71%|███████   | 3973/5616 [50:32<21:16,  1.29it/s, gpt_loss=0.013, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3973/5616 [50:33<21:16,  1.29it/s, gpt_loss=0.00658, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3974/5616 [50:33<20:11,  1.36it/s, gpt_loss=0.00658, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3974/5616 [50:34<20:11,  1.36it/s, gpt_loss=0.0515, lr=1.44e-6] 
+
Train step of epoch 0:  71%|███████   | 3975/5616 [50:34<19:34,  1.40it/s, gpt_loss=0.0515, lr=1.44e-6]
+
Train step of epoch 0:  71%|███████   | 3975/5616 [50:35<19:34,  1.40it/s, gpt_loss=0.0576, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3976/5616 [50:35<20:46,  1.32it/s, gpt_loss=0.0576, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3976/5616 [50:35<20:46,  1.32it/s, gpt_loss=0.0217, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3977/5616 [50:35<21:53,  1.25it/s, gpt_loss=0.0217, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3977/5616 [50:36<21:53,  1.25it/s, gpt_loss=0.103, lr=1.43e-6] 
+
Train step of epoch 0:  71%|███████   | 3978/5616 [50:36<21:00,  1.30it/s, gpt_loss=0.103, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3978/5616 [50:37<21:00,  1.30it/s, gpt_loss=0.0278, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3979/5616 [50:37<20:26,  1.33it/s, gpt_loss=0.0278, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3979/5616 [50:38<20:26,  1.33it/s, gpt_loss=0.0129, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3980/5616 [50:38<20:28,  1.33it/s, gpt_loss=0.0129, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3980/5616 [50:38<20:28,  1.33it/s, gpt_loss=0.091, lr=1.43e-6] 
+
Train step of epoch 0:  71%|███████   | 3981/5616 [50:38<20:14,  1.35it/s, gpt_loss=0.091, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3981/5616 [50:39<20:14,  1.35it/s, gpt_loss=0.0631, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3982/5616 [50:39<19:53,  1.37it/s, gpt_loss=0.0631, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3982/5616 [50:40<19:53,  1.37it/s, gpt_loss=0.00901, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3983/5616 [50:40<19:50,  1.37it/s, gpt_loss=0.00901, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3983/5616 [50:41<19:50,  1.37it/s, gpt_loss=0.00949, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3984/5616 [50:41<20:28,  1.33it/s, gpt_loss=0.00949, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3984/5616 [50:41<20:28,  1.33it/s, gpt_loss=0.0286, lr=1.43e-6] 
+
Train step of epoch 0:  71%|███████   | 3985/5616 [50:41<19:56,  1.36it/s, gpt_loss=0.0286, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3985/5616 [50:42<19:56,  1.36it/s, gpt_loss=0.00958, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3986/5616 [50:42<19:53,  1.37it/s, gpt_loss=0.00958, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3986/5616 [50:43<19:53,  1.37it/s, gpt_loss=0.0196, lr=1.43e-6] 
+
Train step of epoch 0:  71%|███████   | 3987/5616 [50:43<21:33,  1.26it/s, gpt_loss=0.0196, lr=1.43e-6]
+
Train step of epoch 0:  71%|███████   | 3987/5616 [50:44<21:33,  1.26it/s, gpt_loss=0.0355, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3988/5616 [50:44<21:20,  1.27it/s, gpt_loss=0.0355, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3988/5616 [50:44<21:20,  1.27it/s, gpt_loss=0.0397, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3989/5616 [50:44<21:23,  1.27it/s, gpt_loss=0.0397, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3989/5616 [50:45<21:23,  1.27it/s, gpt_loss=0.04, lr=1.42e-6]  
+
Train step of epoch 0:  71%|███████   | 3990/5616 [50:45<22:54,  1.18it/s, gpt_loss=0.04, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3990/5616 [50:46<22:54,  1.18it/s, gpt_loss=0.0136, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3991/5616 [50:46<21:40,  1.25it/s, gpt_loss=0.0136, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3991/5616 [50:47<21:40,  1.25it/s, gpt_loss=0.0356, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3992/5616 [50:47<21:17,  1.27it/s, gpt_loss=0.0356, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3992/5616 [50:48<21:17,  1.27it/s, gpt_loss=0.0302, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3993/5616 [50:48<20:24,  1.33it/s, gpt_loss=0.0302, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3993/5616 [50:48<20:24,  1.33it/s, gpt_loss=0.0588, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3994/5616 [50:48<19:42,  1.37it/s, gpt_loss=0.0588, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3994/5616 [50:49<19:42,  1.37it/s, gpt_loss=0.0236, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3995/5616 [50:49<20:13,  1.34it/s, gpt_loss=0.0236, lr=1.42e-6]
+
Train step of epoch 0:  71%|███████   | 3995/5616 [50:50<20:13,  1.34it/s, gpt_loss=0.0324, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████   | 3996/5616 [50:50<21:04,  1.28it/s, gpt_loss=0.0324, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████   | 3996/5616 [50:51<21:04,  1.28it/s, gpt_loss=0.00457, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████   | 3997/5616 [50:51<20:48,  1.30it/s, gpt_loss=0.00457, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████   | 3997/5616 [50:52<20:48,  1.30it/s, gpt_loss=0.0886, lr=1.41e-6] 
+
Train step of epoch 0:  71%|███████   | 3998/5616 [50:52<21:58,  1.23it/s, gpt_loss=0.0886, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████   | 3998/5616 [50:52<21:58,  1.23it/s, gpt_loss=0.117, lr=1.41e-6] 
+
Train step of epoch 0:  71%|███████   | 3999/5616 [50:52<21:18,  1.26it/s, gpt_loss=0.117, lr=1.41e-6][2025-06-30 07:29:02,108] [INFO] [logging.py:128:log_dist] [Rank 0] step=1000, skipped=0, lr=[1.4094810292680092e-06, 1.4094810292680092e-06], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 07:29:02,109] [INFO] [timer.py:264:stop] epoch=0/micro_step=4000/global_step=1000, RunningAvgSamplesPerSec=42.457971332585956, CurrSamplesPerSec=40.922711494840286, MemAllocated=16.11GB, MaxMemAllocated=54.86GB
+
+
Train step of epoch 0:  71%|███████   | 3999/5616 [50:53<21:18,  1.26it/s, gpt_loss=0.181, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████   | 4000/5616 [50:53<21:08,  1.27it/s, gpt_loss=0.181, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████   | 4000/5616 [50:54<21:08,  1.27it/s, gpt_loss=0.0522, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████   | 4001/5616 [50:54<21:22,  1.26it/s, gpt_loss=0.0522, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████   | 4001/5616 [50:55<21:22,  1.26it/s, gpt_loss=0.0155, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████▏  | 4002/5616 [50:55<20:53,  1.29it/s, gpt_loss=0.0155, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████▏  | 4002/5616 [50:55<20:53,  1.29it/s, gpt_loss=0.0568, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████▏  | 4003/5616 [50:55<20:07,  1.34it/s, gpt_loss=0.0568, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████▏  | 4003/5616 [50:56<20:07,  1.34it/s, gpt_loss=0.0891, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████▏  | 4004/5616 [50:56<20:14,  1.33it/s, gpt_loss=0.0891, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████▏  | 4004/5616 [50:57<20:14,  1.33it/s, gpt_loss=0.0294, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████▏  | 4005/5616 [50:57<19:57,  1.35it/s, gpt_loss=0.0294, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████▏  | 4005/5616 [50:57<19:57,  1.35it/s, gpt_loss=0.0638, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████▏  | 4006/5616 [50:57<19:27,  1.38it/s, gpt_loss=0.0638, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████▏  | 4006/5616 [50:58<19:27,  1.38it/s, gpt_loss=0.0768, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████▏  | 4007/5616 [50:58<19:26,  1.38it/s, gpt_loss=0.0768, lr=1.41e-6]
+
Train step of epoch 0:  71%|███████▏  | 4007/5616 [50:59<19:26,  1.38it/s, gpt_loss=0.0297, lr=1.4e-6] 
+
Train step of epoch 0:  71%|███████▏  | 4008/5616 [50:59<19:36,  1.37it/s, gpt_loss=0.0297, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4008/5616 [51:00<19:36,  1.37it/s, gpt_loss=0.0479, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4009/5616 [51:00<18:56,  1.41it/s, gpt_loss=0.0479, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4009/5616 [51:00<18:56,  1.41it/s, gpt_loss=0.0485, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4010/5616 [51:00<19:05,  1.40it/s, gpt_loss=0.0485, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4010/5616 [51:01<19:05,  1.40it/s, gpt_loss=0.00433, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4011/5616 [51:01<18:51,  1.42it/s, gpt_loss=0.00433, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4011/5616 [51:02<18:51,  1.42it/s, gpt_loss=0.017, lr=1.4e-6]  
+
Train step of epoch 0:  71%|███████▏  | 4012/5616 [51:02<23:01,  1.16it/s, gpt_loss=0.017, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4012/5616 [51:03<23:01,  1.16it/s, gpt_loss=0.0475, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4013/5616 [51:03<24:41,  1.08it/s, gpt_loss=0.0475, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4013/5616 [51:04<24:41,  1.08it/s, gpt_loss=0.0467, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4014/5616 [51:04<22:32,  1.18it/s, gpt_loss=0.0467, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4014/5616 [51:05<22:32,  1.18it/s, gpt_loss=0.0538, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4015/5616 [51:05<21:08,  1.26it/s, gpt_loss=0.0538, lr=1.4e-6]
+
Train step of epoch 0:  71%|███████▏  | 4015/5616 [51:06<21:08,  1.26it/s, gpt_loss=0.0242, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4016/5616 [51:06<21:56,  1.22it/s, gpt_loss=0.0242, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4016/5616 [51:06<21:56,  1.22it/s, gpt_loss=0.0121, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4017/5616 [51:06<22:10,  1.20it/s, gpt_loss=0.0121, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4017/5616 [51:07<22:10,  1.20it/s, gpt_loss=0.0448, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4018/5616 [51:07<21:20,  1.25it/s, gpt_loss=0.0448, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4018/5616 [51:08<21:20,  1.25it/s, gpt_loss=0.0683, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4019/5616 [51:08<21:15,  1.25it/s, gpt_loss=0.0683, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4019/5616 [51:09<21:15,  1.25it/s, gpt_loss=0.0281, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4020/5616 [51:09<21:14,  1.25it/s, gpt_loss=0.0281, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4020/5616 [51:09<21:14,  1.25it/s, gpt_loss=0.00524, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4021/5616 [51:09<20:40,  1.29it/s, gpt_loss=0.00524, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4021/5616 [51:10<20:40,  1.29it/s, gpt_loss=0.0101, lr=1.39e-6] 
+
Train step of epoch 0:  72%|███████▏  | 4022/5616 [51:10<20:30,  1.30it/s, gpt_loss=0.0101, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4022/5616 [51:11<20:30,  1.30it/s, gpt_loss=0.0503, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4023/5616 [51:11<20:17,  1.31it/s, gpt_loss=0.0503, lr=1.39e-6]
+
Train step of epoch 0:  72%|███████▏  | 4023/5616 [51:12<20:17,  1.31it/s, gpt_loss=0.0466, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4024/5616 [51:12<21:33,  1.23it/s, gpt_loss=0.0466, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4024/5616 [51:13<21:33,  1.23it/s, gpt_loss=0.0139, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4025/5616 [51:13<20:15,  1.31it/s, gpt_loss=0.0139, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4025/5616 [51:13<20:15,  1.31it/s, gpt_loss=0.0597, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4026/5616 [51:13<19:46,  1.34it/s, gpt_loss=0.0597, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4026/5616 [51:14<19:46,  1.34it/s, gpt_loss=0.0574, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4027/5616 [51:14<20:26,  1.30it/s, gpt_loss=0.0574, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4027/5616 [51:15<20:26,  1.30it/s, gpt_loss=0.0158, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4028/5616 [51:15<20:37,  1.28it/s, gpt_loss=0.0158, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4028/5616 [51:16<20:37,  1.28it/s, gpt_loss=0.0831, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4029/5616 [51:16<20:25,  1.29it/s, gpt_loss=0.0831, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4029/5616 [51:16<20:25,  1.29it/s, gpt_loss=0.0172, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4030/5616 [51:16<19:25,  1.36it/s, gpt_loss=0.0172, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4030/5616 [51:17<19:25,  1.36it/s, gpt_loss=0.0136, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4031/5616 [51:17<19:23,  1.36it/s, gpt_loss=0.0136, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4031/5616 [51:18<19:23,  1.36it/s, gpt_loss=0.0165, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4032/5616 [51:18<20:15,  1.30it/s, gpt_loss=0.0165, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4032/5616 [51:19<20:15,  1.30it/s, gpt_loss=0.0994, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4033/5616 [51:19<21:16,  1.24it/s, gpt_loss=0.0994, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4033/5616 [51:19<21:16,  1.24it/s, gpt_loss=0.0231, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4034/5616 [51:19<20:24,  1.29it/s, gpt_loss=0.0231, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4034/5616 [51:20<20:24,  1.29it/s, gpt_loss=0.0612, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4035/5616 [51:20<21:34,  1.22it/s, gpt_loss=0.0612, lr=1.38e-6]
+
Train step of epoch 0:  72%|███████▏  | 4035/5616 [51:21<21:34,  1.22it/s, gpt_loss=0.0099, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4036/5616 [51:21<21:51,  1.20it/s, gpt_loss=0.0099, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4036/5616 [51:22<21:51,  1.20it/s, gpt_loss=0.0689, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4037/5616 [51:22<21:22,  1.23it/s, gpt_loss=0.0689, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4037/5616 [51:23<21:22,  1.23it/s, gpt_loss=0.0155, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4038/5616 [51:23<19:51,  1.32it/s, gpt_loss=0.0155, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4038/5616 [51:23<19:51,  1.32it/s, gpt_loss=0.0162, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4039/5616 [51:23<20:10,  1.30it/s, gpt_loss=0.0162, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4039/5616 [51:24<20:10,  1.30it/s, gpt_loss=0.0276, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4040/5616 [51:24<20:34,  1.28it/s, gpt_loss=0.0276, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4040/5616 [51:25<20:34,  1.28it/s, gpt_loss=0.0247, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4041/5616 [51:25<21:14,  1.24it/s, gpt_loss=0.0247, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4041/5616 [51:26<21:14,  1.24it/s, gpt_loss=0.0234, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4042/5616 [51:26<20:46,  1.26it/s, gpt_loss=0.0234, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4042/5616 [51:27<20:46,  1.26it/s, gpt_loss=0.015, lr=1.37e-6] 
+
Train step of epoch 0:  72%|███████▏  | 4043/5616 [51:27<20:16,  1.29it/s, gpt_loss=0.015, lr=1.37e-6]
+
Train step of epoch 0:  72%|███████▏  | 4043/5616 [51:28<20:16,  1.29it/s, gpt_loss=0.117, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4044/5616 [51:28<23:14,  1.13it/s, gpt_loss=0.117, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4044/5616 [51:29<23:14,  1.13it/s, gpt_loss=0.0213, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4045/5616 [51:29<22:43,  1.15it/s, gpt_loss=0.0213, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4045/5616 [51:29<22:43,  1.15it/s, gpt_loss=0.0616, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4046/5616 [51:29<21:17,  1.23it/s, gpt_loss=0.0616, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4046/5616 [51:30<21:17,  1.23it/s, gpt_loss=0.0657, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4047/5616 [51:30<20:25,  1.28it/s, gpt_loss=0.0657, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4047/5616 [51:31<20:25,  1.28it/s, gpt_loss=0.0236, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4048/5616 [51:31<20:15,  1.29it/s, gpt_loss=0.0236, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4048/5616 [51:32<20:15,  1.29it/s, gpt_loss=0.0326, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4049/5616 [51:32<21:41,  1.20it/s, gpt_loss=0.0326, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4049/5616 [51:32<21:41,  1.20it/s, gpt_loss=0.0211, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4050/5616 [51:32<20:36,  1.27it/s, gpt_loss=0.0211, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4050/5616 [51:33<20:36,  1.27it/s, gpt_loss=0.0173, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4051/5616 [51:33<19:41,  1.33it/s, gpt_loss=0.0173, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4051/5616 [51:34<19:41,  1.33it/s, gpt_loss=0.0383, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4052/5616 [51:34<19:26,  1.34it/s, gpt_loss=0.0383, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4052/5616 [51:35<19:26,  1.34it/s, gpt_loss=0.0118, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4053/5616 [51:35<19:41,  1.32it/s, gpt_loss=0.0118, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4053/5616 [51:35<19:41,  1.32it/s, gpt_loss=0.0118, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4054/5616 [51:35<18:44,  1.39it/s, gpt_loss=0.0118, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4054/5616 [51:36<18:44,  1.39it/s, gpt_loss=0.124, lr=1.36e-6] 
+
Train step of epoch 0:  72%|███████▏  | 4055/5616 [51:36<18:48,  1.38it/s, gpt_loss=0.124, lr=1.36e-6]
+
Train step of epoch 0:  72%|███████▏  | 4055/5616 [51:37<18:48,  1.38it/s, gpt_loss=0.0313, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4056/5616 [51:37<18:36,  1.40it/s, gpt_loss=0.0313, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4056/5616 [51:37<18:36,  1.40it/s, gpt_loss=0.0168, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4057/5616 [51:37<18:19,  1.42it/s, gpt_loss=0.0168, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4057/5616 [51:38<18:19,  1.42it/s, gpt_loss=0.0764, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4058/5616 [51:38<19:23,  1.34it/s, gpt_loss=0.0764, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4058/5616 [51:39<19:23,  1.34it/s, gpt_loss=0.0512, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4059/5616 [51:39<19:10,  1.35it/s, gpt_loss=0.0512, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4059/5616 [51:40<19:10,  1.35it/s, gpt_loss=0.0239, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4060/5616 [51:40<19:39,  1.32it/s, gpt_loss=0.0239, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4060/5616 [51:40<19:39,  1.32it/s, gpt_loss=0.011, lr=1.35e-6] 
+
Train step of epoch 0:  72%|███████▏  | 4061/5616 [51:40<19:45,  1.31it/s, gpt_loss=0.011, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4061/5616 [51:41<19:45,  1.31it/s, gpt_loss=0.0102, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4062/5616 [51:41<18:41,  1.39it/s, gpt_loss=0.0102, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4062/5616 [51:42<18:41,  1.39it/s, gpt_loss=0.0403, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4063/5616 [51:42<19:27,  1.33it/s, gpt_loss=0.0403, lr=1.35e-6]
+
Train step of epoch 0:  72%|███████▏  | 4063/5616 [51:43<19:27,  1.33it/s, gpt_loss=0.0242, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4064/5616 [51:43<19:30,  1.33it/s, gpt_loss=0.0242, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4064/5616 [51:43<19:30,  1.33it/s, gpt_loss=0.0818, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4065/5616 [51:43<18:50,  1.37it/s, gpt_loss=0.0818, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4065/5616 [51:44<18:50,  1.37it/s, gpt_loss=0.047, lr=1.34e-6] 
+
Train step of epoch 0:  72%|███████▏  | 4066/5616 [51:44<19:30,  1.32it/s, gpt_loss=0.047, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4066/5616 [51:45<19:30,  1.32it/s, gpt_loss=0.0232, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4067/5616 [51:45<18:53,  1.37it/s, gpt_loss=0.0232, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4067/5616 [51:46<18:53,  1.37it/s, gpt_loss=0.0549, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4068/5616 [51:46<19:27,  1.33it/s, gpt_loss=0.0549, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4068/5616 [51:46<19:27,  1.33it/s, gpt_loss=0.00783, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4069/5616 [51:46<18:38,  1.38it/s, gpt_loss=0.00783, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4069/5616 [51:47<18:38,  1.38it/s, gpt_loss=0.0783, lr=1.34e-6] 
+
Train step of epoch 0:  72%|███████▏  | 4070/5616 [51:47<18:26,  1.40it/s, gpt_loss=0.0783, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4070/5616 [51:48<18:26,  1.40it/s, gpt_loss=0.0182, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4071/5616 [51:48<19:21,  1.33it/s, gpt_loss=0.0182, lr=1.34e-6]
+
Train step of epoch 0:  72%|███████▏  | 4071/5616 [51:49<19:21,  1.33it/s, gpt_loss=0.0375, lr=1.34e-6]
+
Train step of epoch 0:  73%|███████▎  | 4072/5616 [51:49<19:56,  1.29it/s, gpt_loss=0.0375, lr=1.34e-6]
+
Train step of epoch 0:  73%|███████▎  | 4072/5616 [51:49<19:56,  1.29it/s, gpt_loss=0.028, lr=1.34e-6] 
+
Train step of epoch 0:  73%|███████▎  | 4073/5616 [51:49<19:24,  1.32it/s, gpt_loss=0.028, lr=1.34e-6]
+
Train step of epoch 0:  73%|███████▎  | 4073/5616 [51:50<19:24,  1.32it/s, gpt_loss=0.0376, lr=1.34e-6]
+
Train step of epoch 0:  73%|███████▎  | 4074/5616 [51:50<18:59,  1.35it/s, gpt_loss=0.0376, lr=1.34e-6]
+
Train step of epoch 0:  73%|███████▎  | 4074/5616 [51:51<18:59,  1.35it/s, gpt_loss=0.0147, lr=1.34e-6]
+
Train step of epoch 0:  73%|███████▎  | 4075/5616 [51:51<18:25,  1.39it/s, gpt_loss=0.0147, lr=1.34e-6]
+
Train step of epoch 0:  73%|███████▎  | 4075/5616 [51:52<18:25,  1.39it/s, gpt_loss=0.0113, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4076/5616 [51:52<19:32,  1.31it/s, gpt_loss=0.0113, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4076/5616 [51:52<19:32,  1.31it/s, gpt_loss=0.0155, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4077/5616 [51:52<19:36,  1.31it/s, gpt_loss=0.0155, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4077/5616 [51:53<19:36,  1.31it/s, gpt_loss=0.00768, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4078/5616 [51:53<18:52,  1.36it/s, gpt_loss=0.00768, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4078/5616 [51:54<18:52,  1.36it/s, gpt_loss=0.0155, lr=1.33e-6] 
+
Train step of epoch 0:  73%|███████▎  | 4079/5616 [51:54<18:32,  1.38it/s, gpt_loss=0.0155, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4079/5616 [51:54<18:32,  1.38it/s, gpt_loss=0.0247, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4080/5616 [51:54<19:01,  1.35it/s, gpt_loss=0.0247, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4080/5616 [51:55<19:01,  1.35it/s, gpt_loss=0.00355, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4081/5616 [51:55<18:17,  1.40it/s, gpt_loss=0.00355, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4081/5616 [51:56<18:17,  1.40it/s, gpt_loss=0.137, lr=1.33e-6]  
+
Train step of epoch 0:  73%|███████▎  | 4082/5616 [51:56<18:57,  1.35it/s, gpt_loss=0.137, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4082/5616 [51:57<18:57,  1.35it/s, gpt_loss=0.0157, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4083/5616 [51:57<18:20,  1.39it/s, gpt_loss=0.0157, lr=1.33e-6]
+
Train step of epoch 0:  73%|███████▎  | 4083/5616 [51:57<18:20,  1.39it/s, gpt_loss=0.0245, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4084/5616 [51:57<19:24,  1.32it/s, gpt_loss=0.0245, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4084/5616 [51:58<19:24,  1.32it/s, gpt_loss=0.0267, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4085/5616 [51:58<19:20,  1.32it/s, gpt_loss=0.0267, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4085/5616 [51:59<19:20,  1.32it/s, gpt_loss=0.00674, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4086/5616 [51:59<18:48,  1.36it/s, gpt_loss=0.00674, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4086/5616 [52:00<18:48,  1.36it/s, gpt_loss=0.0171, lr=1.32e-6] 
+
Train step of epoch 0:  73%|███████▎  | 4087/5616 [52:00<18:33,  1.37it/s, gpt_loss=0.0171, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4087/5616 [52:00<18:33,  1.37it/s, gpt_loss=0.0662, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4088/5616 [52:00<19:14,  1.32it/s, gpt_loss=0.0662, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4088/5616 [52:01<19:14,  1.32it/s, gpt_loss=0.0408, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4089/5616 [52:01<19:22,  1.31it/s, gpt_loss=0.0408, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4089/5616 [52:02<19:22,  1.31it/s, gpt_loss=0.00449, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4090/5616 [52:02<18:41,  1.36it/s, gpt_loss=0.00449, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4090/5616 [52:03<18:41,  1.36it/s, gpt_loss=0.0797, lr=1.32e-6] 
+
Train step of epoch 0:  73%|███████▎  | 4091/5616 [52:03<18:36,  1.37it/s, gpt_loss=0.0797, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4091/5616 [52:03<18:36,  1.37it/s, gpt_loss=0.0189, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4092/5616 [52:03<19:10,  1.32it/s, gpt_loss=0.0189, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4092/5616 [52:05<19:10,  1.32it/s, gpt_loss=0.00251, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4093/5616 [52:05<22:05,  1.15it/s, gpt_loss=0.00251, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4093/5616 [52:05<22:05,  1.15it/s, gpt_loss=0.0609, lr=1.32e-6] 
+
Train step of epoch 0:  73%|███████▎  | 4094/5616 [52:05<21:53,  1.16it/s, gpt_loss=0.0609, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4094/5616 [52:06<21:53,  1.16it/s, gpt_loss=0.0847, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4095/5616 [52:06<20:24,  1.24it/s, gpt_loss=0.0847, lr=1.32e-6]
+
Train step of epoch 0:  73%|███████▎  | 4095/5616 [52:07<20:24,  1.24it/s, gpt_loss=0.0192, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4096/5616 [52:07<19:42,  1.29it/s, gpt_loss=0.0192, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4096/5616 [52:07<19:42,  1.29it/s, gpt_loss=0.0183, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4097/5616 [52:07<18:43,  1.35it/s, gpt_loss=0.0183, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4097/5616 [52:08<18:43,  1.35it/s, gpt_loss=0.00929, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4098/5616 [52:08<18:10,  1.39it/s, gpt_loss=0.00929, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4098/5616 [52:09<18:10,  1.39it/s, gpt_loss=0.0164, lr=1.31e-6] 
+
Train step of epoch 0:  73%|███████▎  | 4099/5616 [52:09<18:23,  1.37it/s, gpt_loss=0.0164, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4099/5616 [52:10<18:23,  1.37it/s, gpt_loss=0.00781, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4100/5616 [52:10<19:53,  1.27it/s, gpt_loss=0.00781, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4100/5616 [52:10<19:53,  1.27it/s, gpt_loss=0.011, lr=1.31e-6]  
+
Train step of epoch 0:  73%|███████▎  | 4101/5616 [52:10<18:58,  1.33it/s, gpt_loss=0.011, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4101/5616 [52:11<18:58,  1.33it/s, gpt_loss=0.0563, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4102/5616 [52:11<18:22,  1.37it/s, gpt_loss=0.0563, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4102/5616 [52:12<18:22,  1.37it/s, gpt_loss=0.0294, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4103/5616 [52:12<17:59,  1.40it/s, gpt_loss=0.0294, lr=1.31e-6]
+
Train step of epoch 0:  73%|███████▎  | 4103/5616 [52:13<17:59,  1.40it/s, gpt_loss=0.0136, lr=1.3e-6] 
+
Train step of epoch 0:  73%|███████▎  | 4104/5616 [52:13<19:33,  1.29it/s, gpt_loss=0.0136, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4104/5616 [52:13<19:33,  1.29it/s, gpt_loss=0.0232, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4105/5616 [52:13<18:56,  1.33it/s, gpt_loss=0.0232, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4105/5616 [52:14<18:56,  1.33it/s, gpt_loss=0.0504, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4106/5616 [52:14<18:14,  1.38it/s, gpt_loss=0.0504, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4106/5616 [52:15<18:14,  1.38it/s, gpt_loss=0.00498, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4107/5616 [52:15<18:36,  1.35it/s, gpt_loss=0.00498, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4107/5616 [52:16<18:36,  1.35it/s, gpt_loss=0.0396, lr=1.3e-6] 
+
Train step of epoch 0:  73%|███████▎  | 4108/5616 [52:16<19:45,  1.27it/s, gpt_loss=0.0396, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4108/5616 [52:17<19:45,  1.27it/s, gpt_loss=0.0504, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4109/5616 [52:17<19:47,  1.27it/s, gpt_loss=0.0504, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4109/5616 [52:17<19:47,  1.27it/s, gpt_loss=0.00513, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4110/5616 [52:17<18:50,  1.33it/s, gpt_loss=0.00513, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4110/5616 [52:18<18:50,  1.33it/s, gpt_loss=0.0106, lr=1.3e-6] 
+
Train step of epoch 0:  73%|███████▎  | 4111/5616 [52:18<18:00,  1.39it/s, gpt_loss=0.0106, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4111/5616 [52:19<18:00,  1.39it/s, gpt_loss=0.0204, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4112/5616 [52:19<18:29,  1.36it/s, gpt_loss=0.0204, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4112/5616 [52:20<18:29,  1.36it/s, gpt_loss=0.0104, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4113/5616 [52:20<20:44,  1.21it/s, gpt_loss=0.0104, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4113/5616 [52:20<20:44,  1.21it/s, gpt_loss=0.00875, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4114/5616 [52:20<20:41,  1.21it/s, gpt_loss=0.00875, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4114/5616 [52:21<20:41,  1.21it/s, gpt_loss=0.16, lr=1.3e-6]   
+
Train step of epoch 0:  73%|███████▎  | 4115/5616 [52:21<19:33,  1.28it/s, gpt_loss=0.16, lr=1.3e-6]
+
Train step of epoch 0:  73%|███████▎  | 4115/5616 [52:22<19:33,  1.28it/s, gpt_loss=0.12, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4116/5616 [52:22<19:58,  1.25it/s, gpt_loss=0.12, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4116/5616 [52:23<19:58,  1.25it/s, gpt_loss=0.0308, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4117/5616 [52:23<19:02,  1.31it/s, gpt_loss=0.0308, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4117/5616 [52:23<19:02,  1.31it/s, gpt_loss=0.00276, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4118/5616 [52:23<18:52,  1.32it/s, gpt_loss=0.00276, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4118/5616 [52:24<18:52,  1.32it/s, gpt_loss=0.00997, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4119/5616 [52:24<18:28,  1.35it/s, gpt_loss=0.00997, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4119/5616 [52:25<18:28,  1.35it/s, gpt_loss=0.00318, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4120/5616 [52:25<18:56,  1.32it/s, gpt_loss=0.00318, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4120/5616 [52:26<18:56,  1.32it/s, gpt_loss=0.091, lr=1.29e-6]  
+
Train step of epoch 0:  73%|███████▎  | 4121/5616 [52:26<19:18,  1.29it/s, gpt_loss=0.091, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4121/5616 [52:26<19:18,  1.29it/s, gpt_loss=0.162, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4122/5616 [52:26<18:18,  1.36it/s, gpt_loss=0.162, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4122/5616 [52:27<18:18,  1.36it/s, gpt_loss=0.0482, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4123/5616 [52:27<17:48,  1.40it/s, gpt_loss=0.0482, lr=1.29e-6]
+
Train step of epoch 0:  73%|███████▎  | 4123/5616 [52:28<17:48,  1.40it/s, gpt_loss=0.0906, lr=1.28e-6]
+
Train step of epoch 0:  73%|███████▎  | 4124/5616 [52:28<18:30,  1.34it/s, gpt_loss=0.0906, lr=1.28e-6]
+
Train step of epoch 0:  73%|███████▎  | 4124/5616 [52:29<18:30,  1.34it/s, gpt_loss=0.139, lr=1.28e-6] 
+
Train step of epoch 0:  73%|███████▎  | 4125/5616 [52:29<20:08,  1.23it/s, gpt_loss=0.139, lr=1.28e-6]
+
Train step of epoch 0:  73%|███████▎  | 4125/5616 [52:30<20:08,  1.23it/s, gpt_loss=0.0106, lr=1.28e-6]
+
Train step of epoch 0:  73%|███████▎  | 4126/5616 [52:30<19:38,  1.26it/s, gpt_loss=0.0106, lr=1.28e-6]
+
Train step of epoch 0:  73%|███████▎  | 4126/5616 [52:30<19:38,  1.26it/s, gpt_loss=0.00394, lr=1.28e-6]
+
Train step of epoch 0:  73%|███████▎  | 4127/5616 [52:30<19:05,  1.30it/s, gpt_loss=0.00394, lr=1.28e-6]
+
Train step of epoch 0:  73%|███████▎  | 4127/5616 [52:31<19:05,  1.30it/s, gpt_loss=0.0015, lr=1.28e-6] 
+
Train step of epoch 0:  74%|███████▎  | 4128/5616 [52:31<19:47,  1.25it/s, gpt_loss=0.0015, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4128/5616 [52:32<19:47,  1.25it/s, gpt_loss=0.0185, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4129/5616 [52:32<19:22,  1.28it/s, gpt_loss=0.0185, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4129/5616 [52:33<19:22,  1.28it/s, gpt_loss=0.0129, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4130/5616 [52:33<19:09,  1.29it/s, gpt_loss=0.0129, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4130/5616 [52:33<19:09,  1.29it/s, gpt_loss=0.0368, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4131/5616 [52:33<19:05,  1.30it/s, gpt_loss=0.0368, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4131/5616 [52:34<19:05,  1.30it/s, gpt_loss=0.0462, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4132/5616 [52:34<19:24,  1.27it/s, gpt_loss=0.0462, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4132/5616 [52:35<19:24,  1.27it/s, gpt_loss=0.00461, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4133/5616 [52:35<18:58,  1.30it/s, gpt_loss=0.00461, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4133/5616 [52:36<18:58,  1.30it/s, gpt_loss=0.0443, lr=1.28e-6] 
+
Train step of epoch 0:  74%|███████▎  | 4134/5616 [52:36<18:06,  1.36it/s, gpt_loss=0.0443, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4134/5616 [52:36<18:06,  1.36it/s, gpt_loss=0.0168, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4135/5616 [52:36<18:25,  1.34it/s, gpt_loss=0.0168, lr=1.28e-6]
+
Train step of epoch 0:  74%|███████▎  | 4135/5616 [52:37<18:25,  1.34it/s, gpt_loss=0.0376, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▎  | 4136/5616 [52:37<19:06,  1.29it/s, gpt_loss=0.0376, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▎  | 4136/5616 [52:38<19:06,  1.29it/s, gpt_loss=0.00891, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▎  | 4137/5616 [52:38<19:34,  1.26it/s, gpt_loss=0.00891, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▎  | 4137/5616 [52:39<19:34,  1.26it/s, gpt_loss=0.0259, lr=1.27e-6] 
+
Train step of epoch 0:  74%|███████▎  | 4138/5616 [52:39<18:50,  1.31it/s, gpt_loss=0.0259, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▎  | 4138/5616 [52:39<18:50,  1.31it/s, gpt_loss=0.0127, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▎  | 4139/5616 [52:39<18:19,  1.34it/s, gpt_loss=0.0127, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▎  | 4139/5616 [52:40<18:19,  1.34it/s, gpt_loss=0.0564, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▎  | 4140/5616 [52:40<19:21,  1.27it/s, gpt_loss=0.0564, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▎  | 4140/5616 [52:41<19:21,  1.27it/s, gpt_loss=0.149, lr=1.27e-6] 
+
Train step of epoch 0:  74%|███████▎  | 4141/5616 [52:41<18:29,  1.33it/s, gpt_loss=0.149, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▎  | 4141/5616 [52:42<18:29,  1.33it/s, gpt_loss=0.0507, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▍  | 4142/5616 [52:42<18:36,  1.32it/s, gpt_loss=0.0507, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▍  | 4142/5616 [52:43<18:36,  1.32it/s, gpt_loss=0.0101, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▍  | 4143/5616 [52:43<18:53,  1.30it/s, gpt_loss=0.0101, lr=1.27e-6]
+
Train step of epoch 0:  74%|███████▍  | 4143/5616 [52:43<18:53,  1.30it/s, gpt_loss=0.00578, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4144/5616 [52:43<18:53,  1.30it/s, gpt_loss=0.00578, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4144/5616 [52:44<18:53,  1.30it/s, gpt_loss=0.00726, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4145/5616 [52:44<18:59,  1.29it/s, gpt_loss=0.00726, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4145/5616 [52:45<18:59,  1.29it/s, gpt_loss=0.0236, lr=1.26e-6] 
+
Train step of epoch 0:  74%|███████▍  | 4146/5616 [52:45<18:03,  1.36it/s, gpt_loss=0.0236, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4146/5616 [52:45<18:03,  1.36it/s, gpt_loss=0.0573, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4147/5616 [52:45<17:24,  1.41it/s, gpt_loss=0.0573, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4147/5616 [52:46<17:24,  1.41it/s, gpt_loss=0.0129, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4148/5616 [52:46<17:48,  1.37it/s, gpt_loss=0.0129, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4148/5616 [52:47<17:48,  1.37it/s, gpt_loss=0.0284, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4149/5616 [52:47<17:36,  1.39it/s, gpt_loss=0.0284, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4149/5616 [52:48<17:36,  1.39it/s, gpt_loss=0.0549, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4150/5616 [52:48<17:27,  1.40it/s, gpt_loss=0.0549, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4150/5616 [52:49<17:27,  1.40it/s, gpt_loss=0.0383, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4151/5616 [52:49<20:11,  1.21it/s, gpt_loss=0.0383, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4151/5616 [52:49<20:11,  1.21it/s, gpt_loss=0.0136, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4152/5616 [52:49<19:49,  1.23it/s, gpt_loss=0.0136, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4152/5616 [52:50<19:49,  1.23it/s, gpt_loss=0.142, lr=1.26e-6] 
+
Train step of epoch 0:  74%|███████▍  | 4153/5616 [52:50<19:23,  1.26it/s, gpt_loss=0.142, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4153/5616 [52:51<19:23,  1.26it/s, gpt_loss=0.00373, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4154/5616 [52:51<18:55,  1.29it/s, gpt_loss=0.00373, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4154/5616 [52:52<18:55,  1.29it/s, gpt_loss=0.00261, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4155/5616 [52:52<17:55,  1.36it/s, gpt_loss=0.00261, lr=1.26e-6]
+
Train step of epoch 0:  74%|███████▍  | 4155/5616 [52:52<17:55,  1.36it/s, gpt_loss=0.00545, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4156/5616 [52:52<18:32,  1.31it/s, gpt_loss=0.00545, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4156/5616 [52:53<18:32,  1.31it/s, gpt_loss=0.0264, lr=1.25e-6] 
+
Train step of epoch 0:  74%|███████▍  | 4157/5616 [52:53<20:46,  1.17it/s, gpt_loss=0.0264, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4157/5616 [52:54<20:46,  1.17it/s, gpt_loss=0.0215, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4158/5616 [52:54<20:20,  1.19it/s, gpt_loss=0.0215, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4158/5616 [52:55<20:20,  1.19it/s, gpt_loss=0.0204, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4159/5616 [52:55<19:09,  1.27it/s, gpt_loss=0.0204, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4159/5616 [52:56<19:09,  1.27it/s, gpt_loss=0.01, lr=1.25e-6]  
+
Train step of epoch 0:  74%|███████▍  | 4160/5616 [52:56<19:17,  1.26it/s, gpt_loss=0.01, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4160/5616 [52:57<19:17,  1.26it/s, gpt_loss=0.0501, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4161/5616 [52:57<19:10,  1.26it/s, gpt_loss=0.0501, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4161/5616 [52:58<19:10,  1.26it/s, gpt_loss=0.00834, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4162/5616 [52:58<20:21,  1.19it/s, gpt_loss=0.00834, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4162/5616 [52:58<20:21,  1.19it/s, gpt_loss=0.00514, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4163/5616 [52:58<20:13,  1.20it/s, gpt_loss=0.00514, lr=1.25e-6]
+
Train step of epoch 0:  74%|███████▍  | 4163/5616 [52:59<20:13,  1.20it/s, gpt_loss=0.00543, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4164/5616 [52:59<20:37,  1.17it/s, gpt_loss=0.00543, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4164/5616 [53:00<20:37,  1.17it/s, gpt_loss=0.0599, lr=1.24e-6] 
+
Train step of epoch 0:  74%|███████▍  | 4165/5616 [53:00<19:03,  1.27it/s, gpt_loss=0.0599, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4165/5616 [53:01<19:03,  1.27it/s, gpt_loss=0.0222, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4166/5616 [53:01<18:35,  1.30it/s, gpt_loss=0.0222, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4166/5616 [53:01<18:35,  1.30it/s, gpt_loss=0.0114, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4167/5616 [53:01<18:25,  1.31it/s, gpt_loss=0.0114, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4167/5616 [53:02<18:25,  1.31it/s, gpt_loss=0.0119, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4168/5616 [53:02<19:01,  1.27it/s, gpt_loss=0.0119, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4168/5616 [53:03<19:01,  1.27it/s, gpt_loss=0.0216, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4169/5616 [53:03<19:06,  1.26it/s, gpt_loss=0.0216, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4169/5616 [53:04<19:06,  1.26it/s, gpt_loss=0.0338, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4170/5616 [53:04<18:21,  1.31it/s, gpt_loss=0.0338, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4170/5616 [53:04<18:21,  1.31it/s, gpt_loss=0.0245, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4171/5616 [53:04<17:37,  1.37it/s, gpt_loss=0.0245, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4171/5616 [53:05<17:37,  1.37it/s, gpt_loss=0.00477, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4172/5616 [53:05<17:44,  1.36it/s, gpt_loss=0.00477, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4172/5616 [53:06<17:44,  1.36it/s, gpt_loss=0.0186, lr=1.24e-6] 
+
Train step of epoch 0:  74%|███████▍  | 4173/5616 [53:06<17:37,  1.36it/s, gpt_loss=0.0186, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4173/5616 [53:07<17:37,  1.36it/s, gpt_loss=0.0165, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4174/5616 [53:07<17:52,  1.34it/s, gpt_loss=0.0165, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4174/5616 [53:07<17:52,  1.34it/s, gpt_loss=0.0397, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4175/5616 [53:07<18:24,  1.30it/s, gpt_loss=0.0397, lr=1.24e-6]
+
Train step of epoch 0:  74%|███████▍  | 4175/5616 [53:08<18:24,  1.30it/s, gpt_loss=0.00314, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4176/5616 [53:08<18:24,  1.30it/s, gpt_loss=0.00314, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4176/5616 [53:09<18:24,  1.30it/s, gpt_loss=0.063, lr=1.23e-6]  
+
Train step of epoch 0:  74%|███████▍  | 4177/5616 [53:09<18:17,  1.31it/s, gpt_loss=0.063, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4177/5616 [53:10<18:17,  1.31it/s, gpt_loss=0.0418, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4178/5616 [53:10<17:46,  1.35it/s, gpt_loss=0.0418, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4178/5616 [53:10<17:46,  1.35it/s, gpt_loss=0.0312, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4179/5616 [53:10<18:11,  1.32it/s, gpt_loss=0.0312, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4179/5616 [53:11<18:11,  1.32it/s, gpt_loss=0.00225, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4180/5616 [53:11<18:05,  1.32it/s, gpt_loss=0.00225, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4180/5616 [53:12<18:05,  1.32it/s, gpt_loss=0.0271, lr=1.23e-6] 
+
Train step of epoch 0:  74%|███████▍  | 4181/5616 [53:12<17:34,  1.36it/s, gpt_loss=0.0271, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4181/5616 [53:13<17:34,  1.36it/s, gpt_loss=0.0606, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4182/5616 [53:13<17:10,  1.39it/s, gpt_loss=0.0606, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4182/5616 [53:13<17:10,  1.39it/s, gpt_loss=0.0202, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4183/5616 [53:13<18:10,  1.31it/s, gpt_loss=0.0202, lr=1.23e-6]
+
Train step of epoch 0:  74%|███████▍  | 4183/5616 [53:14<18:10,  1.31it/s, gpt_loss=0.00168, lr=1.23e-6]
+
Train step of epoch 0:  75%|███████▍  | 4184/5616 [53:14<19:09,  1.25it/s, gpt_loss=0.00168, lr=1.23e-6]
+
Train step of epoch 0:  75%|███████▍  | 4184/5616 [53:15<19:09,  1.25it/s, gpt_loss=0.0154, lr=1.23e-6] 
+
Train step of epoch 0:  75%|███████▍  | 4185/5616 [53:15<18:46,  1.27it/s, gpt_loss=0.0154, lr=1.23e-6]
+
Train step of epoch 0:  75%|███████▍  | 4185/5616 [53:16<18:46,  1.27it/s, gpt_loss=0.0622, lr=1.23e-6]
+
Train step of epoch 0:  75%|███████▍  | 4186/5616 [53:16<18:21,  1.30it/s, gpt_loss=0.0622, lr=1.23e-6]
+
Train step of epoch 0:  75%|███████▍  | 4186/5616 [53:17<18:21,  1.30it/s, gpt_loss=0.0782, lr=1.23e-6]
+
Train step of epoch 0:  75%|███████▍  | 4187/5616 [53:17<18:04,  1.32it/s, gpt_loss=0.0782, lr=1.23e-6]
+
Train step of epoch 0:  75%|███████▍  | 4187/5616 [53:17<18:04,  1.32it/s, gpt_loss=0.00909, lr=1.22e-6]
+
Train step of epoch 0:  75%|███████▍  | 4188/5616 [53:17<17:57,  1.33it/s, gpt_loss=0.00909, lr=1.22e-6]
+
Train step of epoch 0:  75%|███████▍  | 4188/5616 [53:18<17:57,  1.33it/s, gpt_loss=0.168, lr=1.22e-6]  
+
Train step of epoch 0:  75%|███████▍  | 4189/5616 [53:18<17:53,  1.33it/s, gpt_loss=0.168, lr=1.22e-6]
+
Train step of epoch 0:  75%|███████▍  | 4189/5616 [53:19<17:53,  1.33it/s, gpt_loss=0.00385, lr=1.22e-6]
+
Train step of epoch 0:  75%|███████▍  | 4190/5616 [53:19<17:20,  1.37it/s, gpt_loss=0.00385, lr=1.22e-6]
+
Train step of epoch 0:  75%|███████▍  | 4190/5616 [53:19<17:20,  1.37it/s, gpt_loss=0.0542, lr=1.22e-6] 
+
Train step of epoch 0:  75%|███████▍  | 4191/5616 [53:19<16:45,  1.42it/s, gpt_loss=0.0542, lr=1.22e-6]
+
Train step of epoch 0:  75%|███████▍  | 4191/5616 [53:20<16:45,  1.42it/s, gpt_loss=0.00291, lr=1.22e-6]
+
Train step of epoch 0:  75%|███████▍  | 4192/5616 [53:20<17:06,  1.39it/s, gpt_loss=0.00291, lr=1.22e-6]
+
Train step of epoch 0:  75%|███████▍  | 4192/5616 [53:21<17:06,  1.39it/s, gpt_loss=0.0251, lr=1.22e-6] 
+
Train step of epoch 0:  75%|███████▍  | 4193/5616 [53:21<18:05,  1.31it/s, gpt_loss=0.0251, lr=1.22e-6]
+
Train step of epoch 0:  75%|███████▍  | 4193/5616 [53:22<18:05,  1.31it/s, gpt_loss=0.00111, lr=1.22e-6]
+
Train step of epoch 0:  75%|███████▍  | 4194/5616 [53:22<17:39,  1.34it/s, gpt_loss=0.00111, lr=1.22e-6]
+
Train step of epoch 0:  75%|███████▍  | 4194/5616 [53:22<17:39,  1.34it/s, gpt_loss=0.00782, lr=1.22e-6]
+
Train step of epoch 0:  75%|███████▍  | 4195/5616 [53:22<17:10,  1.38it/s, gpt_loss=0.00782, lr=1.22e-6]
+
Train step of epoch 0:  75%|███████▍  | 4195/5616 [53:23<17:10,  1.38it/s, gpt_loss=0.0557, lr=1.21e-6] 
+
Train step of epoch 0:  75%|███████▍  | 4196/5616 [53:23<17:42,  1.34it/s, gpt_loss=0.0557, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4196/5616 [53:24<17:42,  1.34it/s, gpt_loss=0.0231, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4197/5616 [53:24<17:36,  1.34it/s, gpt_loss=0.0231, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4197/5616 [53:25<17:36,  1.34it/s, gpt_loss=0.00605, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4198/5616 [53:25<17:02,  1.39it/s, gpt_loss=0.00605, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4198/5616 [53:25<17:02,  1.39it/s, gpt_loss=0.0388, lr=1.21e-6] 
+
Train step of epoch 0:  75%|███████▍  | 4199/5616 [53:25<18:24,  1.28it/s, gpt_loss=0.0388, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4199/5616 [53:26<18:24,  1.28it/s, gpt_loss=0.0913, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4200/5616 [53:26<18:19,  1.29it/s, gpt_loss=0.0913, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4200/5616 [53:27<18:19,  1.29it/s, gpt_loss=0.033, lr=1.21e-6] 
+
Train step of epoch 0:  75%|███████��  | 4201/5616 [53:27<18:00,  1.31it/s, gpt_loss=0.033, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4201/5616 [53:28<18:00,  1.31it/s, gpt_loss=0.0906, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4202/5616 [53:28<17:33,  1.34it/s, gpt_loss=0.0906, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4202/5616 [53:28<17:33,  1.34it/s, gpt_loss=0.0199, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4203/5616 [53:28<18:08,  1.30it/s, gpt_loss=0.0199, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4203/5616 [53:29<18:08,  1.30it/s, gpt_loss=0.00432, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4204/5616 [53:29<18:25,  1.28it/s, gpt_loss=0.00432, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4204/5616 [53:30<18:25,  1.28it/s, gpt_loss=0.0031, lr=1.21e-6] 
+
Train step of epoch 0:  75%|███████▍  | 4205/5616 [53:30<17:44,  1.33it/s, gpt_loss=0.0031, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4205/5616 [53:31<17:44,  1.33it/s, gpt_loss=0.0574, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4206/5616 [53:31<17:21,  1.35it/s, gpt_loss=0.0574, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4206/5616 [53:31<17:21,  1.35it/s, gpt_loss=0.0244, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4207/5616 [53:31<17:05,  1.37it/s, gpt_loss=0.0244, lr=1.21e-6]
+
Train step of epoch 0:  75%|███████▍  | 4207/5616 [53:32<17:05,  1.37it/s, gpt_loss=0.0124, lr=1.2e-6] 
+
Train step of epoch 0:  75%|███████▍  | 4208/5616 [53:32<17:28,  1.34it/s, gpt_loss=0.0124, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▍  | 4208/5616 [53:33<17:28,  1.34it/s, gpt_loss=0.0172, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▍  | 4209/5616 [53:33<16:42,  1.40it/s, gpt_loss=0.0172, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▍  | 4209/5616 [53:34<16:42,  1.40it/s, gpt_loss=0.00198, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▍  | 4210/5616 [53:34<16:37,  1.41it/s, gpt_loss=0.00198, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▍  | 4210/5616 [53:34<16:37,  1.41it/s, gpt_loss=0.0384, lr=1.2e-6] 
+
Train step of epoch 0:  75%|███████▍  | 4211/5616 [53:34<16:46,  1.40it/s, gpt_loss=0.0384, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▍  | 4211/5616 [53:35<16:46,  1.40it/s, gpt_loss=0.0879, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4212/5616 [53:35<17:26,  1.34it/s, gpt_loss=0.0879, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4212/5616 [53:36<17:26,  1.34it/s, gpt_loss=0.0142, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4213/5616 [53:36<17:05,  1.37it/s, gpt_loss=0.0142, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4213/5616 [53:36<17:05,  1.37it/s, gpt_loss=0.012, lr=1.2e-6] 
+
Train step of epoch 0:  75%|███████▌  | 4214/5616 [53:36<17:05,  1.37it/s, gpt_loss=0.012, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4214/5616 [53:38<17:05,  1.37it/s, gpt_loss=0.0294, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4215/5616 [53:38<19:36,  1.19it/s, gpt_loss=0.0294, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4215/5616 [53:38<19:36,  1.19it/s, gpt_loss=0.0195, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4216/5616 [53:38<19:09,  1.22it/s, gpt_loss=0.0195, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4216/5616 [53:39<19:09,  1.22it/s, gpt_loss=0.00883, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4217/5616 [53:39<18:14,  1.28it/s, gpt_loss=0.00883, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4217/5616 [53:40<18:14,  1.28it/s, gpt_loss=0.12, lr=1.2e-6]   
+
Train step of epoch 0:  75%|███████▌  | 4218/5616 [53:40<17:26,  1.34it/s, gpt_loss=0.12, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4218/5616 [53:41<17:26,  1.34it/s, gpt_loss=0.0423, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4219/5616 [53:41<17:52,  1.30it/s, gpt_loss=0.0423, lr=1.2e-6]
+
Train step of epoch 0:  75%|███████▌  | 4219/5616 [53:41<17:52,  1.30it/s, gpt_loss=0.0531, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4220/5616 [53:41<18:17,  1.27it/s, gpt_loss=0.0531, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4220/5616 [53:42<18:17,  1.27it/s, gpt_loss=0.0121, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4221/5616 [53:42<17:59,  1.29it/s, gpt_loss=0.0121, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4221/5616 [53:43<17:59,  1.29it/s, gpt_loss=0.0834, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4222/5616 [53:43<17:35,  1.32it/s, gpt_loss=0.0834, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4222/5616 [53:43<17:35,  1.32it/s, gpt_loss=0.00384, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4223/5616 [53:43<16:52,  1.38it/s, gpt_loss=0.00384, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4223/5616 [53:44<16:52,  1.38it/s, gpt_loss=0.0676, lr=1.19e-6] 
+
Train step of epoch 0:  75%|███████▌  | 4224/5616 [53:44<17:51,  1.30it/s, gpt_loss=0.0676, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4224/5616 [53:45<17:51,  1.30it/s, gpt_loss=0.00322, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4225/5616 [53:45<16:57,  1.37it/s, gpt_loss=0.00322, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4225/5616 [53:46<16:57,  1.37it/s, gpt_loss=0.0476, lr=1.19e-6] 
+
Train step of epoch 0:  75%|███████▌  | 4226/5616 [53:46<16:54,  1.37it/s, gpt_loss=0.0476, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4226/5616 [53:47<16:54,  1.37it/s, gpt_loss=0.0215, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4227/5616 [53:47<18:13,  1.27it/s, gpt_loss=0.0215, lr=1.19e-6]
+
Train step of epoch 0:  75%|███████▌  | 4227/5616 [53:47<18:13,  1.27it/s, gpt_loss=0.0179, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4228/5616 [53:47<18:10,  1.27it/s, gpt_loss=0.0179, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4228/5616 [53:48<18:10,  1.27it/s, gpt_loss=0.0133, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4229/5616 [53:48<19:01,  1.21it/s, gpt_loss=0.0133, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4229/5616 [53:49<19:01,  1.21it/s, gpt_loss=0.0312, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4230/5616 [53:49<18:08,  1.27it/s, gpt_loss=0.0312, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4230/5616 [53:50<18:08,  1.27it/s, gpt_loss=0.0155, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4231/5616 [53:50<18:56,  1.22it/s, gpt_loss=0.0155, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4231/5616 [53:51<18:56,  1.22it/s, gpt_loss=0.00311, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4232/5616 [53:51<18:26,  1.25it/s, gpt_loss=0.00311, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4232/5616 [53:51<18:26,  1.25it/s, gpt_loss=0.163, lr=1.18e-6]  
+
Train step of epoch 0:  75%|███████▌  | 4233/5616 [53:51<17:54,  1.29it/s, gpt_loss=0.163, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4233/5616 [53:52<17:54,  1.29it/s, gpt_loss=0.0501, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4234/5616 [53:52<18:05,  1.27it/s, gpt_loss=0.0501, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4234/5616 [53:53<18:05,  1.27it/s, gpt_loss=0.019, lr=1.18e-6] 
+
Train step of epoch 0:  75%|███████▌  | 4235/5616 [53:53<17:46,  1.30it/s, gpt_loss=0.019, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4235/5616 [53:54<17:46,  1.30it/s, gpt_loss=0.00543, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4236/5616 [53:54<18:03,  1.27it/s, gpt_loss=0.00543, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4236/5616 [53:55<18:03,  1.27it/s, gpt_loss=0.039, lr=1.18e-6]  
+
Train step of epoch 0:  75%|███████▌  | 4237/5616 [53:55<18:04,  1.27it/s, gpt_loss=0.039, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4237/5616 [53:55<18:04,  1.27it/s, gpt_loss=0.118, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4238/5616 [53:55<17:27,  1.32it/s, gpt_loss=0.118, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4238/5616 [53:56<17:27,  1.32it/s, gpt_loss=0.111, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4239/5616 [53:56<17:20,  1.32it/s, gpt_loss=0.111, lr=1.18e-6]
+
Train step of epoch 0:  75%|███████▌  | 4239/5616 [53:57<17:20,  1.32it/s, gpt_loss=0.00324, lr=1.17e-6]
+
Train step of epoch 0:  75%|███████▌  | 4240/5616 [53:57<17:56,  1.28it/s, gpt_loss=0.00324, lr=1.17e-6]
+
Train step of epoch 0:  75%|███████▌  | 4240/5616 [53:58<17:56,  1.28it/s, gpt_loss=0.0508, lr=1.17e-6] 
+
Train step of epoch 0:  76%|███████▌  | 4241/5616 [53:58<18:28,  1.24it/s, gpt_loss=0.0508, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4241/5616 [53:58<18:28,  1.24it/s, gpt_loss=0.0473, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████��  | 4242/5616 [53:58<17:38,  1.30it/s, gpt_loss=0.0473, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4242/5616 [53:59<17:38,  1.30it/s, gpt_loss=0.0485, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4243/5616 [53:59<17:20,  1.32it/s, gpt_loss=0.0485, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4243/5616 [54:00<17:20,  1.32it/s, gpt_loss=0.0291, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4244/5616 [54:00<17:06,  1.34it/s, gpt_loss=0.0291, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4244/5616 [54:01<17:06,  1.34it/s, gpt_loss=0.126, lr=1.17e-6] 
+
Train step of epoch 0:  76%|███████▌  | 4245/5616 [54:01<17:05,  1.34it/s, gpt_loss=0.126, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4245/5616 [54:01<17:05,  1.34it/s, gpt_loss=0.0629, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4246/5616 [54:01<17:50,  1.28it/s, gpt_loss=0.0629, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4246/5616 [54:02<17:50,  1.28it/s, gpt_loss=0.00102, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4247/5616 [54:02<18:33,  1.23it/s, gpt_loss=0.00102, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4247/5616 [54:03<18:33,  1.23it/s, gpt_loss=0.00283, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4248/5616 [54:03<18:27,  1.24it/s, gpt_loss=0.00283, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4248/5616 [54:04<18:27,  1.24it/s, gpt_loss=0.00868, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4249/5616 [54:04<17:54,  1.27it/s, gpt_loss=0.00868, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4249/5616 [54:05<17:54,  1.27it/s, gpt_loss=0.0206, lr=1.17e-6] 
+
Train step of epoch 0:  76%|███████▌  | 4250/5616 [54:05<17:27,  1.30it/s, gpt_loss=0.0206, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4250/5616 [54:05<17:27,  1.30it/s, gpt_loss=0.0121, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4251/5616 [54:05<17:06,  1.33it/s, gpt_loss=0.0121, lr=1.17e-6]
+
Train step of epoch 0:  76%|███████▌  | 4251/5616 [54:06<17:06,  1.33it/s, gpt_loss=0.0222, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4252/5616 [54:06<17:19,  1.31it/s, gpt_loss=0.0222, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4252/5616 [54:07<17:19,  1.31it/s, gpt_loss=0.0177, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4253/5616 [54:07<16:57,  1.34it/s, gpt_loss=0.0177, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4253/5616 [54:07<16:57,  1.34it/s, gpt_loss=0.00311, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4254/5616 [54:07<16:34,  1.37it/s, gpt_loss=0.00311, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4254/5616 [54:08<16:34,  1.37it/s, gpt_loss=0.00896, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4255/5616 [54:08<16:27,  1.38it/s, gpt_loss=0.00896, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4255/5616 [54:09<16:27,  1.38it/s, gpt_loss=0.0349, lr=1.16e-6] 
+
Train step of epoch 0:  76%|███████▌  | 4256/5616 [54:09<16:35,  1.37it/s, gpt_loss=0.0349, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4256/5616 [54:10<16:35,  1.37it/s, gpt_loss=0.00269, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4257/5616 [54:10<16:48,  1.35it/s, gpt_loss=0.00269, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4257/5616 [54:10<16:48,  1.35it/s, gpt_loss=0.0496, lr=1.16e-6] 
+
Train step of epoch 0:  76%|███████▌  | 4258/5616 [54:10<16:01,  1.41it/s, gpt_loss=0.0496, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4258/5616 [54:11<16:01,  1.41it/s, gpt_loss=0.0203, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4259/5616 [54:11<16:34,  1.36it/s, gpt_loss=0.0203, lr=1.16e-6]
+
Train step of epoch 0:  76%|███████▌  | 4259/5616 [54:12<16:34,  1.36it/s, gpt_loss=0.0144, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4260/5616 [54:12<17:18,  1.31it/s, gpt_loss=0.0144, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4260/5616 [54:13<17:18,  1.31it/s, gpt_loss=0.0249, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4261/5616 [54:13<17:15,  1.31it/s, gpt_loss=0.0249, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4261/5616 [54:14<17:15,  1.31it/s, gpt_loss=0.0588, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4262/5616 [54:14<17:26,  1.29it/s, gpt_loss=0.0588, lr=1.15e-6]
+
Train step of epoch 0:  76%|█���█████▌  | 4262/5616 [54:14<17:26,  1.29it/s, gpt_loss=0.0437, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4263/5616 [54:14<17:02,  1.32it/s, gpt_loss=0.0437, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4263/5616 [54:15<17:02,  1.32it/s, gpt_loss=0.0203, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4264/5616 [54:15<16:57,  1.33it/s, gpt_loss=0.0203, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4264/5616 [54:16<16:57,  1.33it/s, gpt_loss=0.00446, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4265/5616 [54:16<17:31,  1.28it/s, gpt_loss=0.00446, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4265/5616 [54:17<17:31,  1.28it/s, gpt_loss=0.0314, lr=1.15e-6] 
+
Train step of epoch 0:  76%|███████▌  | 4266/5616 [54:17<17:29,  1.29it/s, gpt_loss=0.0314, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4266/5616 [54:17<17:29,  1.29it/s, gpt_loss=0.0378, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4267/5616 [54:17<16:59,  1.32it/s, gpt_loss=0.0378, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4267/5616 [54:18<16:59,  1.32it/s, gpt_loss=0.00237, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4268/5616 [54:18<17:27,  1.29it/s, gpt_loss=0.00237, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4268/5616 [54:19<17:27,  1.29it/s, gpt_loss=0.00641, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4269/5616 [54:19<16:57,  1.32it/s, gpt_loss=0.00641, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4269/5616 [54:20<16:57,  1.32it/s, gpt_loss=0.0319, lr=1.15e-6] 
+
Train step of epoch 0:  76%|███████▌  | 4270/5616 [54:20<18:54,  1.19it/s, gpt_loss=0.0319, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4270/5616 [54:21<18:54,  1.19it/s, gpt_loss=0.0124, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4271/5616 [54:21<18:49,  1.19it/s, gpt_loss=0.0124, lr=1.15e-6]
+
Train step of epoch 0:  76%|███████▌  | 4271/5616 [54:21<18:49,  1.19it/s, gpt_loss=0.0383, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4272/5616 [54:21<18:19,  1.22it/s, gpt_loss=0.0383, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4272/5616 [54:22<18:19,  1.22it/s, gpt_loss=0.00947, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4273/5616 [54:22<17:47,  1.26it/s, gpt_loss=0.00947, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4273/5616 [54:23<17:47,  1.26it/s, gpt_loss=0.044, lr=1.14e-6]  
+
Train step of epoch 0:  76%|███████▌  | 4274/5616 [54:23<17:08,  1.31it/s, gpt_loss=0.044, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4274/5616 [54:24<17:08,  1.31it/s, gpt_loss=0.00596, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4275/5616 [54:24<18:38,  1.20it/s, gpt_loss=0.00596, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4275/5616 [54:25<18:38,  1.20it/s, gpt_loss=0.0111, lr=1.14e-6] 
+
Train step of epoch 0:  76%|███████▌  | 4276/5616 [54:25<18:06,  1.23it/s, gpt_loss=0.0111, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4276/5616 [54:25<18:06,  1.23it/s, gpt_loss=0.00222, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4277/5616 [54:25<18:04,  1.23it/s, gpt_loss=0.00222, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4277/5616 [54:26<18:04,  1.23it/s, gpt_loss=0.0067, lr=1.14e-6] 
+
Train step of epoch 0:  76%|███████▌  | 4278/5616 [54:26<17:13,  1.30it/s, gpt_loss=0.0067, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4278/5616 [54:27<17:13,  1.30it/s, gpt_loss=0.0215, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4279/5616 [54:27<16:44,  1.33it/s, gpt_loss=0.0215, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4279/5616 [54:28<16:44,  1.33it/s, gpt_loss=0.0621, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4280/5616 [54:28<16:44,  1.33it/s, gpt_loss=0.0621, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4280/5616 [54:28<16:44,  1.33it/s, gpt_loss=0.0213, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4281/5616 [54:28<17:18,  1.29it/s, gpt_loss=0.0213, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4281/5616 [54:29<17:18,  1.29it/s, gpt_loss=0.00191, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4282/5616 [54:29<16:34,  1.34it/s, gpt_loss=0.00191, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▌  | 4282/5616 [54:30<16:34,  1.34it/s, gpt_loss=0.00161, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▋  | 4283/5616 [54:30<16:09,  1.38it/s, gpt_loss=0.00161, lr=1.14e-6]
+
Train step of epoch 0:  76%|███████▋  | 4283/5616 [54:31<16:09,  1.38it/s, gpt_loss=0.00381, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4284/5616 [54:31<16:44,  1.33it/s, gpt_loss=0.00381, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4284/5616 [54:31<16:44,  1.33it/s, gpt_loss=0.00959, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4285/5616 [54:31<16:07,  1.38it/s, gpt_loss=0.00959, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4285/5616 [54:32<16:07,  1.38it/s, gpt_loss=0.0917, lr=1.13e-6] 
+
Train step of epoch 0:  76%|███████▋  | 4286/5616 [54:32<16:21,  1.35it/s, gpt_loss=0.0917, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4286/5616 [54:33<16:21,  1.35it/s, gpt_loss=0.036, lr=1.13e-6] 
+
Train step of epoch 0:  76%|███████▋  | 4287/5616 [54:33<16:07,  1.37it/s, gpt_loss=0.036, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4287/5616 [54:34<16:07,  1.37it/s, gpt_loss=0.00483, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4288/5616 [54:34<17:06,  1.29it/s, gpt_loss=0.00483, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4288/5616 [54:34<17:06,  1.29it/s, gpt_loss=0.0609, lr=1.13e-6] 
+
Train step of epoch 0:  76%|███████▋  | 4289/5616 [54:34<16:40,  1.33it/s, gpt_loss=0.0609, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4289/5616 [54:35<16:40,  1.33it/s, gpt_loss=0.0145, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4290/5616 [54:35<16:44,  1.32it/s, gpt_loss=0.0145, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4290/5616 [54:36<16:44,  1.32it/s, gpt_loss=0.039, lr=1.13e-6] 
+
Train step of epoch 0:  76%|███████▋  | 4291/5616 [54:36<16:04,  1.37it/s, gpt_loss=0.039, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4291/5616 [54:37<16:04,  1.37it/s, gpt_loss=0.0287, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4292/5616 [54:37<16:10,  1.36it/s, gpt_loss=0.0287, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4292/5616 [54:37<16:10,  1.36it/s, gpt_loss=0.0107, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4293/5616 [54:37<16:34,  1.33it/s, gpt_loss=0.0107, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4293/5616 [54:38<16:34,  1.33it/s, gpt_loss=0.00573, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4294/5616 [54:38<16:58,  1.30it/s, gpt_loss=0.00573, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4294/5616 [54:39<16:58,  1.30it/s, gpt_loss=0.0247, lr=1.13e-6] 
+
Train step of epoch 0:  76%|███████▋  | 4295/5616 [54:39<17:07,  1.29it/s, gpt_loss=0.0247, lr=1.13e-6]
+
Train step of epoch 0:  76%|███████▋  | 4295/5616 [54:40<17:07,  1.29it/s, gpt_loss=0.0344, lr=1.12e-6]
+
Train step of epoch 0:  76%|███████▋  | 4296/5616 [54:40<17:01,  1.29it/s, gpt_loss=0.0344, lr=1.12e-6]
+
Train step of epoch 0:  76%|███████▋  | 4296/5616 [54:40<17:01,  1.29it/s, gpt_loss=0.0031, lr=1.12e-6]
+
Train step of epoch 0:  77%|███████▋  | 4297/5616 [54:40<16:29,  1.33it/s, gpt_loss=0.0031, lr=1.12e-6]
+
Train step of epoch 0:  77%|███████▋  | 4297/5616 [54:41<16:29,  1.33it/s, gpt_loss=0.0106, lr=1.12e-6]
+
Train step of epoch 0:  77%|███████▋  | 4298/5616 [54:41<16:19,  1.35it/s, gpt_loss=0.0106, lr=1.12e-6]
+
Train step of epoch 0:  77%|███████▋  | 4298/5616 [54:42<16:19,  1.35it/s, gpt_loss=0.00398, lr=1.12e-6]
+
Train step of epoch 0:  77%|███████▋  | 4299/5616 [54:42<15:52,  1.38it/s, gpt_loss=0.00398, lr=1.12e-6]
+
Train step of epoch 0:  77%|███████▋  | 4299/5616 [54:43<15:52,  1.38it/s, gpt_loss=0.0186, lr=1.12e-6] 
+
Train step of epoch 0:  77%|███████▋  | 4300/5616 [54:43<16:31,  1.33it/s, gpt_loss=0.0186, lr=1.12e-6]
+
Train step of epoch 0:  77%|███████▋  | 4300/5616 [54:43<16:31,  1.33it/s, gpt_loss=0.042, lr=1.12e-6] 
+
Train step of epoch 0:  77%|███████▋  | 4301/5616 [54:43<15:48,  1.39it/s, gpt_loss=0.042, lr=1.12e-6]
+
Train step of epoch 0:  77%|███████▋  | 4301/5616 [54:44<15:48,  1.39it/s, gpt_loss=0.147, lr=1.12e-6]
+
Train step of epoch 0:  77%|███████▋  | 4302/5616 [54:44<15:43,  1.39it/s, gpt_loss=0.147, lr=1.12e-6]
+
Train step of epoch 0:  77%|███████▋  | 4302/5616 [54:45<15:43,  1.39it/s, gpt_loss=0.00201, lr=1.12e-6]
+
Train step of epoch 0:  77%|███████▋  | 4303/5616 [54:45<16:02,  1.36it/s, gpt_loss=0.00201, lr=1.12e-6]
+
Train step of epoch 0:  77%|███████▋  | 4303/5616 [54:45<16:02,  1.36it/s, gpt_loss=0.00714, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4304/5616 [54:45<16:08,  1.35it/s, gpt_loss=0.00714, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4304/5616 [54:46<16:08,  1.35it/s, gpt_loss=0.0529, lr=1.11e-6] 
+
Train step of epoch 0:  77%|███████▋  | 4305/5616 [54:46<16:42,  1.31it/s, gpt_loss=0.0529, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4305/5616 [54:47<16:42,  1.31it/s, gpt_loss=0.00259, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4306/5616 [54:47<16:16,  1.34it/s, gpt_loss=0.00259, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4306/5616 [54:48<16:16,  1.34it/s, gpt_loss=0.0284, lr=1.11e-6] 
+
Train step of epoch 0:  77%|███████▋  | 4307/5616 [54:48<15:49,  1.38it/s, gpt_loss=0.0284, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4307/5616 [54:48<15:49,  1.38it/s, gpt_loss=0.05, lr=1.11e-6]  
+
Train step of epoch 0:  77%|███████▋  | 4308/5616 [54:48<16:09,  1.35it/s, gpt_loss=0.05, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4308/5616 [54:49<16:09,  1.35it/s, gpt_loss=0.0233, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4309/5616 [54:49<16:03,  1.36it/s, gpt_loss=0.0233, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4309/5616 [54:50<16:03,  1.36it/s, gpt_loss=0.0227, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4310/5616 [54:50<15:27,  1.41it/s, gpt_loss=0.0227, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4310/5616 [54:51<15:27,  1.41it/s, gpt_loss=0.0945, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4311/5616 [54:51<15:11,  1.43it/s, gpt_loss=0.0945, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4311/5616 [54:51<15:11,  1.43it/s, gpt_loss=0.0425, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4312/5616 [54:51<16:16,  1.34it/s, gpt_loss=0.0425, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4312/5616 [54:52<16:16,  1.34it/s, gpt_loss=0.0142, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4313/5616 [54:52<15:43,  1.38it/s, gpt_loss=0.0142, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4313/5616 [54:53<15:43,  1.38it/s, gpt_loss=0.00309, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4314/5616 [54:53<15:12,  1.43it/s, gpt_loss=0.00309, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4314/5616 [54:53<15:12,  1.43it/s, gpt_loss=0.0692, lr=1.11e-6] 
+
Train step of epoch 0:  77%|███████▋  | 4315/5616 [54:53<14:56,  1.45it/s, gpt_loss=0.0692, lr=1.11e-6]
+
Train step of epoch 0:  77%|███████▋  | 4315/5616 [54:54<14:56,  1.45it/s, gpt_loss=0.0628, lr=1.1e-6] 
+
Train step of epoch 0:  77%|███████▋  | 4316/5616 [54:54<15:24,  1.41it/s, gpt_loss=0.0628, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4316/5616 [54:55<15:24,  1.41it/s, gpt_loss=0.0146, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4317/5616 [54:55<15:31,  1.39it/s, gpt_loss=0.0146, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4317/5616 [54:56<15:31,  1.39it/s, gpt_loss=0.0055, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4318/5616 [54:56<16:32,  1.31it/s, gpt_loss=0.0055, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4318/5616 [54:56<16:32,  1.31it/s, gpt_loss=0.0237, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4319/5616 [54:56<16:25,  1.32it/s, gpt_loss=0.0237, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4319/5616 [54:57<16:25,  1.32it/s, gpt_loss=0.0112, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4320/5616 [54:57<16:23,  1.32it/s, gpt_loss=0.0112, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4320/5616 [54:58<16:23,  1.32it/s, gpt_loss=0.061, lr=1.1e-6] 
+
Train step of epoch 0:  77%|███████▋  | 4321/5616 [54:58<15:32,  1.39it/s, gpt_loss=0.061, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4321/5616 [54:59<15:32,  1.39it/s, gpt_loss=0.0671, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4322/5616 [54:59<16:39,  1.29it/s, gpt_loss=0.0671, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4322/5616 [54:59<16:39,  1.29it/s, gpt_loss=0.00673, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4323/5616 [54:59<15:58,  1.35it/s, gpt_loss=0.00673, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4323/5616 [55:00<15:58,  1.35it/s, gpt_loss=0.029, lr=1.1e-6]  
+
Train step of epoch 0:  77%|███████▋  | 4324/5616 [55:00<17:44,  1.21it/s, gpt_loss=0.029, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4324/5616 [55:01<17:44,  1.21it/s, gpt_loss=0.0134, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4325/5616 [55:01<16:54,  1.27it/s, gpt_loss=0.0134, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4325/5616 [55:02<16:54,  1.27it/s, gpt_loss=0.0264, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4326/5616 [55:02<16:23,  1.31it/s, gpt_loss=0.0264, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4326/5616 [55:03<16:23,  1.31it/s, gpt_loss=0.011, lr=1.1e-6] 
+
Train step of epoch 0:  77%|███████▋  | 4327/5616 [55:03<16:25,  1.31it/s, gpt_loss=0.011, lr=1.1e-6]
+
Train step of epoch 0:  77%|███████▋  | 4327/5616 [55:03<16:25,  1.31it/s, gpt_loss=0.0145, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4328/5616 [55:03<16:38,  1.29it/s, gpt_loss=0.0145, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4328/5616 [55:04<16:38,  1.29it/s, gpt_loss=0.0089, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4329/5616 [55:04<16:18,  1.31it/s, gpt_loss=0.0089, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4329/5616 [55:05<16:18,  1.31it/s, gpt_loss=0.0155, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4330/5616 [55:05<16:08,  1.33it/s, gpt_loss=0.0155, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4330/5616 [55:06<16:08,  1.33it/s, gpt_loss=0.0888, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4331/5616 [55:06<16:35,  1.29it/s, gpt_loss=0.0888, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4331/5616 [55:07<16:35,  1.29it/s, gpt_loss=0.0172, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4332/5616 [55:07<16:51,  1.27it/s, gpt_loss=0.0172, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4332/5616 [55:07<16:51,  1.27it/s, gpt_loss=0.0111, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4333/5616 [55:07<16:12,  1.32it/s, gpt_loss=0.0111, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4333/5616 [55:08<16:12,  1.32it/s, gpt_loss=0.0047, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4334/5616 [55:08<16:19,  1.31it/s, gpt_loss=0.0047, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4334/5616 [55:09<16:19,  1.31it/s, gpt_loss=0.0597, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4335/5616 [55:09<15:46,  1.35it/s, gpt_loss=0.0597, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4335/5616 [55:10<15:46,  1.35it/s, gpt_loss=0.033, lr=1.09e-6] 
+
Train step of epoch 0:  77%|███████▋  | 4336/5616 [55:10<16:19,  1.31it/s, gpt_loss=0.033, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4336/5616 [55:10<16:19,  1.31it/s, gpt_loss=0.0315, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4337/5616 [55:10<15:43,  1.36it/s, gpt_loss=0.0315, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4337/5616 [55:11<15:43,  1.36it/s, gpt_loss=0.0101, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4338/5616 [55:11<17:48,  1.20it/s, gpt_loss=0.0101, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4338/5616 [55:12<17:48,  1.20it/s, gpt_loss=0.015, lr=1.09e-6] 
+
Train step of epoch 0:  77%|███████▋  | 4339/5616 [55:12<16:44,  1.27it/s, gpt_loss=0.015, lr=1.09e-6]
+
Train step of epoch 0:  77%|███████▋  | 4339/5616 [55:13<16:44,  1.27it/s, gpt_loss=0.00305, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4340/5616 [55:13<16:41,  1.27it/s, gpt_loss=0.00305, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4340/5616 [55:14<16:41,  1.27it/s, gpt_loss=0.0344, lr=1.08e-6] 
+
Train step of epoch 0:  77%|███████▋  | 4341/5616 [55:14<16:52,  1.26it/s, gpt_loss=0.0344, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4341/5616 [55:14<16:52,  1.26it/s, gpt_loss=0.00801, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4342/5616 [55:14<17:04,  1.24it/s, gpt_loss=0.00801, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4342/5616 [55:15<17:04,  1.24it/s, gpt_loss=0.0361, lr=1.08e-6] 
+
Train step of epoch 0:  77%|███████▋  | 4343/5616 [55:15<17:36,  1.20it/s, gpt_loss=0.0361, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4343/5616 [55:16<17:36,  1.20it/s, gpt_loss=0.00285, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4344/5616 [55:16<17:52,  1.19it/s, gpt_loss=0.00285, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4344/5616 [55:17<17:52,  1.19it/s, gpt_loss=0.0215, lr=1.08e-6] 
+
Train step of epoch 0:  77%|███████▋  | 4345/5616 [55:17<16:56,  1.25it/s, gpt_loss=0.0215, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4345/5616 [55:17<16:56,  1.25it/s, gpt_loss=0.0264, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4346/5616 [55:17<15:52,  1.33it/s, gpt_loss=0.0264, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4346/5616 [55:19<15:52,  1.33it/s, gpt_loss=0.0144, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4347/5616 [55:19<17:52,  1.18it/s, gpt_loss=0.0144, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4347/5616 [55:19<17:52,  1.18it/s, gpt_loss=0.0233, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4348/5616 [55:19<17:19,  1.22it/s, gpt_loss=0.0233, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4348/5616 [55:20<17:19,  1.22it/s, gpt_loss=0.0145, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4349/5616 [55:20<16:59,  1.24it/s, gpt_loss=0.0145, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4349/5616 [55:21<16:59,  1.24it/s, gpt_loss=0.0614, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4350/5616 [55:21<16:22,  1.29it/s, gpt_loss=0.0614, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4350/5616 [55:22<16:22,  1.29it/s, gpt_loss=0.0113, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4351/5616 [55:22<16:39,  1.27it/s, gpt_loss=0.0113, lr=1.08e-6]
+
Train step of epoch 0:  77%|███████▋  | 4351/5616 [55:22<16:39,  1.27it/s, gpt_loss=0.00807, lr=1.07e-6]
+
Train step of epoch 0:  77%|███████▋  | 4352/5616 [55:22<16:36,  1.27it/s, gpt_loss=0.00807, lr=1.07e-6]
+
Train step of epoch 0:  77%|███████▋  | 4352/5616 [55:23<16:36,  1.27it/s, gpt_loss=0.158, lr=1.07e-6]  
+
Train step of epoch 0:  78%|███████▊  | 4353/5616 [55:23<16:09,  1.30it/s, gpt_loss=0.158, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4353/5616 [55:24<16:09,  1.30it/s, gpt_loss=0.0146, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4354/5616 [55:24<15:33,  1.35it/s, gpt_loss=0.0146, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4354/5616 [55:24<15:33,  1.35it/s, gpt_loss=0.00586, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4355/5616 [55:24<15:37,  1.34it/s, gpt_loss=0.00586, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4355/5616 [55:25<15:37,  1.34it/s, gpt_loss=0.00734, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4356/5616 [55:25<15:46,  1.33it/s, gpt_loss=0.00734, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4356/5616 [55:26<15:46,  1.33it/s, gpt_loss=0.0133, lr=1.07e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4357/5616 [55:26<16:44,  1.25it/s, gpt_loss=0.0133, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4357/5616 [55:27<16:44,  1.25it/s, gpt_loss=0.00158, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4358/5616 [55:27<16:15,  1.29it/s, gpt_loss=0.00158, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4358/5616 [55:28<16:15,  1.29it/s, gpt_loss=0.000656, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4359/5616 [55:28<16:15,  1.29it/s, gpt_loss=0.000656, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4359/5616 [55:28<16:15,  1.29it/s, gpt_loss=0.00641, lr=1.07e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4360/5616 [55:28<16:11,  1.29it/s, gpt_loss=0.00641, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4360/5616 [55:29<16:11,  1.29it/s, gpt_loss=0.0255, lr=1.07e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4361/5616 [55:29<16:26,  1.27it/s, gpt_loss=0.0255, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4361/5616 [55:30<16:26,  1.27it/s, gpt_loss=0.00943, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4362/5616 [55:30<16:17,  1.28it/s, gpt_loss=0.00943, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4362/5616 [55:31<16:17,  1.28it/s, gpt_loss=0.0411, lr=1.07e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4363/5616 [55:31<15:34,  1.34it/s, gpt_loss=0.0411, lr=1.07e-6]
+
Train step of epoch 0:  78%|███████▊  | 4363/5616 [55:31<15:34,  1.34it/s, gpt_loss=0.0248, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4364/5616 [55:31<15:38,  1.33it/s, gpt_loss=0.0248, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4364/5616 [55:32<15:38,  1.33it/s, gpt_loss=0.0221, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4365/5616 [55:32<15:39,  1.33it/s, gpt_loss=0.0221, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4365/5616 [55:33<15:39,  1.33it/s, gpt_loss=0.187, lr=1.06e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4366/5616 [55:33<15:38,  1.33it/s, gpt_loss=0.187, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4366/5616 [55:34<15:38,  1.33it/s, gpt_loss=0.0102, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4367/5616 [55:34<15:47,  1.32it/s, gpt_loss=0.0102, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4367/5616 [55:35<15:47,  1.32it/s, gpt_loss=0.0101, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4368/5616 [55:35<16:11,  1.28it/s, gpt_loss=0.0101, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4368/5616 [55:35<16:11,  1.28it/s, gpt_loss=0.0347, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4369/5616 [55:35<15:30,  1.34it/s, gpt_loss=0.0347, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4369/5616 [55:36<15:30,  1.34it/s, gpt_loss=0.00898, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4370/5616 [55:36<15:59,  1.30it/s, gpt_loss=0.00898, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4370/5616 [55:37<15:59,  1.30it/s, gpt_loss=0.00887, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4371/5616 [55:37<16:09,  1.28it/s, gpt_loss=0.00887, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4371/5616 [55:38<16:09,  1.28it/s, gpt_loss=0.0192, lr=1.06e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4372/5616 [55:38<16:17,  1.27it/s, gpt_loss=0.0192, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4372/5616 [55:39<16:17,  1.27it/s, gpt_loss=0.0433, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4373/5616 [55:39<17:14,  1.20it/s, gpt_loss=0.0433, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4373/5616 [55:39<17:14,  1.20it/s, gpt_loss=0.0181, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4374/5616 [55:39<16:35,  1.25it/s, gpt_loss=0.0181, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4374/5616 [55:40<16:35,  1.25it/s, gpt_loss=0.0339, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4375/5616 [55:40<15:32,  1.33it/s, gpt_loss=0.0339, lr=1.06e-6]
+
Train step of epoch 0:  78%|███████▊  | 4375/5616 [55:41<15:32,  1.33it/s, gpt_loss=0.00236, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4376/5616 [55:41<15:54,  1.30it/s, gpt_loss=0.00236, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4376/5616 [55:41<15:54,  1.30it/s, gpt_loss=0.0412, lr=1.05e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4377/5616 [55:41<15:25,  1.34it/s, gpt_loss=0.0412, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4377/5616 [55:42<15:25,  1.34it/s, gpt_loss=0.00729, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4378/5616 [55:42<15:07,  1.36it/s, gpt_loss=0.00729, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4378/5616 [55:43<15:07,  1.36it/s, gpt_loss=0.0931, lr=1.05e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4379/5616 [55:43<15:02,  1.37it/s, gpt_loss=0.0931, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4379/5616 [55:44<15:02,  1.37it/s, gpt_loss=0.0354, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4380/5616 [55:44<15:27,  1.33it/s, gpt_loss=0.0354, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4380/5616 [55:44<15:27,  1.33it/s, gpt_loss=0.0169, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4381/5616 [55:44<15:25,  1.33it/s, gpt_loss=0.0169, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4381/5616 [55:45<15:25,  1.33it/s, gpt_loss=0.114, lr=1.05e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4382/5616 [55:45<15:07,  1.36it/s, gpt_loss=0.114, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4382/5616 [55:46<15:07,  1.36it/s, gpt_loss=0.00321, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4383/5616 [55:46<16:56,  1.21it/s, gpt_loss=0.00321, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4383/5616 [55:47<16:56,  1.21it/s, gpt_loss=0.0531, lr=1.05e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4384/5616 [55:47<17:31,  1.17it/s, gpt_loss=0.0531, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4384/5616 [55:48<17:31,  1.17it/s, gpt_loss=0.00178, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4385/5616 [55:48<16:18,  1.26it/s, gpt_loss=0.00178, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4385/5616 [55:49<16:18,  1.26it/s, gpt_loss=0.0223, lr=1.05e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4386/5616 [55:49<16:05,  1.27it/s, gpt_loss=0.0223, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4386/5616 [55:49<16:05,  1.27it/s, gpt_loss=0.0133, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4387/5616 [55:49<15:12,  1.35it/s, gpt_loss=0.0133, lr=1.05e-6]
+
Train step of epoch 0:  78%|███████▊  | 4387/5616 [55:50<15:12,  1.35it/s, gpt_loss=0.000655, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4388/5616 [55:50<17:19,  1.18it/s, gpt_loss=0.000655, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4388/5616 [55:51<17:19,  1.18it/s, gpt_loss=0.0133, lr=1.04e-6]  
+
Train step of epoch 0:  78%|███████▊  | 4389/5616 [55:51<16:52,  1.21it/s, gpt_loss=0.0133, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4389/5616 [55:52<16:52,  1.21it/s, gpt_loss=0.0225, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4390/5616 [55:52<16:21,  1.25it/s, gpt_loss=0.0225, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4390/5616 [55:52<16:21,  1.25it/s, gpt_loss=0.0858, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4391/5616 [55:52<15:26,  1.32it/s, gpt_loss=0.0858, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4391/5616 [55:53<15:26,  1.32it/s, gpt_loss=0.0158, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4392/5616 [55:53<15:27,  1.32it/s, gpt_loss=0.0158, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4392/5616 [55:54<15:27,  1.32it/s, gpt_loss=0.00349, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4393/5616 [55:54<15:24,  1.32it/s, gpt_loss=0.00349, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4393/5616 [55:55<15:24,  1.32it/s, gpt_loss=0.0267, lr=1.04e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4394/5616 [55:55<15:47,  1.29it/s, gpt_loss=0.0267, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4394/5616 [55:55<15:47,  1.29it/s, gpt_loss=0.000407, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4395/5616 [55:55<15:26,  1.32it/s, gpt_loss=0.000407, lr=1.04e-6]
+
Train step of epoch 0:  78%|███████▊  | 4395/5616 [55:56<15:26,  1.32it/s, gpt_loss=0.00508, lr=1.03e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4396/5616 [55:56<15:26,  1.32it/s, gpt_loss=0.00508, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4396/5616 [55:57<15:26,  1.32it/s, gpt_loss=0.0251, lr=1.03e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4397/5616 [55:57<15:25,  1.32it/s, gpt_loss=0.0251, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4397/5616 [55:58<15:25,  1.32it/s, gpt_loss=0.101, lr=1.03e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4398/5616 [55:58<15:29,  1.31it/s, gpt_loss=0.101, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4398/5616 [55:58<15:29,  1.31it/s, gpt_loss=0.0394, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4399/5616 [55:58<14:56,  1.36it/s, gpt_loss=0.0394, lr=1.03e-6][2025-06-30 07:34:08,522] [INFO] [logging.py:128:log_dist] [Rank 0] step=1100, skipped=0, lr=[1.031604210223522e-06, 1.031604210223522e-06], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 07:34:08,522] [INFO] [timer.py:264:stop] epoch=0/micro_step=4400/global_step=1100, RunningAvgSamplesPerSec=42.445291418058055, CurrSamplesPerSec=39.716201617657994, MemAllocated=16.16GB, MaxMemAllocated=54.86GB
+
+
Train step of epoch 0:  78%|███████▊  | 4399/5616 [55:59<14:56,  1.36it/s, gpt_loss=0.0221, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4400/5616 [55:59<16:53,  1.20it/s, gpt_loss=0.0221, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4400/5616 [56:00<16:53,  1.20it/s, gpt_loss=0.00607, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4401/5616 [56:00<16:15,  1.25it/s, gpt_loss=0.00607, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4401/5616 [56:01<16:15,  1.25it/s, gpt_loss=0.0983, lr=1.03e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4402/5616 [56:01<16:10,  1.25it/s, gpt_loss=0.0983, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4402/5616 [56:02<16:10,  1.25it/s, gpt_loss=0.0406, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4403/5616 [56:02<15:25,  1.31it/s, gpt_loss=0.0406, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4403/5616 [56:02<15:25,  1.31it/s, gpt_loss=0.00213, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4404/5616 [56:02<15:18,  1.32it/s, gpt_loss=0.00213, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4404/5616 [56:03<15:18,  1.32it/s, gpt_loss=0.00755, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4405/5616 [56:03<15:57,  1.26it/s, gpt_loss=0.00755, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4405/5616 [56:04<15:57,  1.26it/s, gpt_loss=0.00877, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4406/5616 [56:04<15:48,  1.28it/s, gpt_loss=0.00877, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4406/5616 [56:05<15:48,  1.28it/s, gpt_loss=0.0139, lr=1.03e-6] 
+
Train step of epoch 0:  78%|███████▊  | 4407/5616 [56:05<16:16,  1.24it/s, gpt_loss=0.0139, lr=1.03e-6]
+
Train step of epoch 0:  78%|███████▊  | 4407/5616 [56:06<16:16,  1.24it/s, gpt_loss=0.0111, lr=1.02e-6]
+
Train step of epoch 0:  78%|███████▊  | 4408/5616 [56:06<15:56,  1.26it/s, gpt_loss=0.0111, lr=1.02e-6]
+
Train step of epoch 0:  78%|███████▊  | 4408/5616 [56:06<15:56,  1.26it/s, gpt_loss=0.00519, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4409/5616 [56:06<15:31,  1.30it/s, gpt_loss=0.00519, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4409/5616 [56:07<15:31,  1.30it/s, gpt_loss=0.0172, lr=1.02e-6] 
+
Train step of epoch 0:  79%|███████▊  | 4410/5616 [56:07<15:14,  1.32it/s, gpt_loss=0.0172, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4410/5616 [56:08<15:14,  1.32it/s, gpt_loss=0.00174, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4411/5616 [56:08<14:44,  1.36it/s, gpt_loss=0.00174, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4411/5616 [56:09<14:44,  1.36it/s, gpt_loss=0.113, lr=1.02e-6]  
+
Train step of epoch 0:  79%|███████▊  | 4412/5616 [56:09<15:23,  1.30it/s, gpt_loss=0.113, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4412/5616 [56:09<15:23,  1.30it/s, gpt_loss=0.00397, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4413/5616 [56:09<15:23,  1.30it/s, gpt_loss=0.00397, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4413/5616 [56:10<15:23,  1.30it/s, gpt_loss=0.00598, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4414/5616 [56:10<14:40,  1.36it/s, gpt_loss=0.00598, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4414/5616 [56:11<14:40,  1.36it/s, gpt_loss=0.0237, lr=1.02e-6] 
+
Train step of epoch 0:  79%|███████▊  | 4415/5616 [56:11<15:28,  1.29it/s, gpt_loss=0.0237, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4415/5616 [56:12<15:28,  1.29it/s, gpt_loss=0.0974, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4416/5616 [56:12<15:58,  1.25it/s, gpt_loss=0.0974, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4416/5616 [56:12<15:58,  1.25it/s, gpt_loss=0.0846, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4417/5616 [56:12<15:10,  1.32it/s, gpt_loss=0.0846, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4417/5616 [56:13<15:10,  1.32it/s, gpt_loss=0.00109, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4418/5616 [56:13<15:00,  1.33it/s, gpt_loss=0.00109, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4418/5616 [56:14<15:00,  1.33it/s, gpt_loss=0.0561, lr=1.02e-6] 
+
Train step of epoch 0:  79%|███████▊  | 4419/5616 [56:14<14:45,  1.35it/s, gpt_loss=0.0561, lr=1.02e-6]
+
Train step of epoch 0:  79%|███████▊  | 4419/5616 [56:15<14:45,  1.35it/s, gpt_loss=0.112, lr=1.01e-6] 
+
Train step of epoch 0:  79%|███████▊  | 4420/5616 [56:15<14:39,  1.36it/s, gpt_loss=0.112, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▊  | 4420/5616 [56:15<14:39,  1.36it/s, gpt_loss=0.0908, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▊  | 4421/5616 [56:15<14:08,  1.41it/s, gpt_loss=0.0908, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▊  | 4421/5616 [56:16<14:08,  1.41it/s, gpt_loss=0.00454, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▊  | 4422/5616 [56:16<13:52,  1.43it/s, gpt_loss=0.00454, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▊  | 4422/5616 [56:17<13:52,  1.43it/s, gpt_loss=0.000431, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4423/5616 [56:17<14:14,  1.40it/s, gpt_loss=0.000431, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4423/5616 [56:18<14:14,  1.40it/s, gpt_loss=0.0581, lr=1.01e-6]  
+
Train step of epoch 0:  79%|███████▉  | 4424/5616 [56:18<15:01,  1.32it/s, gpt_loss=0.0581, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4424/5616 [56:18<15:01,  1.32it/s, gpt_loss=0.174, lr=1.01e-6] 
+
Train step of epoch 0:  79%|███████▉  | 4425/5616 [56:18<14:25,  1.38it/s, gpt_loss=0.174, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4425/5616 [56:19<14:25,  1.38it/s, gpt_loss=0.201, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4426/5616 [56:19<14:15,  1.39it/s, gpt_loss=0.201, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4426/5616 [56:20<14:15,  1.39it/s, gpt_loss=0.00976, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4427/5616 [56:20<15:48,  1.25it/s, gpt_loss=0.00976, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4427/5616 [56:21<15:48,  1.25it/s, gpt_loss=0.0596, lr=1.01e-6] 
+
Train step of epoch 0:  79%|███████▉  | 4428/5616 [56:21<16:12,  1.22it/s, gpt_loss=0.0596, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4428/5616 [56:22<16:12,  1.22it/s, gpt_loss=0.0436, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4429/5616 [56:22<16:12,  1.22it/s, gpt_loss=0.0436, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4429/5616 [56:22<16:12,  1.22it/s, gpt_loss=0.201, lr=1.01e-6] 
+
Train step of epoch 0:  79%|███████▉  | 4430/5616 [56:22<15:54,  1.24it/s, gpt_loss=0.201, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4430/5616 [56:23<15:54,  1.24it/s, gpt_loss=0.0247, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4431/5616 [56:23<15:52,  1.24it/s, gpt_loss=0.0247, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4431/5616 [56:24<15:52,  1.24it/s, gpt_loss=0.00784, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4432/5616 [56:24<15:37,  1.26it/s, gpt_loss=0.00784, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4432/5616 [56:25<15:37,  1.26it/s, gpt_loss=0.0411, lr=1.01e-6] 
+
Train step of epoch 0:  79%|███████▉  | 4433/5616 [56:25<15:51,  1.24it/s, gpt_loss=0.0411, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4433/5616 [56:26<15:51,  1.24it/s, gpt_loss=0.0357, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4434/5616 [56:26<15:54,  1.24it/s, gpt_loss=0.0357, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4434/5616 [56:26<15:54,  1.24it/s, gpt_loss=0.0218, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4435/5616 [56:26<15:38,  1.26it/s, gpt_loss=0.0218, lr=1.01e-6]
+
Train step of epoch 0:  79%|███████▉  | 4435/5616 [56:27<15:38,  1.26it/s, gpt_loss=0.0138, lr=1e-6]   
+
Train step of epoch 0:  79%|███████▉  | 4436/5616 [56:27<15:49,  1.24it/s, gpt_loss=0.0138, lr=1e-6]
+
Train step of epoch 0:  79%|███████▉  | 4436/5616 [56:28<15:49,  1.24it/s, gpt_loss=0.00507, lr=1e-6]
+
Train step of epoch 0:  79%|███████▉  | 4437/5616 [56:28<15:37,  1.26it/s, gpt_loss=0.00507, lr=1e-6]
+
Train step of epoch 0:  79%|███████▉  | 4437/5616 [56:29<15:37,  1.26it/s, gpt_loss=0.00139, lr=1e-6]
+
Train step of epoch 0:  79%|███████▉  | 4438/5616 [56:29<14:59,  1.31it/s, gpt_loss=0.00139, lr=1e-6]
+
Train step of epoch 0:  79%|███████▉  | 4438/5616 [56:29<14:59,  1.31it/s, gpt_loss=0.00683, lr=1e-6]
+
Train step of epoch 0:  79%|███████▉  | 4439/5616 [56:29<14:36,  1.34it/s, gpt_loss=0.00683, lr=1e-6]
+
Train step of epoch 0:  79%|███████▉  | 4439/5616 [56:30<14:36,  1.34it/s, gpt_loss=0.0338, lr=9.99e-7]
+
Train step of epoch 0:  79%|███████▉  | 4440/5616 [56:30<14:49,  1.32it/s, gpt_loss=0.0338, lr=9.99e-7]
+
Train step of epoch 0:  79%|███████▉  | 4440/5616 [56:31<14:49,  1.32it/s, gpt_loss=0.0629, lr=9.99e-7]
+
Train step of epoch 0:  79%|███████▉  | 4441/5616 [56:31<14:13,  1.38it/s, gpt_loss=0.0629, lr=9.99e-7]
+
Train step of epoch 0:  79%|███████▉  | 4441/5616 [56:31<14:13,  1.38it/s, gpt_loss=0.0485, lr=9.99e-7]
+
Train step of epoch 0:  79%|███████▉  | 4442/5616 [56:31<13:54,  1.41it/s, gpt_loss=0.0485, lr=9.99e-7]
+
Train step of epoch 0:  79%|███████▉  | 4442/5616 [56:32<13:54,  1.41it/s, gpt_loss=0.0258, lr=9.99e-7]
+
Train step of epoch 0:  79%|███████▉  | 4443/5616 [56:32<14:31,  1.35it/s, gpt_loss=0.0258, lr=9.99e-7]
+
Train step of epoch 0:  79%|███████▉  | 4443/5616 [56:33<14:31,  1.35it/s, gpt_loss=0.0747, lr=9.95e-7]
+
Train step of epoch 0:  79%|███████▉  | 4444/5616 [56:33<16:08,  1.21it/s, gpt_loss=0.0747, lr=9.95e-7]
+
Train step of epoch 0:  79%|███████▉  | 4444/5616 [56:34<16:08,  1.21it/s, gpt_loss=0.0735, lr=9.95e-7]
+
Train step of epoch 0:  79%|███████▉  | 4445/5616 [56:34<16:02,  1.22it/s, gpt_loss=0.0735, lr=9.95e-7]
+
Train step of epoch 0:  79%|███████▉  | 4445/5616 [56:35<16:02,  1.22it/s, gpt_loss=0.00396, lr=9.95e-7]
+
Train step of epoch 0:  79%|███████▉  | 4446/5616 [56:35<15:16,  1.28it/s, gpt_loss=0.00396, lr=9.95e-7]
+
Train step of epoch 0:  79%|███████▉  | 4446/5616 [56:35<15:16,  1.28it/s, gpt_loss=0.0293, lr=9.95e-7] 
+
Train step of epoch 0:  79%|███████▉  | 4447/5616 [56:35<14:37,  1.33it/s, gpt_loss=0.0293, lr=9.95e-7]
+
Train step of epoch 0:  79%|███████▉  | 4447/5616 [56:36<14:37,  1.33it/s, gpt_loss=0.0414, lr=9.92e-7]
+
Train step of epoch 0:  79%|███████▉  | 4448/5616 [56:36<14:23,  1.35it/s, gpt_loss=0.0414, lr=9.92e-7]
+
Train step of epoch 0:  79%|███████▉  | 4448/5616 [56:37<14:23,  1.35it/s, gpt_loss=0.0147, lr=9.92e-7]
+
Train step of epoch 0:  79%|███████▉  | 4449/5616 [56:37<14:03,  1.38it/s, gpt_loss=0.0147, lr=9.92e-7]
+
Train step of epoch 0:  79%|███████▉  | 4449/5616 [56:38<14:03,  1.38it/s, gpt_loss=0.00218, lr=9.92e-7]
+
Train step of epoch 0:  79%|███████▉  | 4450/5616 [56:38<14:21,  1.35it/s, gpt_loss=0.00218, lr=9.92e-7]
+
Train step of epoch 0:  79%|███████▉  | 4450/5616 [56:38<14:21,  1.35it/s, gpt_loss=0.0271, lr=9.92e-7] 
+
Train step of epoch 0:  79%|███████▉  | 4451/5616 [56:38<14:52,  1.31it/s, gpt_loss=0.0271, lr=9.92e-7]
+
Train step of epoch 0:  79%|███████▉  | 4451/5616 [56:39<14:52,  1.31it/s, gpt_loss=0.0447, lr=9.89e-7]
+
Train step of epoch 0:  79%|███████▉  | 4452/5616 [56:39<14:45,  1.31it/s, gpt_loss=0.0447, lr=9.89e-7]
+
Train step of epoch 0:  79%|███████▉  | 4452/5616 [56:40<14:45,  1.31it/s, gpt_loss=0.034, lr=9.89e-7] 
+
Train step of epoch 0:  79%|███████▉  | 4453/5616 [56:40<14:15,  1.36it/s, gpt_loss=0.034, lr=9.89e-7]
+
Train step of epoch 0:  79%|███████▉  | 4453/5616 [56:41<14:15,  1.36it/s, gpt_loss=0.0117, lr=9.89e-7]
+
Train step of epoch 0:  79%|███████▉  | 4454/5616 [56:41<15:02,  1.29it/s, gpt_loss=0.0117, lr=9.89e-7]
+
Train step of epoch 0:  79%|███████▉  | 4454/5616 [56:42<15:02,  1.29it/s, gpt_loss=0.0178, lr=9.89e-7]
+
Train step of epoch 0:  79%|███████▉  | 4455/5616 [56:42<14:46,  1.31it/s, gpt_loss=0.0178, lr=9.89e-7]
+
Train step of epoch 0:  79%|███████▉  | 4455/5616 [56:42<14:46,  1.31it/s, gpt_loss=0.0584, lr=9.86e-7]
+
Train step of epoch 0:  79%|███████▉  | 4456/5616 [56:42<14:58,  1.29it/s, gpt_loss=0.0584, lr=9.86e-7]
+
Train step of epoch 0:  79%|███████▉  | 4456/5616 [56:43<14:58,  1.29it/s, gpt_loss=0.0946, lr=9.86e-7]
+
Train step of epoch 0:  79%|███████▉  | 4457/5616 [56:43<14:39,  1.32it/s, gpt_loss=0.0946, lr=9.86e-7]
+
Train step of epoch 0:  79%|███████▉  | 4457/5616 [56:44<14:39,  1.32it/s, gpt_loss=0.0503, lr=9.86e-7]
+
Train step of epoch 0:  79%|███████▉  | 4458/5616 [56:44<14:58,  1.29it/s, gpt_loss=0.0503, lr=9.86e-7]
+
Train step of epoch 0:  79%|███████▉  | 4458/5616 [56:45<14:58,  1.29it/s, gpt_loss=0.0148, lr=9.86e-7]
+
Train step of epoch 0:  79%|███████▉  | 4459/5616 [56:45<15:11,  1.27it/s, gpt_loss=0.0148, lr=9.86e-7]
+
Train step of epoch 0:  79%|███████▉  | 4459/5616 [56:46<15:11,  1.27it/s, gpt_loss=0.037, lr=9.82e-7] 
+
Train step of epoch 0:  79%|███████▉  | 4460/5616 [56:46<17:15,  1.12it/s, gpt_loss=0.037, lr=9.82e-7]
+
Train step of epoch 0:  79%|███████▉  | 4460/5616 [56:46<17:15,  1.12it/s, gpt_loss=0.0395, lr=9.82e-7]
+
Train step of epoch 0:  79%|███████▉  | 4461/5616 [56:46<15:55,  1.21it/s, gpt_loss=0.0395, lr=9.82e-7]
+
Train step of epoch 0:  79%|███████▉  | 4461/5616 [56:47<15:55,  1.21it/s, gpt_loss=0.11, lr=9.82e-7]  
+
Train step of epoch 0:  79%|███████▉  | 4462/5616 [56:47<15:09,  1.27it/s, gpt_loss=0.11, lr=9.82e-7]
+
Train step of epoch 0:  79%|███████▉  | 4462/5616 [56:48<15:09,  1.27it/s, gpt_loss=0.0646, lr=9.82e-7]
+
Train step of epoch 0:  79%|███████▉  | 4463/5616 [56:48<14:49,  1.30it/s, gpt_loss=0.0646, lr=9.82e-7]
+
Train step of epoch 0:  79%|███████▉  | 4463/5616 [56:49<14:49,  1.30it/s, gpt_loss=0.00404, lr=9.79e-7]
+
Train step of epoch 0:  79%|███████▉  | 4464/5616 [56:49<14:58,  1.28it/s, gpt_loss=0.00404, lr=9.79e-7]
+
Train step of epoch 0:  79%|███████▉  | 4464/5616 [56:49<14:58,  1.28it/s, gpt_loss=0.0124, lr=9.79e-7] 
+
Train step of epoch 0:  80%|███████▉  | 4465/5616 [56:49<14:46,  1.30it/s, gpt_loss=0.0124, lr=9.79e-7]
+
Train step of epoch 0:  80%|███████▉  | 4465/5616 [56:50<14:46,  1.30it/s, gpt_loss=0.0318, lr=9.79e-7]
+
Train step of epoch 0:  80%|███████▉  | 4466/5616 [56:50<13:58,  1.37it/s, gpt_loss=0.0318, lr=9.79e-7]
+
Train step of epoch 0:  80%|███████▉  | 4466/5616 [56:51<13:58,  1.37it/s, gpt_loss=0.0268, lr=9.79e-7]
+
Train step of epoch 0:  80%|███████▉  | 4467/5616 [56:51<14:22,  1.33it/s, gpt_loss=0.0268, lr=9.79e-7]
+
Train step of epoch 0:  80%|███████▉  | 4467/5616 [56:52<14:22,  1.33it/s, gpt_loss=0.0318, lr=9.76e-7]
+
Train step of epoch 0:  80%|███████▉  | 4468/5616 [56:52<15:23,  1.24it/s, gpt_loss=0.0318, lr=9.76e-7]
+
Train step of epoch 0:  80%|███████▉  | 4468/5616 [56:53<15:23,  1.24it/s, gpt_loss=0.0755, lr=9.76e-7]
+
Train step of epoch 0:  80%|███████▉  | 4469/5616 [56:53<16:19,  1.17it/s, gpt_loss=0.0755, lr=9.76e-7]
+
Train step of epoch 0:  80%|███████▉  | 4469/5616 [56:54<16:19,  1.17it/s, gpt_loss=0.0196, lr=9.76e-7]
+
Train step of epoch 0:  80%|███████▉  | 4470/5616 [56:54<15:36,  1.22it/s, gpt_loss=0.0196, lr=9.76e-7]
+
Train step of epoch 0:  80%|███████▉  | 4470/5616 [56:54<15:36,  1.22it/s, gpt_loss=0.0793, lr=9.76e-7]
+
Train step of epoch 0:  80%|███████▉  | 4471/5616 [56:54<14:54,  1.28it/s, gpt_loss=0.0793, lr=9.76e-7]
+
Train step of epoch 0:  80%|███████▉  | 4471/5616 [56:55<14:54,  1.28it/s, gpt_loss=0.0704, lr=9.73e-7]
+
Train step of epoch 0:  80%|███████▉  | 4472/5616 [56:55<15:09,  1.26it/s, gpt_loss=0.0704, lr=9.73e-7]
+
Train step of epoch 0:  80%|███████▉  | 4472/5616 [56:56<15:09,  1.26it/s, gpt_loss=0.0269, lr=9.73e-7]
+
Train step of epoch 0:  80%|███████▉  | 4473/5616 [56:56<15:02,  1.27it/s, gpt_loss=0.0269, lr=9.73e-7]
+
Train step of epoch 0:  80%|███████▉  | 4473/5616 [56:57<15:02,  1.27it/s, gpt_loss=0.0721, lr=9.73e-7]
+
Train step of epoch 0:  80%|███████▉  | 4474/5616 [56:57<14:57,  1.27it/s, gpt_loss=0.0721, lr=9.73e-7]
+
Train step of epoch 0:  80%|███████▉  | 4474/5616 [56:57<14:57,  1.27it/s, gpt_loss=0.0178, lr=9.73e-7]
+
Train step of epoch 0:  80%|███████▉  | 4475/5616 [56:57<14:36,  1.30it/s, gpt_loss=0.0178, lr=9.73e-7]
+
Train step of epoch 0:  80%|███████▉  | 4475/5616 [56:58<14:36,  1.30it/s, gpt_loss=0.00236, lr=9.7e-7]
+
Train step of epoch 0:  80%|███████▉  | 4476/5616 [56:58<14:45,  1.29it/s, gpt_loss=0.00236, lr=9.7e-7]
+
Train step of epoch 0:  80%|███████▉  | 4476/5616 [56:59<14:45,  1.29it/s, gpt_loss=0.034, lr=9.7e-7]  
+
Train step of epoch 0:  80%|███████▉  | 4477/5616 [56:59<14:13,  1.34it/s, gpt_loss=0.034, lr=9.7e-7]
+
Train step of epoch 0:  80%|███████▉  | 4477/5616 [56:59<14:13,  1.34it/s, gpt_loss=0.0242, lr=9.7e-7]
+
Train step of epoch 0:  80%|███████▉  | 4478/5616 [56:59<13:54,  1.36it/s, gpt_loss=0.0242, lr=9.7e-7]
+
Train step of epoch 0:  80%|███████▉  | 4478/5616 [57:00<13:54,  1.36it/s, gpt_loss=0.056, lr=9.7e-7] 
+
Train step of epoch 0:  80%|███████▉  | 4479/5616 [57:00<13:34,  1.40it/s, gpt_loss=0.056, lr=9.7e-7]
+
Train step of epoch 0:  80%|███████▉  | 4479/5616 [57:01<13:34,  1.40it/s, gpt_loss=0.0159, lr=9.66e-7]
+
Train step of epoch 0:  80%|███████▉  | 4480/5616 [57:01<13:43,  1.38it/s, gpt_loss=0.0159, lr=9.66e-7]
+
Train step of epoch 0:  80%|███████▉  | 4480/5616 [57:02<13:43,  1.38it/s, gpt_loss=0.04, lr=9.66e-7]  
+
Train step of epoch 0:  80%|███████▉  | 4481/5616 [57:02<13:29,  1.40it/s, gpt_loss=0.04, lr=9.66e-7]
+
Train step of epoch 0:  80%|███████▉  | 4481/5616 [57:02<13:29,  1.40it/s, gpt_loss=0.00971, lr=9.66e-7]
+
Train step of epoch 0:  80%|███████▉  | 4482/5616 [57:02<13:38,  1.38it/s, gpt_loss=0.00971, lr=9.66e-7]
+
Train step of epoch 0:  80%|███████▉  | 4482/5616 [57:03<13:38,  1.38it/s, gpt_loss=0.0266, lr=9.66e-7] 
+
Train step of epoch 0:  80%|███████▉  | 4483/5616 [57:03<14:20,  1.32it/s, gpt_loss=0.0266, lr=9.66e-7]
+
Train step of epoch 0:  80%|███████▉  | 4483/5616 [57:04<14:20,  1.32it/s, gpt_loss=0.048, lr=9.63e-7] 
+
Train step of epoch 0:  80%|███████▉  | 4484/5616 [57:04<15:32,  1.21it/s, gpt_loss=0.048, lr=9.63e-7]
+
Train step of epoch 0:  80%|███████▉  | 4484/5616 [57:05<15:32,  1.21it/s, gpt_loss=0.0134, lr=9.63e-7]
+
Train step of epoch 0:  80%|███████▉  | 4485/5616 [57:05<14:56,  1.26it/s, gpt_loss=0.0134, lr=9.63e-7]
+
Train step of epoch 0:  80%|███████▉  | 4485/5616 [57:06<14:56,  1.26it/s, gpt_loss=0.0197, lr=9.63e-7]
+
Train step of epoch 0:  80%|███████▉  | 4486/5616 [57:06<14:05,  1.34it/s, gpt_loss=0.0197, lr=9.63e-7]
+
Train step of epoch 0:  80%|███████▉  | 4486/5616 [57:06<14:05,  1.34it/s, gpt_loss=0.0898, lr=9.63e-7]
+
Train step of epoch 0:  80%|███████▉  | 4487/5616 [57:06<13:32,  1.39it/s, gpt_loss=0.0898, lr=9.63e-7]
+
Train step of epoch 0:  80%|███████▉  | 4487/5616 [57:07<13:32,  1.39it/s, gpt_loss=0.0226, lr=9.6e-7] 
+
Train step of epoch 0:  80%|███████▉  | 4488/5616 [57:07<13:43,  1.37it/s, gpt_loss=0.0226, lr=9.6e-7]
+
Train step of epoch 0:  80%|███████▉  | 4488/5616 [57:08<13:43,  1.37it/s, gpt_loss=0.017, lr=9.6e-7] 
+
Train step of epoch 0:  80%|███████▉  | 4489/5616 [57:08<13:28,  1.39it/s, gpt_loss=0.017, lr=9.6e-7]
+
Train step of epoch 0:  80%|███████▉  | 4489/5616 [57:08<13:28,  1.39it/s, gpt_loss=0.0217, lr=9.6e-7]
+
Train step of epoch 0:  80%|███████▉  | 4490/5616 [57:08<13:11,  1.42it/s, gpt_loss=0.0217, lr=9.6e-7]
+
Train step of epoch 0:  80%|███████▉  | 4490/5616 [57:09<13:11,  1.42it/s, gpt_loss=0.0286, lr=9.6e-7]
+
Train step of epoch 0:  80%|███████▉  | 4491/5616 [57:09<13:52,  1.35it/s, gpt_loss=0.0286, lr=9.6e-7]
+
Train step of epoch 0:  80%|███████▉  | 4491/5616 [57:10<13:52,  1.35it/s, gpt_loss=0.0399, lr=9.57e-7]
+
Train step of epoch 0:  80%|███████▉  | 4492/5616 [57:10<14:16,  1.31it/s, gpt_loss=0.0399, lr=9.57e-7]
+
Train step of epoch 0:  80%|███████▉  | 4492/5616 [57:11<14:16,  1.31it/s, gpt_loss=0.0127, lr=9.57e-7]
+
Train step of epoch 0:  80%|████████  | 4493/5616 [57:11<13:39,  1.37it/s, gpt_loss=0.0127, lr=9.57e-7]
+
Train step of epoch 0:  80%|████████  | 4493/5616 [57:11<13:39,  1.37it/s, gpt_loss=0.02, lr=9.57e-7]  
+
Train step of epoch 0:  80%|████████  | 4494/5616 [57:11<14:11,  1.32it/s, gpt_loss=0.02, lr=9.57e-7]
+
Train step of epoch 0:  80%|████████  | 4494/5616 [57:12<14:11,  1.32it/s, gpt_loss=0.0486, lr=9.57e-7]
+
Train step of epoch 0:  80%|████████  | 4495/5616 [57:12<14:15,  1.31it/s, gpt_loss=0.0486, lr=9.57e-7]
+
Train step of epoch 0:  80%|████████  | 4495/5616 [57:13<14:15,  1.31it/s, gpt_loss=0.0321, lr=9.54e-7]
+
Train step of epoch 0:  80%|████████  | 4496/5616 [57:13<16:25,  1.14it/s, gpt_loss=0.0321, lr=9.54e-7]
+
Train step of epoch 0:  80%|████████  | 4496/5616 [57:14<16:25,  1.14it/s, gpt_loss=0.0324, lr=9.54e-7]
+
Train step of epoch 0:  80%|████████  | 4497/5616 [57:14<16:21,  1.14it/s, gpt_loss=0.0324, lr=9.54e-7]
+
Train step of epoch 0:  80%|████████  | 4497/5616 [57:15<16:21,  1.14it/s, gpt_loss=0.0175, lr=9.54e-7]
+
Train step of epoch 0:  80%|████████  | 4498/5616 [57:15<15:38,  1.19it/s, gpt_loss=0.0175, lr=9.54e-7]
+
Train step of epoch 0:  80%|████████  | 4498/5616 [57:16<15:38,  1.19it/s, gpt_loss=0.0573, lr=9.54e-7]
+
Train step of epoch 0:  80%|████████  | 4499/5616 [57:16<14:54,  1.25it/s, gpt_loss=0.0573, lr=9.54e-7]
+
Train step of epoch 0:  80%|████████  | 4499/5616 [57:17<14:54,  1.25it/s, gpt_loss=0.0688, lr=9.51e-7]
+
Train step of epoch 0:  80%|████████  | 4500/5616 [57:17<15:01,  1.24it/s, gpt_loss=0.0688, lr=9.51e-7]
+
Train step of epoch 0:  80%|████████  | 4500/5616 [57:18<15:01,  1.24it/s, gpt_loss=0.00491, lr=9.51e-7]
+
Train step of epoch 0:  80%|████████  | 4501/5616 [57:18<16:56,  1.10it/s, gpt_loss=0.00491, lr=9.51e-7]
+
Train step of epoch 0:  80%|████████  | 4501/5616 [57:18<16:56,  1.10it/s, gpt_loss=0.00528, lr=9.51e-7]
+
Train step of epoch 0:  80%|████████  | 4502/5616 [57:18<15:43,  1.18it/s, gpt_loss=0.00528, lr=9.51e-7]
+
Train step of epoch 0:  80%|████████  | 4502/5616 [57:19<15:43,  1.18it/s, gpt_loss=0.00822, lr=9.51e-7]
+
Train step of epoch 0:  80%|████████  | 4503/5616 [57:19<14:54,  1.24it/s, gpt_loss=0.00822, lr=9.51e-7]
+
Train step of epoch 0:  80%|████████  | 4503/5616 [57:20<14:54,  1.24it/s, gpt_loss=0.0607, lr=9.48e-7] 
+
Train step of epoch 0:  80%|████████  | 4504/5616 [57:20<15:08,  1.22it/s, gpt_loss=0.0607, lr=9.48e-7]
+
Train step of epoch 0:  80%|████████  | 4504/5616 [57:21<15:08,  1.22it/s, gpt_loss=0.0646, lr=9.48e-7]
+
Train step of epoch 0:  80%|████████  | 4505/5616 [57:21<14:24,  1.28it/s, gpt_loss=0.0646, lr=9.48e-7]
+
Train step of epoch 0:  80%|████████  | 4505/5616 [57:21<14:24,  1.28it/s, gpt_loss=0.106, lr=9.48e-7] 
+
Train step of epoch 0:  80%|████████  | 4506/5616 [57:21<14:01,  1.32it/s, gpt_loss=0.106, lr=9.48e-7]
+
Train step of epoch 0:  80%|████████  | 4506/5616 [57:22<14:01,  1.32it/s, gpt_loss=0.0575, lr=9.48e-7]
+
Train step of epoch 0:  80%|████████  | 4507/5616 [57:22<13:35,  1.36it/s, gpt_loss=0.0575, lr=9.48e-7]
+
Train step of epoch 0:  80%|████████  | 4507/5616 [57:23<13:35,  1.36it/s, gpt_loss=0.0124, lr=9.44e-7]
+
Train step of epoch 0:  80%|████████  | 4508/5616 [57:23<14:55,  1.24it/s, gpt_loss=0.0124, lr=9.44e-7]
+
Train step of epoch 0:  80%|████████  | 4508/5616 [57:24<14:55,  1.24it/s, gpt_loss=0.0408, lr=9.44e-7]
+
Train step of epoch 0:  80%|████████  | 4509/5616 [57:24<14:20,  1.29it/s, gpt_loss=0.0408, lr=9.44e-7]
+
Train step of epoch 0:  80%|████████  | 4509/5616 [57:25<14:20,  1.29it/s, gpt_loss=0.0309, lr=9.44e-7]
+
Train step of epoch 0:  80%|████████  | 4510/5616 [57:25<15:49,  1.17it/s, gpt_loss=0.0309, lr=9.44e-7]
+
Train step of epoch 0:  80%|████████  | 4510/5616 [57:25<15:49,  1.17it/s, gpt_loss=0.0227, lr=9.44e-7]
+
Train step of epoch 0:  80%|████████  | 4511/5616 [57:25<14:52,  1.24it/s, gpt_loss=0.0227, lr=9.44e-7]
+
Train step of epoch 0:  80%|████████  | 4511/5616 [57:26<14:52,  1.24it/s, gpt_loss=0.0518, lr=9.41e-7]
+
Train step of epoch 0:  80%|████████  | 4512/5616 [57:26<15:52,  1.16it/s, gpt_loss=0.0518, lr=9.41e-7]
+
Train step of epoch 0:  80%|████████  | 4512/5616 [57:27<15:52,  1.16it/s, gpt_loss=0.013, lr=9.41e-7] 
+
Train step of epoch 0:  80%|████████  | 4513/5616 [57:27<14:52,  1.24it/s, gpt_loss=0.013, lr=9.41e-7]
+
Train step of epoch 0:  80%|████████  | 4513/5616 [57:28<14:52,  1.24it/s, gpt_loss=0.00636, lr=9.41e-7]
+
Train step of epoch 0:  80%|████████  | 4514/5616 [57:28<14:05,  1.30it/s, gpt_loss=0.00636, lr=9.41e-7]
+
Train step of epoch 0:  80%|████████  | 4514/5616 [57:29<14:05,  1.30it/s, gpt_loss=0.0703, lr=9.41e-7] 
+
Train step of epoch 0:  80%|████████  | 4515/5616 [57:29<14:20,  1.28it/s, gpt_loss=0.0703, lr=9.41e-7]
+
Train step of epoch 0:  80%|████████  | 4515/5616 [57:29<14:20,  1.28it/s, gpt_loss=0.0798, lr=9.38e-7]
+
Train step of epoch 0:  80%|████████  | 4516/5616 [57:29<14:41,  1.25it/s, gpt_loss=0.0798, lr=9.38e-7]
+
Train step of epoch 0:  80%|████████  | 4516/5616 [57:30<14:41,  1.25it/s, gpt_loss=0.0516, lr=9.38e-7]
+
Train step of epoch 0:  80%|████████  | 4517/5616 [57:30<14:22,  1.27it/s, gpt_loss=0.0516, lr=9.38e-7]
+
Train step of epoch 0:  80%|████████  | 4517/5616 [57:31<14:22,  1.27it/s, gpt_loss=0.0201, lr=9.38e-7]
+
Train step of epoch 0:  80%|████████  | 4518/5616 [57:31<13:38,  1.34it/s, gpt_loss=0.0201, lr=9.38e-7]
+
Train step of epoch 0:  80%|████████  | 4518/5616 [57:32<13:38,  1.34it/s, gpt_loss=0.0231, lr=9.38e-7]
+
Train step of epoch 0:  80%|████████  | 4519/5616 [57:32<13:38,  1.34it/s, gpt_loss=0.0231, lr=9.38e-7]
+
Train step of epoch 0:  80%|████████  | 4519/5616 [57:32<13:38,  1.34it/s, gpt_loss=0.00436, lr=9.35e-7]
+
Train step of epoch 0:  80%|████████  | 4520/5616 [57:32<13:43,  1.33it/s, gpt_loss=0.00436, lr=9.35e-7]
+
Train step of epoch 0:  80%|████████  | 4520/5616 [57:33<13:43,  1.33it/s, gpt_loss=0.0272, lr=9.35e-7] 
+
Train step of epoch 0:  81%|████████  | 4521/5616 [57:33<13:10,  1.39it/s, gpt_loss=0.0272, lr=9.35e-7]
+
Train step of epoch 0:  81%|████████  | 4521/5616 [57:34<13:10,  1.39it/s, gpt_loss=0.0555, lr=9.35e-7]
+
Train step of epoch 0:  81%|████████  | 4522/5616 [57:34<13:18,  1.37it/s, gpt_loss=0.0555, lr=9.35e-7]
+
Train step of epoch 0:  81%|████████  | 4522/5616 [57:34<13:18,  1.37it/s, gpt_loss=0.00283, lr=9.35e-7]
+
Train step of epoch 0:  81%|████████  | 4523/5616 [57:34<13:10,  1.38it/s, gpt_loss=0.00283, lr=9.35e-7]
+
Train step of epoch 0:  81%|████████  | 4523/5616 [57:35<13:10,  1.38it/s, gpt_loss=0.0129, lr=9.32e-7] 
+
Train step of epoch 0:  81%|████████  | 4524/5616 [57:35<13:19,  1.37it/s, gpt_loss=0.0129, lr=9.32e-7]
+
Train step of epoch 0:  81%|████████  | 4524/5616 [57:36<13:19,  1.37it/s, gpt_loss=0.0285, lr=9.32e-7]
+
Train step of epoch 0:  81%|████████  | 4525/5616 [57:36<13:24,  1.36it/s, gpt_loss=0.0285, lr=9.32e-7]
+
Train step of epoch 0:  81%|████████  | 4525/5616 [57:37<13:24,  1.36it/s, gpt_loss=0.0307, lr=9.32e-7]
+
Train step of epoch 0:  81%|████████  | 4526/5616 [57:37<13:18,  1.37it/s, gpt_loss=0.0307, lr=9.32e-7]
+
Train step of epoch 0:  81%|████████  | 4526/5616 [57:37<13:18,  1.37it/s, gpt_loss=0.0238, lr=9.32e-7]
+
Train step of epoch 0:  81%|████████  | 4527/5616 [57:37<12:59,  1.40it/s, gpt_loss=0.0238, lr=9.32e-7]
+
Train step of epoch 0:  81%|████████  | 4527/5616 [57:38<12:59,  1.40it/s, gpt_loss=0.0169, lr=9.29e-7]
+
Train step of epoch 0:  81%|████████  | 4528/5616 [57:38<14:04,  1.29it/s, gpt_loss=0.0169, lr=9.29e-7]
+
Train step of epoch 0:  81%|████████  | 4528/5616 [57:39<14:04,  1.29it/s, gpt_loss=0.0222, lr=9.29e-7]
+
Train step of epoch 0:  81%|████████  | 4529/5616 [57:39<13:40,  1.32it/s, gpt_loss=0.0222, lr=9.29e-7]
+
Train step of epoch 0:  81%|████████  | 4529/5616 [57:40<13:40,  1.32it/s, gpt_loss=0.0995, lr=9.29e-7]
+
Train step of epoch 0:  81%|████████  | 4530/5616 [57:40<13:51,  1.31it/s, gpt_loss=0.0995, lr=9.29e-7]
+
Train step of epoch 0:  81%|████████  | 4530/5616 [57:41<13:51,  1.31it/s, gpt_loss=0.0146, lr=9.29e-7]
+
Train step of epoch 0:  81%|████████  | 4531/5616 [57:41<14:49,  1.22it/s, gpt_loss=0.0146, lr=9.29e-7]
+
Train step of epoch 0:  81%|████████  | 4531/5616 [57:41<14:49,  1.22it/s, gpt_loss=0.0513, lr=9.26e-7]
+
Train step of epoch 0:  81%|████████  | 4532/5616 [57:41<14:44,  1.23it/s, gpt_loss=0.0513, lr=9.26e-7]
+
Train step of epoch 0:  81%|████████  | 4532/5616 [57:42<14:44,  1.23it/s, gpt_loss=0.0133, lr=9.26e-7]
+
Train step of epoch 0:  81%|████████  | 4533/5616 [57:42<14:07,  1.28it/s, gpt_loss=0.0133, lr=9.26e-7]
+
Train step of epoch 0:  81%|████████  | 4533/5616 [57:43<14:07,  1.28it/s, gpt_loss=0.0121, lr=9.26e-7]
+
Train step of epoch 0:  81%|████████  | 4534/5616 [57:43<13:55,  1.29it/s, gpt_loss=0.0121, lr=9.26e-7]
+
Train step of epoch 0:  81%|████████  | 4534/5616 [57:44<13:55,  1.29it/s, gpt_loss=0.00237, lr=9.26e-7]
+
Train step of epoch 0:  81%|████████  | 4535/5616 [57:44<13:10,  1.37it/s, gpt_loss=0.00237, lr=9.26e-7]
+
Train step of epoch 0:  81%|████████  | 4535/5616 [57:44<13:10,  1.37it/s, gpt_loss=0.0331, lr=9.23e-7] 
+
Train step of epoch 0:  81%|████████  | 4536/5616 [57:44<13:46,  1.31it/s, gpt_loss=0.0331, lr=9.23e-7]
+
Train step of epoch 0:  81%|████████  | 4536/5616 [57:45<13:46,  1.31it/s, gpt_loss=0.0612, lr=9.23e-7]
+
Train step of epoch 0:  81%|████████  | 4537/5616 [57:45<14:19,  1.26it/s, gpt_loss=0.0612, lr=9.23e-7]
+
Train step of epoch 0:  81%|████████  | 4537/5616 [57:46<14:19,  1.26it/s, gpt_loss=0.054, lr=9.23e-7] 
+
Train step of epoch 0:  81%|████████  | 4538/5616 [57:46<13:54,  1.29it/s, gpt_loss=0.054, lr=9.23e-7]
+
Train step of epoch 0:  81%|████████  | 4538/5616 [57:47<13:54,  1.29it/s, gpt_loss=0.0119, lr=9.23e-7]
+
Train step of epoch 0:  81%|████████  | 4539/5616 [57:47<13:53,  1.29it/s, gpt_loss=0.0119, lr=9.23e-7]
+
Train step of epoch 0:  81%|████████  | 4539/5616 [57:48<13:53,  1.29it/s, gpt_loss=0.00745, lr=9.2e-7]
+
Train step of epoch 0:  81%|████████  | 4540/5616 [57:48<13:36,  1.32it/s, gpt_loss=0.00745, lr=9.2e-7]
+
Train step of epoch 0:  81%|████████  | 4540/5616 [57:48<13:36,  1.32it/s, gpt_loss=0.00847, lr=9.2e-7]
+
Train step of epoch 0:  81%|████████  | 4541/5616 [57:48<13:25,  1.33it/s, gpt_loss=0.00847, lr=9.2e-7]
+
Train step of epoch 0:  81%|████████  | 4541/5616 [57:49<13:25,  1.33it/s, gpt_loss=0.103, lr=9.2e-7]  
+
Train step of epoch 0:  81%|████████  | 4542/5616 [57:49<12:52,  1.39it/s, gpt_loss=0.103, lr=9.2e-7]
+
Train step of epoch 0:  81%|████████  | 4542/5616 [57:50<12:52,  1.39it/s, gpt_loss=0.0203, lr=9.2e-7]
+
Train step of epoch 0:  81%|████████  | 4543/5616 [57:50<12:40,  1.41it/s, gpt_loss=0.0203, lr=9.2e-7]
+
Train step of epoch 0:  81%|████████  | 4543/5616 [57:50<12:40,  1.41it/s, gpt_loss=0.0101, lr=9.17e-7]
+
Train step of epoch 0:  81%|████████  | 4544/5616 [57:50<13:02,  1.37it/s, gpt_loss=0.0101, lr=9.17e-7]
+
Train step of epoch 0:  81%|████████  | 4544/5616 [57:51<13:02,  1.37it/s, gpt_loss=0.00381, lr=9.17e-7]
+
Train step of epoch 0:  81%|████████  | 4545/5616 [57:51<12:44,  1.40it/s, gpt_loss=0.00381, lr=9.17e-7]
+
Train step of epoch 0:  81%|████████  | 4545/5616 [57:52<12:44,  1.40it/s, gpt_loss=0.0306, lr=9.17e-7] 
+
Train step of epoch 0:  81%|████████  | 4546/5616 [57:52<12:41,  1.41it/s, gpt_loss=0.0306, lr=9.17e-7]
+
Train step of epoch 0:  81%|████████  | 4546/5616 [57:52<12:41,  1.41it/s, gpt_loss=0.0543, lr=9.17e-7]
+
Train step of epoch 0:  81%|████████  | 4547/5616 [57:52<12:27,  1.43it/s, gpt_loss=0.0543, lr=9.17e-7]
+
Train step of epoch 0:  81%|████████  | 4547/5616 [57:53<12:27,  1.43it/s, gpt_loss=0.0563, lr=9.14e-7]
+
Train step of epoch 0:  81%|████████  | 4548/5616 [57:53<13:34,  1.31it/s, gpt_loss=0.0563, lr=9.14e-7]
+
Train step of epoch 0:  81%|████████  | 4548/5616 [57:54<13:34,  1.31it/s, gpt_loss=0.0395, lr=9.14e-7]
+
Train step of epoch 0:  81%|████████  | 4549/5616 [57:54<13:00,  1.37it/s, gpt_loss=0.0395, lr=9.14e-7]
+
Train step of epoch 0:  81%|████████  | 4549/5616 [57:55<13:00,  1.37it/s, gpt_loss=0.00489, lr=9.14e-7]
+
Train step of epoch 0:  81%|████████  | 4550/5616 [57:55<13:23,  1.33it/s, gpt_loss=0.00489, lr=9.14e-7]
+
Train step of epoch 0:  81%|████████  | 4550/5616 [57:56<13:23,  1.33it/s, gpt_loss=0.0502, lr=9.14e-7] 
+
Train step of epoch 0:  81%|████████  | 4551/5616 [57:56<13:17,  1.34it/s, gpt_loss=0.0502, lr=9.14e-7]
+
Train step of epoch 0:  81%|████████  | 4551/5616 [57:56<13:17,  1.34it/s, gpt_loss=0.0294, lr=9.11e-7]
+
Train step of epoch 0:  81%|████████  | 4552/5616 [57:56<13:35,  1.30it/s, gpt_loss=0.0294, lr=9.11e-7]
+
Train step of epoch 0:  81%|████████  | 4552/5616 [57:57<13:35,  1.30it/s, gpt_loss=0.0234, lr=9.11e-7]
+
Train step of epoch 0:  81%|████████  | 4553/5616 [57:57<13:03,  1.36it/s, gpt_loss=0.0234, lr=9.11e-7]
+
Train step of epoch 0:  81%|████████  | 4553/5616 [57:58<13:03,  1.36it/s, gpt_loss=0.0186, lr=9.11e-7]
+
Train step of epoch 0:  81%|████████  | 4554/5616 [57:58<12:31,  1.41it/s, gpt_loss=0.0186, lr=9.11e-7]
+
Train step of epoch 0:  81%|████████  | 4554/5616 [57:58<12:31,  1.41it/s, gpt_loss=0.0362, lr=9.11e-7]
+
Train step of epoch 0:  81%|████████  | 4555/5616 [57:58<12:27,  1.42it/s, gpt_loss=0.0362, lr=9.11e-7]
+
Train step of epoch 0:  81%|████████  | 4555/5616 [57:59<12:27,  1.42it/s, gpt_loss=0.0985, lr=9.08e-7]
+
Train step of epoch 0:  81%|████████  | 4556/5616 [57:59<12:57,  1.36it/s, gpt_loss=0.0985, lr=9.08e-7]
+
Train step of epoch 0:  81%|████████  | 4556/5616 [58:00<12:57,  1.36it/s, gpt_loss=0.0455, lr=9.08e-7]
+
Train step of epoch 0:  81%|████████  | 4557/5616 [58:00<12:45,  1.38it/s, gpt_loss=0.0455, lr=9.08e-7]
+
Train step of epoch 0:  81%|████████  | 4557/5616 [58:01<12:45,  1.38it/s, gpt_loss=0.00881, lr=9.08e-7]
+
Train step of epoch 0:  81%|████████  | 4558/5616 [58:01<12:39,  1.39it/s, gpt_loss=0.00881, lr=9.08e-7]
+
Train step of epoch 0:  81%|████████  | 4558/5616 [58:01<12:39,  1.39it/s, gpt_loss=0.00402, lr=9.08e-7]
+
Train step of epoch 0:  81%|████████  | 4559/5616 [58:01<12:27,  1.41it/s, gpt_loss=0.00402, lr=9.08e-7]
+
Train step of epoch 0:  81%|████████  | 4559/5616 [58:02<12:27,  1.41it/s, gpt_loss=0.0324, lr=9.05e-7] 
+
Train step of epoch 0:  81%|████████  | 4560/5616 [58:02<12:40,  1.39it/s, gpt_loss=0.0324, lr=9.05e-7]
+
Train step of epoch 0:  81%|████████  | 4560/5616 [58:03<12:40,  1.39it/s, gpt_loss=0.0292, lr=9.05e-7]
+
Train step of epoch 0:  81%|████████  | 4561/5616 [58:03<13:26,  1.31it/s, gpt_loss=0.0292, lr=9.05e-7]
+
Train step of epoch 0:  81%|████████  | 4561/5616 [58:04<13:26,  1.31it/s, gpt_loss=0.0218, lr=9.05e-7]
+
Train step of epoch 0:  81%|████████  | 4562/5616 [58:04<13:22,  1.31it/s, gpt_loss=0.0218, lr=9.05e-7]
+
Train step of epoch 0:  81%|████████  | 4562/5616 [58:04<13:22,  1.31it/s, gpt_loss=0.00217, lr=9.05e-7]
+
Train step of epoch 0:  81%|████████▏ | 4563/5616 [58:04<13:09,  1.33it/s, gpt_loss=0.00217, lr=9.05e-7]
+
Train step of epoch 0:  81%|████████▏ | 4563/5616 [58:05<13:09,  1.33it/s, gpt_loss=0.0366, lr=9.02e-7] 
+
Train step of epoch 0:  81%|████████▏ | 4564/5616 [58:05<13:17,  1.32it/s, gpt_loss=0.0366, lr=9.02e-7]
+
Train step of epoch 0:  81%|████████▏ | 4564/5616 [58:06<13:17,  1.32it/s, gpt_loss=0.0427, lr=9.02e-7]
+
Train step of epoch 0:  81%|████████▏ | 4565/5616 [58:06<13:06,  1.34it/s, gpt_loss=0.0427, lr=9.02e-7]
+
Train step of epoch 0:  81%|████████▏ | 4565/5616 [58:07<13:06,  1.34it/s, gpt_loss=0.0433, lr=9.02e-7]
+
Train step of epoch 0:  81%|████████▏ | 4566/5616 [58:07<12:52,  1.36it/s, gpt_loss=0.0433, lr=9.02e-7]
+
Train step of epoch 0:  81%|████████▏ | 4566/5616 [58:07<12:52,  1.36it/s, gpt_loss=0.0275, lr=9.02e-7]
+
Train step of epoch 0:  81%|████████▏ | 4567/5616 [58:07<12:37,  1.38it/s, gpt_loss=0.0275, lr=9.02e-7]
+
Train step of epoch 0:  81%|████████▏ | 4567/5616 [58:08<12:37,  1.38it/s, gpt_loss=0.0233, lr=8.99e-7]
+
Train step of epoch 0:  81%|████████▏ | 4568/5616 [58:08<13:00,  1.34it/s, gpt_loss=0.0233, lr=8.99e-7]
+
Train step of epoch 0:  81%|████████▏ | 4568/5616 [58:09<13:00,  1.34it/s, gpt_loss=0.07, lr=8.99e-7]  
+
Train step of epoch 0:  81%|████████▏ | 4569/5616 [58:09<13:02,  1.34it/s, gpt_loss=0.07, lr=8.99e-7]
+
Train step of epoch 0:  81%|████████▏ | 4569/5616 [58:10<13:02,  1.34it/s, gpt_loss=0.0219, lr=8.99e-7]
+
Train step of epoch 0:  81%|████████▏ | 4570/5616 [58:10<13:06,  1.33it/s, gpt_loss=0.0219, lr=8.99e-7]
+
Train step of epoch 0:  81%|████████▏ | 4570/5616 [58:10<13:06,  1.33it/s, gpt_loss=0.014, lr=8.99e-7] 
+
Train step of epoch 0:  81%|████████▏ | 4571/5616 [58:10<12:50,  1.36it/s, gpt_loss=0.014, lr=8.99e-7]
+
Train step of epoch 0:  81%|████████▏ | 4571/5616 [58:11<12:50,  1.36it/s, gpt_loss=0.0133, lr=8.96e-7]
+
Train step of epoch 0:  81%|████████▏ | 4572/5616 [58:11<13:03,  1.33it/s, gpt_loss=0.0133, lr=8.96e-7]
+
Train step of epoch 0:  81%|████████▏ | 4572/5616 [58:12<13:03,  1.33it/s, gpt_loss=0.0117, lr=8.96e-7]
+
Train step of epoch 0:  81%|████████▏ | 4573/5616 [58:12<12:25,  1.40it/s, gpt_loss=0.0117, lr=8.96e-7]
+
Train step of epoch 0:  81%|████████▏ | 4573/5616 [58:12<12:25,  1.40it/s, gpt_loss=0.14, lr=8.96e-7]  
+
Train step of epoch 0:  81%|████████▏ | 4574/5616 [58:12<11:58,  1.45it/s, gpt_loss=0.14, lr=8.96e-7]
+
Train step of epoch 0:  81%|████████▏ | 4574/5616 [58:13<11:58,  1.45it/s, gpt_loss=0.00427, lr=8.96e-7]
+
Train step of epoch 0:  81%|████████▏ | 4575/5616 [58:13<11:51,  1.46it/s, gpt_loss=0.00427, lr=8.96e-7]
+
Train step of epoch 0:  81%|████████▏ | 4575/5616 [58:14<11:51,  1.46it/s, gpt_loss=0.045, lr=8.93e-7]  
+
Train step of epoch 0:  81%|████████▏ | 4576/5616 [58:14<12:36,  1.37it/s, gpt_loss=0.045, lr=8.93e-7]
+
Train step of epoch 0:  81%|████████▏ | 4576/5616 [58:15<12:36,  1.37it/s, gpt_loss=0.000841, lr=8.93e-7]
+
Train step of epoch 0:  81%|████████▏ | 4577/5616 [58:15<12:50,  1.35it/s, gpt_loss=0.000841, lr=8.93e-7]
+
Train step of epoch 0:  81%|████████▏ | 4577/5616 [58:15<12:50,  1.35it/s, gpt_loss=0.0452, lr=8.93e-7]  
+
Train step of epoch 0:  82%|████████▏ | 4578/5616 [58:15<12:27,  1.39it/s, gpt_loss=0.0452, lr=8.93e-7]
+
Train step of epoch 0:  82%|████████▏ | 4578/5616 [58:16<12:27,  1.39it/s, gpt_loss=0.0256, lr=8.93e-7]
+
Train step of epoch 0:  82%|████████▏ | 4579/5616 [58:16<12:49,  1.35it/s, gpt_loss=0.0256, lr=8.93e-7]
+
Train step of epoch 0:  82%|████████▏ | 4579/5616 [58:17<12:49,  1.35it/s, gpt_loss=0.0153, lr=8.9e-7] 
+
Train step of epoch 0:  82%|████████▏ | 4580/5616 [58:17<12:49,  1.35it/s, gpt_loss=0.0153, lr=8.9e-7]
+
Train step of epoch 0:  82%|████████▏ | 4580/5616 [58:17<12:49,  1.35it/s, gpt_loss=0.111, lr=8.9e-7] 
+
Train step of epoch 0:  82%|████████▏ | 4581/5616 [58:17<12:46,  1.35it/s, gpt_loss=0.111, lr=8.9e-7]
+
Train step of epoch 0:  82%|████████▏ | 4581/5616 [58:18<12:46,  1.35it/s, gpt_loss=0.00987, lr=8.9e-7]
+
Train step of epoch 0:  82%|████████▏ | 4582/5616 [58:18<13:01,  1.32it/s, gpt_loss=0.00987, lr=8.9e-7]
+
Train step of epoch 0:  82%|████████▏ | 4582/5616 [58:19<13:01,  1.32it/s, gpt_loss=0.0126, lr=8.9e-7] 
+
Train step of epoch 0:  82%|████████▏ | 4583/5616 [58:19<14:35,  1.18it/s, gpt_loss=0.0126, lr=8.9e-7]
+
Train step of epoch 0:  82%|████████▏ | 4583/5616 [58:20<14:35,  1.18it/s, gpt_loss=0.061, lr=8.87e-7]
+
Train step of epoch 0:  82%|████████▏ | 4584/5616 [58:20<14:07,  1.22it/s, gpt_loss=0.061, lr=8.87e-7]
+
Train step of epoch 0:  82%|████████▏ | 4584/5616 [58:21<14:07,  1.22it/s, gpt_loss=0.00561, lr=8.87e-7]
+
Train step of epoch 0:  82%|████████▏ | 4585/5616 [58:21<13:30,  1.27it/s, gpt_loss=0.00561, lr=8.87e-7]
+
Train step of epoch 0:  82%|████████▏ | 4585/5616 [58:22<13:30,  1.27it/s, gpt_loss=0.012, lr=8.87e-7]  
+
Train step of epoch 0:  82%|████████▏ | 4586/5616 [58:22<13:33,  1.27it/s, gpt_loss=0.012, lr=8.87e-7]
+
Train step of epoch 0:  82%|████████▏ | 4586/5616 [58:22<13:33,  1.27it/s, gpt_loss=0.0446, lr=8.87e-7]
+
Train step of epoch 0:  82%|████████▏ | 4587/5616 [58:22<13:24,  1.28it/s, gpt_loss=0.0446, lr=8.87e-7]
+
Train step of epoch 0:  82%|████████▏ | 4587/5616 [58:23<13:24,  1.28it/s, gpt_loss=0.079, lr=8.84e-7] 
+
Train step of epoch 0:  82%|████████▏ | 4588/5616 [58:23<13:57,  1.23it/s, gpt_loss=0.079, lr=8.84e-7]
+
Train step of epoch 0:  82%|████████▏ | 4588/5616 [58:24<13:57,  1.23it/s, gpt_loss=0.0526, lr=8.84e-7]
+
Train step of epoch 0:  82%|████████▏ | 4589/5616 [58:24<13:09,  1.30it/s, gpt_loss=0.0526, lr=8.84e-7]
+
Train step of epoch 0:  82%|████████▏ | 4589/5616 [58:25<13:09,  1.30it/s, gpt_loss=0.0151, lr=8.84e-7]
+
Train step of epoch 0:  82%|████████▏ | 4590/5616 [58:25<12:39,  1.35it/s, gpt_loss=0.0151, lr=8.84e-7]
+
Train step of epoch 0:  82%|████████▏ | 4590/5616 [58:25<12:39,  1.35it/s, gpt_loss=0.0728, lr=8.84e-7]
+
Train step of epoch 0:  82%|████████▏ | 4591/5616 [58:25<12:28,  1.37it/s, gpt_loss=0.0728, lr=8.84e-7]
+
Train step of epoch 0:  82%|████████▏ | 4591/5616 [58:26<12:28,  1.37it/s, gpt_loss=0.00687, lr=8.82e-7]
+
Train step of epoch 0:  82%|████████▏ | 4592/5616 [58:26<12:37,  1.35it/s, gpt_loss=0.00687, lr=8.82e-7]
+
Train step of epoch 0:  82%|████████▏ | 4592/5616 [58:27<12:37,  1.35it/s, gpt_loss=0.0683, lr=8.82e-7] 
+
Train step of epoch 0:  82%|████████▏ | 4593/5616 [58:27<12:34,  1.36it/s, gpt_loss=0.0683, lr=8.82e-7]
+
Train step of epoch 0:  82%|████████▏ | 4593/5616 [58:28<12:34,  1.36it/s, gpt_loss=0.0292, lr=8.82e-7]
+
Train step of epoch 0:  82%|████████▏ | 4594/5616 [58:28<12:20,  1.38it/s, gpt_loss=0.0292, lr=8.82e-7]
+
Train step of epoch 0:  82%|████████▏ | 4594/5616 [58:28<12:20,  1.38it/s, gpt_loss=0.0282, lr=8.82e-7]
+
Train step of epoch 0:  82%|████████▏ | 4595/5616 [58:28<12:12,  1.39it/s, gpt_loss=0.0282, lr=8.82e-7]
+
Train step of epoch 0:  82%|████████▏ | 4595/5616 [58:29<12:12,  1.39it/s, gpt_loss=0.0713, lr=8.79e-7]
+
Train step of epoch 0:  82%|████████▏ | 4596/5616 [58:29<12:29,  1.36it/s, gpt_loss=0.0713, lr=8.79e-7]
+
Train step of epoch 0:  82%|████████▏ | 4596/5616 [58:30<12:29,  1.36it/s, gpt_loss=0.00318, lr=8.79e-7]
+
Train step of epoch 0:  82%|████████▏ | 4597/5616 [58:30<12:39,  1.34it/s, gpt_loss=0.00318, lr=8.79e-7]
+
Train step of epoch 0:  82%|████████▏ | 4597/5616 [58:30<12:39,  1.34it/s, gpt_loss=0.00166, lr=8.79e-7]
+
Train step of epoch 0:  82%|████████▏ | 4598/5616 [58:30<12:25,  1.37it/s, gpt_loss=0.00166, lr=8.79e-7]
+
Train step of epoch 0:  82%|████████▏ | 4598/5616 [58:31<12:25,  1.37it/s, gpt_loss=0.0187, lr=8.79e-7] 
+
Train step of epoch 0:  82%|████████▏ | 4599/5616 [58:31<12:37,  1.34it/s, gpt_loss=0.0187, lr=8.79e-7]
+
Train step of epoch 0:  82%|████████▏ | 4599/5616 [58:32<12:37,  1.34it/s, gpt_loss=0.00705, lr=8.76e-7]
+
Train step of epoch 0:  82%|████████▏ | 4600/5616 [58:32<12:55,  1.31it/s, gpt_loss=0.00705, lr=8.76e-7]
+
Train step of epoch 0:  82%|████████▏ | 4600/5616 [58:33<12:55,  1.31it/s, gpt_loss=0.0282, lr=8.76e-7] 
+
Train step of epoch 0:  82%|████████▏ | 4601/5616 [58:33<12:44,  1.33it/s, gpt_loss=0.0282, lr=8.76e-7]
+
Train step of epoch 0:  82%|████████▏ | 4601/5616 [58:33<12:44,  1.33it/s, gpt_loss=0.0241, lr=8.76e-7]
+
Train step of epoch 0:  82%|████████▏ | 4602/5616 [58:33<12:36,  1.34it/s, gpt_loss=0.0241, lr=8.76e-7]
+
Train step of epoch 0:  82%|████████▏ | 4602/5616 [58:34<12:36,  1.34it/s, gpt_loss=0.0495, lr=8.76e-7]
+
Train step of epoch 0:  82%|████████▏ | 4603/5616 [58:34<12:19,  1.37it/s, gpt_loss=0.0495, lr=8.76e-7]
+
Train step of epoch 0:  82%|████████▏ | 4603/5616 [58:35<12:19,  1.37it/s, gpt_loss=0.0702, lr=8.73e-7]
+
Train step of epoch 0:  82%|████████▏ | 4604/5616 [58:35<12:41,  1.33it/s, gpt_loss=0.0702, lr=8.73e-7]
+
Train step of epoch 0:  82%|████████▏ | 4604/5616 [58:36<12:41,  1.33it/s, gpt_loss=0.00738, lr=8.73e-7]
+
Train step of epoch 0:  82%|████████▏ | 4605/5616 [58:36<12:49,  1.31it/s, gpt_loss=0.00738, lr=8.73e-7]
+
Train step of epoch 0:  82%|████████▏ | 4605/5616 [58:36<12:49,  1.31it/s, gpt_loss=0.113, lr=8.73e-7]  
+
Train step of epoch 0:  82%|████████▏ | 4606/5616 [58:36<12:32,  1.34it/s, gpt_loss=0.113, lr=8.73e-7]
+
Train step of epoch 0:  82%|████████▏ | 4606/5616 [58:37<12:32,  1.34it/s, gpt_loss=0.0177, lr=8.73e-7]
+
Train step of epoch 0:  82%|████████▏ | 4607/5616 [58:37<12:58,  1.30it/s, gpt_loss=0.0177, lr=8.73e-7]
+
Train step of epoch 0:  82%|████████▏ | 4607/5616 [58:38<12:58,  1.30it/s, gpt_loss=0.0406, lr=8.7e-7] 
+
Train step of epoch 0:  82%|████████▏ | 4608/5616 [58:38<13:13,  1.27it/s, gpt_loss=0.0406, lr=8.7e-7]
+
Train step of epoch 0:  82%|████████▏ | 4608/5616 [58:39<13:13,  1.27it/s, gpt_loss=0.00426, lr=8.7e-7]
+
Train step of epoch 0:  82%|████████▏ | 4609/5616 [58:39<15:27,  1.09it/s, gpt_loss=0.00426, lr=8.7e-7]
+
Train step of epoch 0:  82%|████████▏ | 4609/5616 [58:40<15:27,  1.09it/s, gpt_loss=0.0708, lr=8.7e-7] 
+
Train step of epoch 0:  82%|████████▏ | 4610/5616 [58:40<14:15,  1.18it/s, gpt_loss=0.0708, lr=8.7e-7]
+
Train step of epoch 0:  82%|████████▏ | 4610/5616 [58:41<14:15,  1.18it/s, gpt_loss=0.0183, lr=8.7e-7]
+
Train step of epoch 0:  82%|████████▏ | 4611/5616 [58:41<13:36,  1.23it/s, gpt_loss=0.0183, lr=8.7e-7]
+
Train step of epoch 0:  82%|████████▏ | 4611/5616 [58:42<13:36,  1.23it/s, gpt_loss=0.00846, lr=8.67e-7]
+
Train step of epoch 0:  82%|████████▏ | 4612/5616 [58:42<13:19,  1.26it/s, gpt_loss=0.00846, lr=8.67e-7]
+
Train step of epoch 0:  82%|████████▏ | 4612/5616 [58:42<13:19,  1.26it/s, gpt_loss=0.0417, lr=8.67e-7] 
+
Train step of epoch 0:  82%|████████▏ | 4613/5616 [58:42<14:09,  1.18it/s, gpt_loss=0.0417, lr=8.67e-7]
+
Train step of epoch 0:  82%|████████▏ | 4613/5616 [58:44<14:09,  1.18it/s, gpt_loss=0.0452, lr=8.67e-7]
+
Train step of epoch 0:  82%|████████▏ | 4614/5616 [58:44<15:04,  1.11it/s, gpt_loss=0.0452, lr=8.67e-7]
+
Train step of epoch 0:  82%|████████▏ | 4614/5616 [58:44<15:04,  1.11it/s, gpt_loss=0.0177, lr=8.67e-7]
+
Train step of epoch 0:  82%|████████▏ | 4615/5616 [58:44<13:53,  1.20it/s, gpt_loss=0.0177, lr=8.67e-7]
+
Train step of epoch 0:  82%|████████▏ | 4615/5616 [58:45<13:53,  1.20it/s, gpt_loss=0.0216, lr=8.64e-7]
+
Train step of epoch 0:  82%|████████▏ | 4616/5616 [58:45<13:56,  1.20it/s, gpt_loss=0.0216, lr=8.64e-7]
+
Train step of epoch 0:  82%|████████▏ | 4616/5616 [58:46<13:56,  1.20it/s, gpt_loss=0.0209, lr=8.64e-7]
+
Train step of epoch 0:  82%|████████▏ | 4617/5616 [58:46<13:07,  1.27it/s, gpt_loss=0.0209, lr=8.64e-7]
+
Train step of epoch 0:  82%|████████▏ | 4617/5616 [58:46<13:07,  1.27it/s, gpt_loss=0.0144, lr=8.64e-7]
+
Train step of epoch 0:  82%|████████▏ | 4618/5616 [58:46<12:36,  1.32it/s, gpt_loss=0.0144, lr=8.64e-7]
+
Train step of epoch 0:  82%|████████▏ | 4618/5616 [58:47<12:36,  1.32it/s, gpt_loss=0.0119, lr=8.64e-7]
+
Train step of epoch 0:  82%|████████▏ | 4619/5616 [58:47<12:08,  1.37it/s, gpt_loss=0.0119, lr=8.64e-7]
+
Train step of epoch 0:  82%|████████▏ | 4619/5616 [58:48<12:08,  1.37it/s, gpt_loss=0.0839, lr=8.62e-7]
+
Train step of epoch 0:  82%|████████▏ | 4620/5616 [58:48<12:23,  1.34it/s, gpt_loss=0.0839, lr=8.62e-7]
+
Train step of epoch 0:  82%|████████▏ | 4620/5616 [58:49<12:23,  1.34it/s, gpt_loss=0.0434, lr=8.62e-7]
+
Train step of epoch 0:  82%|████████▏ | 4621/5616 [58:49<12:45,  1.30it/s, gpt_loss=0.0434, lr=8.62e-7]
+
Train step of epoch 0:  82%|████████▏ | 4621/5616 [58:49<12:45,  1.30it/s, gpt_loss=0.0362, lr=8.62e-7]
+
Train step of epoch 0:  82%|████████▏ | 4622/5616 [58:49<12:15,  1.35it/s, gpt_loss=0.0362, lr=8.62e-7]
+
Train step of epoch 0:  82%|████████▏ | 4622/5616 [58:50<12:15,  1.35it/s, gpt_loss=0.0601, lr=8.62e-7]
+
Train step of epoch 0:  82%|████████▏ | 4623/5616 [58:50<11:55,  1.39it/s, gpt_loss=0.0601, lr=8.62e-7]
+
Train step of epoch 0:  82%|████████▏ | 4623/5616 [58:51<11:55,  1.39it/s, gpt_loss=0.00612, lr=8.59e-7]
+
Train step of epoch 0:  82%|████████▏ | 4624/5616 [58:51<12:21,  1.34it/s, gpt_loss=0.00612, lr=8.59e-7]
+
Train step of epoch 0:  82%|████████▏ | 4624/5616 [58:52<12:21,  1.34it/s, gpt_loss=0.0451, lr=8.59e-7] 
+
Train step of epoch 0:  82%|████████▏ | 4625/5616 [58:52<12:08,  1.36it/s, gpt_loss=0.0451, lr=8.59e-7]
+
Train step of epoch 0:  82%|████████▏ | 4625/5616 [58:52<12:08,  1.36it/s, gpt_loss=0.047, lr=8.59e-7] 
+
Train step of epoch 0:  82%|████████▏ | 4626/5616 [58:52<11:51,  1.39it/s, gpt_loss=0.047, lr=8.59e-7]
+
Train step of epoch 0:  82%|████████▏ | 4626/5616 [58:53<11:51,  1.39it/s, gpt_loss=0.0638, lr=8.59e-7]
+
Train step of epoch 0:  82%|████████▏ | 4627/5616 [58:53<11:46,  1.40it/s, gpt_loss=0.0638, lr=8.59e-7]
+
Train step of epoch 0:  82%|████████▏ | 4627/5616 [58:54<11:46,  1.40it/s, gpt_loss=0.0296, lr=8.56e-7]
+
Train step of epoch 0:  82%|█████��██▏ | 4628/5616 [58:54<12:33,  1.31it/s, gpt_loss=0.0296, lr=8.56e-7]
+
Train step of epoch 0:  82%|████████▏ | 4628/5616 [58:54<12:33,  1.31it/s, gpt_loss=0.0102, lr=8.56e-7]
+
Train step of epoch 0:  82%|████████▏ | 4629/5616 [58:54<12:11,  1.35it/s, gpt_loss=0.0102, lr=8.56e-7]
+
Train step of epoch 0:  82%|████████▏ | 4629/5616 [58:55<12:11,  1.35it/s, gpt_loss=0.0726, lr=8.56e-7]
+
Train step of epoch 0:  82%|████████▏ | 4630/5616 [58:55<12:06,  1.36it/s, gpt_loss=0.0726, lr=8.56e-7]
+
Train step of epoch 0:  82%|████████▏ | 4630/5616 [58:56<12:06,  1.36it/s, gpt_loss=0.013, lr=8.56e-7] 
+
Train step of epoch 0:  82%|████████▏ | 4631/5616 [58:56<12:39,  1.30it/s, gpt_loss=0.013, lr=8.56e-7]
+
Train step of epoch 0:  82%|████████▏ | 4631/5616 [58:57<12:39,  1.30it/s, gpt_loss=0.149, lr=8.53e-7]
+
Train step of epoch 0:  82%|████████▏ | 4632/5616 [58:57<12:33,  1.31it/s, gpt_loss=0.149, lr=8.53e-7]
+
Train step of epoch 0:  82%|████████▏ | 4632/5616 [58:57<12:33,  1.31it/s, gpt_loss=0.0542, lr=8.53e-7]
+
Train step of epoch 0:  82%|████████▏ | 4633/5616 [58:57<11:59,  1.37it/s, gpt_loss=0.0542, lr=8.53e-7]
+
Train step of epoch 0:  82%|████████▏ | 4633/5616 [58:58<11:59,  1.37it/s, gpt_loss=0.0246, lr=8.53e-7]
+
Train step of epoch 0:  83%|████████▎ | 4634/5616 [58:58<12:40,  1.29it/s, gpt_loss=0.0246, lr=8.53e-7]
+
Train step of epoch 0:  83%|████████▎ | 4634/5616 [58:59<12:40,  1.29it/s, gpt_loss=0.0362, lr=8.53e-7]
+
Train step of epoch 0:  83%|████████▎ | 4635/5616 [58:59<13:01,  1.26it/s, gpt_loss=0.0362, lr=8.53e-7]
+
Train step of epoch 0:  83%|████████▎ | 4635/5616 [59:00<13:01,  1.26it/s, gpt_loss=0.0615, lr=8.5e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4636/5616 [59:00<12:54,  1.27it/s, gpt_loss=0.0615, lr=8.5e-7]
+
Train step of epoch 0:  83%|████████▎ | 4636/5616 [59:01<12:54,  1.27it/s, gpt_loss=0.0312, lr=8.5e-7]
+
Train step of epoch 0:  83%|████████▎ | 4637/5616 [59:01<12:24,  1.31it/s, gpt_loss=0.0312, lr=8.5e-7]
+
Train step of epoch 0:  83%|████████▎ | 4637/5616 [59:01<12:24,  1.31it/s, gpt_loss=0.00438, lr=8.5e-7]
+
Train step of epoch 0:  83%|████████▎ | 4638/5616 [59:01<12:01,  1.35it/s, gpt_loss=0.00438, lr=8.5e-7]
+
Train step of epoch 0:  83%|████████▎ | 4638/5616 [59:02<12:01,  1.35it/s, gpt_loss=0.0135, lr=8.5e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4639/5616 [59:02<11:49,  1.38it/s, gpt_loss=0.0135, lr=8.5e-7]
+
Train step of epoch 0:  83%|████████▎ | 4639/5616 [59:03<11:49,  1.38it/s, gpt_loss=0.0363, lr=8.48e-7]
+
Train step of epoch 0:  83%|████████▎ | 4640/5616 [59:03<12:16,  1.32it/s, gpt_loss=0.0363, lr=8.48e-7]
+
Train step of epoch 0:  83%|████████▎ | 4640/5616 [59:04<12:16,  1.32it/s, gpt_loss=0.0598, lr=8.48e-7]
+
Train step of epoch 0:  83%|████████▎ | 4641/5616 [59:04<12:36,  1.29it/s, gpt_loss=0.0598, lr=8.48e-7]
+
Train step of epoch 0:  83%|████████▎ | 4641/5616 [59:04<12:36,  1.29it/s, gpt_loss=0.0124, lr=8.48e-7]
+
Train step of epoch 0:  83%|████████▎ | 4642/5616 [59:04<12:35,  1.29it/s, gpt_loss=0.0124, lr=8.48e-7]
+
Train step of epoch 0:  83%|████████▎ | 4642/5616 [59:05<12:35,  1.29it/s, gpt_loss=0.0679, lr=8.48e-7]
+
Train step of epoch 0:  83%|████████▎ | 4643/5616 [59:05<12:42,  1.28it/s, gpt_loss=0.0679, lr=8.48e-7]
+
Train step of epoch 0:  83%|████████▎ | 4643/5616 [59:06<12:42,  1.28it/s, gpt_loss=0.0208, lr=8.45e-7]
+
Train step of epoch 0:  83%|████████▎ | 4644/5616 [59:06<12:37,  1.28it/s, gpt_loss=0.0208, lr=8.45e-7]
+
Train step of epoch 0:  83%|████████▎ | 4644/5616 [59:07<12:37,  1.28it/s, gpt_loss=0.059, lr=8.45e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4645/5616 [59:07<12:23,  1.31it/s, gpt_loss=0.059, lr=8.45e-7]
+
Train step of epoch 0:  83%|████████▎ | 4645/5616 [59:08<12:23,  1.31it/s, gpt_loss=0.0319, lr=8.45e-7]
+
Train step of epoch 0:  83%|████████▎ | 4646/5616 [59:08<12:16,  1.32it/s, gpt_loss=0.0319, lr=8.45e-7]
+
Train step of epoch 0:  83%|████████▎ | 4646/5616 [59:08<12:16,  1.32it/s, gpt_loss=0.0454, lr=8.45e-7]
+
Train step of epoch 0:  83%|████████▎ | 4647/5616 [59:08<12:34,  1.28it/s, gpt_loss=0.0454, lr=8.45e-7]
+
Train step of epoch 0:  83%|████████▎ | 4647/5616 [59:09<12:34,  1.28it/s, gpt_loss=0.0919, lr=8.42e-7]
+
Train step of epoch 0:  83%|████████▎ | 4648/5616 [59:09<12:42,  1.27it/s, gpt_loss=0.0919, lr=8.42e-7]
+
Train step of epoch 0:  83%|████████▎ | 4648/5616 [59:10<12:42,  1.27it/s, gpt_loss=0.00823, lr=8.42e-7]
+
Train step of epoch 0:  83%|████████▎ | 4649/5616 [59:10<12:47,  1.26it/s, gpt_loss=0.00823, lr=8.42e-7]
+
Train step of epoch 0:  83%|████████▎ | 4649/5616 [59:11<12:47,  1.26it/s, gpt_loss=0.0279, lr=8.42e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4650/5616 [59:11<12:03,  1.33it/s, gpt_loss=0.0279, lr=8.42e-7]
+
Train step of epoch 0:  83%|████████▎ | 4650/5616 [59:11<12:03,  1.33it/s, gpt_loss=0.0134, lr=8.42e-7]
+
Train step of epoch 0:  83%|████████▎ | 4651/5616 [59:11<12:01,  1.34it/s, gpt_loss=0.0134, lr=8.42e-7]
+
Train step of epoch 0:  83%|████████▎ | 4651/5616 [59:12<12:01,  1.34it/s, gpt_loss=0.00513, lr=8.39e-7]
+
Train step of epoch 0:  83%|████████▎ | 4652/5616 [59:12<11:59,  1.34it/s, gpt_loss=0.00513, lr=8.39e-7]
+
Train step of epoch 0:  83%|████████▎ | 4652/5616 [59:13<11:59,  1.34it/s, gpt_loss=0.00843, lr=8.39e-7]
+
Train step of epoch 0:  83%|████████▎ | 4653/5616 [59:13<11:36,  1.38it/s, gpt_loss=0.00843, lr=8.39e-7]
+
Train step of epoch 0:  83%|████████▎ | 4653/5616 [59:13<11:36,  1.38it/s, gpt_loss=0.0124, lr=8.39e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4654/5616 [59:13<11:35,  1.38it/s, gpt_loss=0.0124, lr=8.39e-7]
+
Train step of epoch 0:  83%|████████▎ | 4654/5616 [59:14<11:35,  1.38it/s, gpt_loss=0.0511, lr=8.39e-7]
+
Train step of epoch 0:  83%|████████▎ | 4655/5616 [59:14<11:24,  1.40it/s, gpt_loss=0.0511, lr=8.39e-7]
+
Train step of epoch 0:  83%|████████▎ | 4655/5616 [59:15<11:24,  1.40it/s, gpt_loss=0.00602, lr=8.37e-7]
+
Train step of epoch 0:  83%|████████▎ | 4656/5616 [59:15<11:42,  1.37it/s, gpt_loss=0.00602, lr=8.37e-7]
+
Train step of epoch 0:  83%|████████▎ | 4656/5616 [59:16<11:42,  1.37it/s, gpt_loss=0.0672, lr=8.37e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4657/5616 [59:16<12:48,  1.25it/s, gpt_loss=0.0672, lr=8.37e-7]
+
Train step of epoch 0:  83%|████████▎ | 4657/5616 [59:17<12:48,  1.25it/s, gpt_loss=0.1, lr=8.37e-7]   
+
Train step of epoch 0:  83%|████████▎ | 4658/5616 [59:17<12:37,  1.26it/s, gpt_loss=0.1, lr=8.37e-7]
+
Train step of epoch 0:  83%|████████▎ | 4658/5616 [59:17<12:37,  1.26it/s, gpt_loss=0.0133, lr=8.37e-7]
+
Train step of epoch 0:  83%|████████▎ | 4659/5616 [59:17<12:19,  1.29it/s, gpt_loss=0.0133, lr=8.37e-7]
+
Train step of epoch 0:  83%|████████▎ | 4659/5616 [59:18<12:19,  1.29it/s, gpt_loss=0.00173, lr=8.34e-7]
+
Train step of epoch 0:  83%|████████▎ | 4660/5616 [59:18<12:21,  1.29it/s, gpt_loss=0.00173, lr=8.34e-7]
+
Train step of epoch 0:  83%|████████▎ | 4660/5616 [59:19<12:21,  1.29it/s, gpt_loss=0.0233, lr=8.34e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4661/5616 [59:19<11:44,  1.36it/s, gpt_loss=0.0233, lr=8.34e-7]
+
Train step of epoch 0:  83%|████████▎ | 4661/5616 [59:20<11:44,  1.36it/s, gpt_loss=0.0161, lr=8.34e-7]
+
Train step of epoch 0:  83%|████████▎ | 4662/5616 [59:20<11:29,  1.38it/s, gpt_loss=0.0161, lr=8.34e-7]
+
Train step of epoch 0:  83%|████████▎ | 4662/5616 [59:20<11:29,  1.38it/s, gpt_loss=0.066, lr=8.34e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4663/5616 [59:20<11:22,  1.40it/s, gpt_loss=0.066, lr=8.34e-7]
+
Train step of epoch 0:  83%|████████▎ | 4663/5616 [59:21<11:22,  1.40it/s, gpt_loss=0.0227, lr=8.31e-7]
+
Train step of epoch 0:  83%|████████▎ | 4664/5616 [59:21<11:53,  1.33it/s, gpt_loss=0.0227, lr=8.31e-7]
+
Train step of epoch 0:  83%|████████▎ | 4664/5616 [59:22<11:53,  1.33it/s, gpt_loss=0.0363, lr=8.31e-7]
+
Train step of epoch 0:  83%|████████▎ | 4665/5616 [59:22<11:34,  1.37it/s, gpt_loss=0.0363, lr=8.31e-7]
+
Train step of epoch 0:  83%|████████▎ | 4665/5616 [59:22<11:34,  1.37it/s, gpt_loss=0.00705, lr=8.31e-7]
+
Train step of epoch 0:  83%|████████▎ | 4666/5616 [59:22<11:21,  1.39it/s, gpt_loss=0.00705, lr=8.31e-7]
+
Train step of epoch 0:  83%|████████▎ | 4666/5616 [59:23<11:21,  1.39it/s, gpt_loss=0.0376, lr=8.31e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4667/5616 [59:23<10:59,  1.44it/s, gpt_loss=0.0376, lr=8.31e-7]
+
Train step of epoch 0:  83%|████████▎ | 4667/5616 [59:24<10:59,  1.44it/s, gpt_loss=0.00517, lr=8.28e-7]
+
Train step of epoch 0:  83%|████████▎ | 4668/5616 [59:24<11:49,  1.34it/s, gpt_loss=0.00517, lr=8.28e-7]
+
Train step of epoch 0:  83%|████████▎ | 4668/5616 [59:25<11:49,  1.34it/s, gpt_loss=0.0158, lr=8.28e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4669/5616 [59:25<12:10,  1.30it/s, gpt_loss=0.0158, lr=8.28e-7]
+
Train step of epoch 0:  83%|████████▎ | 4669/5616 [59:25<12:10,  1.30it/s, gpt_loss=0.031, lr=8.28e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4670/5616 [59:25<11:51,  1.33it/s, gpt_loss=0.031, lr=8.28e-7]
+
Train step of epoch 0:  83%|████████▎ | 4670/5616 [59:26<11:51,  1.33it/s, gpt_loss=0.0132, lr=8.28e-7]
+
Train step of epoch 0:  83%|████████▎ | 4671/5616 [59:26<11:52,  1.33it/s, gpt_loss=0.0132, lr=8.28e-7]
+
Train step of epoch 0:  83%|████████▎ | 4671/5616 [59:27<11:52,  1.33it/s, gpt_loss=0.0109, lr=8.26e-7]
+
Train step of epoch 0:  83%|████████▎ | 4672/5616 [59:27<12:04,  1.30it/s, gpt_loss=0.0109, lr=8.26e-7]
+
Train step of epoch 0:  83%|████████▎ | 4672/5616 [59:28<12:04,  1.30it/s, gpt_loss=0.0281, lr=8.26e-7]
+
Train step of epoch 0:  83%|████████▎ | 4673/5616 [59:28<12:20,  1.27it/s, gpt_loss=0.0281, lr=8.26e-7]
+
Train step of epoch 0:  83%|████████▎ | 4673/5616 [59:29<12:20,  1.27it/s, gpt_loss=0.0499, lr=8.26e-7]
+
Train step of epoch 0:  83%|████████▎ | 4674/5616 [59:29<11:56,  1.31it/s, gpt_loss=0.0499, lr=8.26e-7]
+
Train step of epoch 0:  83%|████████▎ | 4674/5616 [59:29<11:56,  1.31it/s, gpt_loss=0.00898, lr=8.26e-7]
+
Train step of epoch 0:  83%|████████▎ | 4675/5616 [59:29<12:10,  1.29it/s, gpt_loss=0.00898, lr=8.26e-7]
+
Train step of epoch 0:  83%|████████▎ | 4675/5616 [59:30<12:10,  1.29it/s, gpt_loss=0.0069, lr=8.23e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4676/5616 [59:30<12:44,  1.23it/s, gpt_loss=0.0069, lr=8.23e-7]
+
Train step of epoch 0:  83%|████████▎ | 4676/5616 [59:31<12:44,  1.23it/s, gpt_loss=0.00624, lr=8.23e-7]
+
Train step of epoch 0:  83%|████████▎ | 4677/5616 [59:31<11:56,  1.31it/s, gpt_loss=0.00624, lr=8.23e-7]
+
Train step of epoch 0:  83%|████████▎ | 4677/5616 [59:32<11:56,  1.31it/s, gpt_loss=0.0126, lr=8.23e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4678/5616 [59:32<12:08,  1.29it/s, gpt_loss=0.0126, lr=8.23e-7]
+
Train step of epoch 0:  83%|████████▎ | 4678/5616 [59:32<12:08,  1.29it/s, gpt_loss=0.0127, lr=8.23e-7]
+
Train step of epoch 0:  83%|████████▎ | 4679/5616 [59:32<11:47,  1.32it/s, gpt_loss=0.0127, lr=8.23e-7]
+
Train step of epoch 0:  83%|████████▎ | 4679/5616 [59:33<11:47,  1.32it/s, gpt_loss=0.0228, lr=8.2e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4680/5616 [59:33<12:19,  1.27it/s, gpt_loss=0.0228, lr=8.2e-7]
+
Train step of epoch 0:  83%|████████▎ | 4680/5616 [59:34<12:19,  1.27it/s, gpt_loss=0.0218, lr=8.2e-7]
+
Train step of epoch 0:  83%|████████▎ | 4681/5616 [59:34<11:47,  1.32it/s, gpt_loss=0.0218, lr=8.2e-7]
+
Train step of epoch 0:  83%|████████▎ | 4681/5616 [59:35<11:47,  1.32it/s, gpt_loss=0.0381, lr=8.2e-7]
+
Train step of epoch 0:  83%|████████▎ | 4682/5616 [59:35<12:11,  1.28it/s, gpt_loss=0.0381, lr=8.2e-7]
+
Train step of epoch 0:  83%|████████▎ | 4682/5616 [59:36<12:11,  1.28it/s, gpt_loss=0.0375, lr=8.2e-7]
+
Train step of epoch 0:  83%|████████▎ | 4683/5616 [59:36<12:13,  1.27it/s, gpt_loss=0.0375, lr=8.2e-7]
+
Train step of epoch 0:  83%|████████▎ | 4683/5616 [59:36<12:13,  1.27it/s, gpt_loss=0.00928, lr=8.18e-7]
+
Train step of epoch 0:  83%|████████▎ | 4684/5616 [59:36<12:07,  1.28it/s, gpt_loss=0.00928, lr=8.18e-7]
+
Train step of epoch 0:  83%|████████▎ | 4684/5616 [59:37<12:07,  1.28it/s, gpt_loss=0.0577, lr=8.18e-7] 
+
Train step of epoch 0:  83%|████████▎ | 4685/5616 [59:37<11:44,  1.32it/s, gpt_loss=0.0577, lr=8.18e-7]
+
Train step of epoch 0:  83%|████████▎ | 4685/5616 [59:38<11:44,  1.32it/s, gpt_loss=0.0349, lr=8.18e-7]
+
Train step of epoch 0:  83%|████████▎ | 4686/5616 [59:38<11:13,  1.38it/s, gpt_loss=0.0349, lr=8.18e-7]
+
Train step of epoch 0:  83%|████████▎ | 4686/5616 [59:39<11:13,  1.38it/s, gpt_loss=0.0258, lr=8.18e-7]
+
Train step of epoch 0:  83%|████████▎ | 4687/5616 [59:39<11:27,  1.35it/s, gpt_loss=0.0258, lr=8.18e-7]
+
Train step of epoch 0:  83%|████████▎ | 4687/5616 [59:39<11:27,  1.35it/s, gpt_loss=0.0629, lr=8.15e-7]
+
Train step of epoch 0:  83%|████████▎ | 4688/5616 [59:39<11:49,  1.31it/s, gpt_loss=0.0629, lr=8.15e-7]
+
Train step of epoch 0:  83%|���███████▎ | 4688/5616 [59:40<11:49,  1.31it/s, gpt_loss=0.0485, lr=8.15e-7]
+
Train step of epoch 0:  83%|████████▎ | 4689/5616 [59:40<11:32,  1.34it/s, gpt_loss=0.0485, lr=8.15e-7]
+
Train step of epoch 0:  83%|████████▎ | 4689/5616 [59:41<11:32,  1.34it/s, gpt_loss=0.0775, lr=8.15e-7]
+
Train step of epoch 0:  84%|████████▎ | 4690/5616 [59:41<11:53,  1.30it/s, gpt_loss=0.0775, lr=8.15e-7]
+
Train step of epoch 0:  84%|████████▎ | 4690/5616 [59:42<11:53,  1.30it/s, gpt_loss=0.0326, lr=8.15e-7]
+
Train step of epoch 0:  84%|████████▎ | 4691/5616 [59:42<13:01,  1.18it/s, gpt_loss=0.0326, lr=8.15e-7]
+
Train step of epoch 0:  84%|████████▎ | 4691/5616 [59:43<13:01,  1.18it/s, gpt_loss=0.0236, lr=8.12e-7]
+
Train step of epoch 0:  84%|████████▎ | 4692/5616 [59:43<12:49,  1.20it/s, gpt_loss=0.0236, lr=8.12e-7]
+
Train step of epoch 0:  84%|████████▎ | 4692/5616 [59:43<12:49,  1.20it/s, gpt_loss=0.0172, lr=8.12e-7]
+
Train step of epoch 0:  84%|████████▎ | 4693/5616 [59:43<12:32,  1.23it/s, gpt_loss=0.0172, lr=8.12e-7]
+
Train step of epoch 0:  84%|████████▎ | 4693/5616 [59:44<12:32,  1.23it/s, gpt_loss=0.0123, lr=8.12e-7]
+
Train step of epoch 0:  84%|████████▎ | 4694/5616 [59:44<11:52,  1.29it/s, gpt_loss=0.0123, lr=8.12e-7]
+
Train step of epoch 0:  84%|████████▎ | 4694/5616 [59:45<11:52,  1.29it/s, gpt_loss=0.0664, lr=8.12e-7]
+
Train step of epoch 0:  84%|████████▎ | 4695/5616 [59:45<12:11,  1.26it/s, gpt_loss=0.0664, lr=8.12e-7]
+
Train step of epoch 0:  84%|████████▎ | 4695/5616 [59:46<12:11,  1.26it/s, gpt_loss=0.00507, lr=8.1e-7]
+
Train step of epoch 0:  84%|████████▎ | 4696/5616 [59:46<11:54,  1.29it/s, gpt_loss=0.00507, lr=8.1e-7]
+
Train step of epoch 0:  84%|████████▎ | 4696/5616 [59:46<11:54,  1.29it/s, gpt_loss=0.064, lr=8.1e-7]  
+
Train step of epoch 0:  84%|████████▎ | 4697/5616 [59:46<11:14,  1.36it/s, gpt_loss=0.064, lr=8.1e-7]
+
Train step of epoch 0:  84%|████████▎ | 4697/5616 [59:47<11:14,  1.36it/s, gpt_loss=0.0469, lr=8.1e-7]
+
Train step of epoch 0:  84%|████████▎ | 4698/5616 [59:47<10:59,  1.39it/s, gpt_loss=0.0469, lr=8.1e-7]
+
Train step of epoch 0:  84%|████████▎ | 4698/5616 [59:48<10:59,  1.39it/s, gpt_loss=0.0243, lr=8.1e-7]
+
Train step of epoch 0:  84%|████████▎ | 4699/5616 [59:48<11:16,  1.36it/s, gpt_loss=0.0243, lr=8.1e-7]
+
Train step of epoch 0:  84%|████████▎ | 4699/5616 [59:49<11:16,  1.36it/s, gpt_loss=0.0427, lr=8.07e-7]
+
Train step of epoch 0:  84%|████████▎ | 4700/5616 [59:49<11:19,  1.35it/s, gpt_loss=0.0427, lr=8.07e-7]
+
Train step of epoch 0:  84%|████████▎ | 4700/5616 [59:49<11:19,  1.35it/s, gpt_loss=0.026, lr=8.07e-7] 
+
Train step of epoch 0:  84%|████████▎ | 4701/5616 [59:49<11:27,  1.33it/s, gpt_loss=0.026, lr=8.07e-7]
+
Train step of epoch 0:  84%|████████▎ | 4701/5616 [59:50<11:27,  1.33it/s, gpt_loss=0.00517, lr=8.07e-7]
+
Train step of epoch 0:  84%|████████▎ | 4702/5616 [59:50<11:08,  1.37it/s, gpt_loss=0.00517, lr=8.07e-7]
+
Train step of epoch 0:  84%|████████▎ | 4702/5616 [59:51<11:08,  1.37it/s, gpt_loss=0.0135, lr=8.07e-7] 
+
Train step of epoch 0:  84%|████████▎ | 4703/5616 [59:51<10:56,  1.39it/s, gpt_loss=0.0135, lr=8.07e-7]
+
Train step of epoch 0:  84%|████████▎ | 4703/5616 [59:52<10:56,  1.39it/s, gpt_loss=0.0416, lr=8.04e-7]
+
Train step of epoch 0:  84%|████████▍ | 4704/5616 [59:52<11:14,  1.35it/s, gpt_loss=0.0416, lr=8.04e-7]
+
Train step of epoch 0:  84%|████████▍ | 4704/5616 [59:52<11:14,  1.35it/s, gpt_loss=0.0079, lr=8.04e-7]
+
Train step of epoch 0:  84%|████████▍ | 4705/5616 [59:52<11:15,  1.35it/s, gpt_loss=0.0079, lr=8.04e-7]
+
Train step of epoch 0:  84%|████████▍ | 4705/5616 [59:53<11:15,  1.35it/s, gpt_loss=0.0211, lr=8.04e-7]
+
Train step of epoch 0:  84%|████████▍ | 4706/5616 [59:53<13:01,  1.16it/s, gpt_loss=0.0211, lr=8.04e-7]
+
Train step of epoch 0:  84%|████████▍ | 4706/5616 [59:54<13:01,  1.16it/s, gpt_loss=0.0104, lr=8.04e-7]
+
Train step of epoch 0:  84%|████████▍ | 4707/5616 [59:54<12:08,  1.25it/s, gpt_loss=0.0104, lr=8.04e-7]
+
Train step of epoch 0:  84%|████████▍ | 4707/5616 [59:55<12:08,  1.25it/s, gpt_loss=0.031, lr=8.02e-7] 
+
Train step of epoch 0:  84%|████████▍ | 4708/5616 [59:55<12:14,  1.24it/s, gpt_loss=0.031, lr=8.02e-7]
+
Train step of epoch 0:  84%|████████▍ | 4708/5616 [59:56<12:14,  1.24it/s, gpt_loss=0.0186, lr=8.02e-7]
+
Train step of epoch 0:  84%|████████▍ | 4709/5616 [59:56<11:51,  1.27it/s, gpt_loss=0.0186, lr=8.02e-7]
+
Train step of epoch 0:  84%|████████▍ | 4709/5616 [59:57<11:51,  1.27it/s, gpt_loss=0.0322, lr=8.02e-7]
+
Train step of epoch 0:  84%|████████▍ | 4710/5616 [59:57<12:45,  1.18it/s, gpt_loss=0.0322, lr=8.02e-7]
+
Train step of epoch 0:  84%|████████▍ | 4710/5616 [59:57<12:45,  1.18it/s, gpt_loss=0.0324, lr=8.02e-7]
+
Train step of epoch 0:  84%|████████▍ | 4711/5616 [59:57<11:58,  1.26it/s, gpt_loss=0.0324, lr=8.02e-7]
+
Train step of epoch 0:  84%|████████▍ | 4711/5616 [59:58<11:58,  1.26it/s, gpt_loss=0.041, lr=7.99e-7] 
+
Train step of epoch 0:  84%|████████▍ | 4712/5616 [59:58<11:55,  1.26it/s, gpt_loss=0.041, lr=7.99e-7]
+
Train step of epoch 0:  84%|████████▍ | 4712/5616 [59:59<11:55,  1.26it/s, gpt_loss=0.0326, lr=7.99e-7]
+
Train step of epoch 0:  84%|████████▍ | 4713/5616 [59:59<11:30,  1.31it/s, gpt_loss=0.0326, lr=7.99e-7]
+
Train step of epoch 0:  84%|████████▍ | 4713/5616 [1:00:00<11:30,  1.31it/s, gpt_loss=0.0232, lr=7.99e-7]
+
Train step of epoch 0:  84%|████████▍ | 4714/5616 [1:00:00<11:32,  1.30it/s, gpt_loss=0.0232, lr=7.99e-7]
+
Train step of epoch 0:  84%|████████▍ | 4714/5616 [1:00:00<11:32,  1.30it/s, gpt_loss=0.0172, lr=7.99e-7]
+
Train step of epoch 0:  84%|████████▍ | 4715/5616 [1:00:00<11:14,  1.34it/s, gpt_loss=0.0172, lr=7.99e-7]
+
Train step of epoch 0:  84%|████████▍ | 4715/5616 [1:00:01<11:14,  1.34it/s, gpt_loss=0.0277, lr=7.97e-7]
+
Train step of epoch 0:  84%|████████▍ | 4716/5616 [1:00:01<11:47,  1.27it/s, gpt_loss=0.0277, lr=7.97e-7]
+
Train step of epoch 0:  84%|████████▍ | 4716/5616 [1:00:02<11:47,  1.27it/s, gpt_loss=0.00279, lr=7.97e-7]
+
Train step of epoch 0:  84%|████████▍ | 4717/5616 [1:00:02<11:21,  1.32it/s, gpt_loss=0.00279, lr=7.97e-7]
+
Train step of epoch 0:  84%|████████▍ | 4717/5616 [1:00:03<11:21,  1.32it/s, gpt_loss=0.00785, lr=7.97e-7]
+
Train step of epoch 0:  84%|████████▍ | 4718/5616 [1:00:03<11:12,  1.34it/s, gpt_loss=0.00785, lr=7.97e-7]
+
Train step of epoch 0:  84%|████████▍ | 4718/5616 [1:00:03<11:12,  1.34it/s, gpt_loss=0.0668, lr=7.97e-7] 
+
Train step of epoch 0:  84%|████████▍ | 4719/5616 [1:00:03<10:58,  1.36it/s, gpt_loss=0.0668, lr=7.97e-7]
+
Train step of epoch 0:  84%|████████▍ | 4719/5616 [1:00:04<10:58,  1.36it/s, gpt_loss=0.0291, lr=7.94e-7]
+
Train step of epoch 0:  84%|████████▍ | 4720/5616 [1:00:04<11:20,  1.32it/s, gpt_loss=0.0291, lr=7.94e-7]
+
Train step of epoch 0:  84%|████████▍ | 4720/5616 [1:00:05<11:20,  1.32it/s, gpt_loss=0.0381, lr=7.94e-7]
+
Train step of epoch 0:  84%|████████▍ | 4721/5616 [1:00:05<11:34,  1.29it/s, gpt_loss=0.0381, lr=7.94e-7]
+
Train step of epoch 0:  84%|████████▍ | 4721/5616 [1:00:06<11:34,  1.29it/s, gpt_loss=0.0288, lr=7.94e-7]
+
Train step of epoch 0:  84%|████████▍ | 4722/5616 [1:00:06<12:46,  1.17it/s, gpt_loss=0.0288, lr=7.94e-7]
+
Train step of epoch 0:  84%|████████▍ | 4722/5616 [1:00:07<12:46,  1.17it/s, gpt_loss=0.0713, lr=7.94e-7]
+
Train step of epoch 0:  84%|████████▍ | 4723/5616 [1:00:07<12:50,  1.16it/s, gpt_loss=0.0713, lr=7.94e-7]
+
Train step of epoch 0:  84%|████████▍ | 4723/5616 [1:00:08<12:50,  1.16it/s, gpt_loss=0.0599, lr=7.92e-7]
+
Train step of epoch 0:  84%|████████▍ | 4724/5616 [1:00:08<13:02,  1.14it/s, gpt_loss=0.0599, lr=7.92e-7]
+
Train step of epoch 0:  84%|████████▍ | 4724/5616 [1:00:08<13:02,  1.14it/s, gpt_loss=0.00286, lr=7.92e-7]
+
Train step of epoch 0:  84%|████████▍ | 4725/5616 [1:00:08<12:12,  1.22it/s, gpt_loss=0.00286, lr=7.92e-7]
+
Train step of epoch 0:  84%|████████▍ | 4725/5616 [1:00:09<12:12,  1.22it/s, gpt_loss=0.00402, lr=7.92e-7]
+
Train step of epoch 0:  84%|████████▍ | 4726/5616 [1:00:09<11:34,  1.28it/s, gpt_loss=0.00402, lr=7.92e-7]
+
Train step of epoch 0:  84%|████████▍ | 4726/5616 [1:00:10<11:34,  1.28it/s, gpt_loss=0.0176, lr=7.92e-7] 
+
Train step of epoch 0:  84%|████████▍ | 4727/5616 [1:00:10<12:50,  1.15it/s, gpt_loss=0.0176, lr=7.92e-7]
+
Train step of epoch 0:  84%|████████▍ | 4727/5616 [1:00:11<12:50,  1.15it/s, gpt_loss=0.0371, lr=7.89e-7]
+
Train step of epoch 0:  84%|████████▍ | 4728/5616 [1:00:11<12:17,  1.20it/s, gpt_loss=0.0371, lr=7.89e-7]
+
Train step of epoch 0:  84%|█████��██▍ | 4728/5616 [1:00:12<12:17,  1.20it/s, gpt_loss=0.0462, lr=7.89e-7]
+
Train step of epoch 0:  84%|████████▍ | 4729/5616 [1:00:12<11:45,  1.26it/s, gpt_loss=0.0462, lr=7.89e-7]
+
Train step of epoch 0:  84%|████████▍ | 4729/5616 [1:00:12<11:45,  1.26it/s, gpt_loss=0.0111, lr=7.89e-7]
+
Train step of epoch 0:  84%|████████▍ | 4730/5616 [1:00:12<11:11,  1.32it/s, gpt_loss=0.0111, lr=7.89e-7]
+
Train step of epoch 0:  84%|████████▍ | 4730/5616 [1:00:13<11:11,  1.32it/s, gpt_loss=0.0132, lr=7.89e-7]
+
Train step of epoch 0:  84%|████████▍ | 4731/5616 [1:00:13<10:48,  1.36it/s, gpt_loss=0.0132, lr=7.89e-7]
+
Train step of epoch 0:  84%|████████▍ | 4731/5616 [1:00:14<10:48,  1.36it/s, gpt_loss=0.0124, lr=7.86e-7]
+
Train step of epoch 0:  84%|████████▍ | 4732/5616 [1:00:14<11:07,  1.32it/s, gpt_loss=0.0124, lr=7.86e-7]
+
Train step of epoch 0:  84%|████████▍ | 4732/5616 [1:00:15<11:07,  1.32it/s, gpt_loss=0.00789, lr=7.86e-7]
+
Train step of epoch 0:  84%|████████▍ | 4733/5616 [1:00:15<11:24,  1.29it/s, gpt_loss=0.00789, lr=7.86e-7]
+
Train step of epoch 0:  84%|████████▍ | 4733/5616 [1:00:15<11:24,  1.29it/s, gpt_loss=0.00172, lr=7.86e-7]
+
Train step of epoch 0:  84%|████████▍ | 4734/5616 [1:00:15<11:01,  1.33it/s, gpt_loss=0.00172, lr=7.86e-7]
+
Train step of epoch 0:  84%|████████▍ | 4734/5616 [1:00:16<11:01,  1.33it/s, gpt_loss=0.0246, lr=7.86e-7] 
+
Train step of epoch 0:  84%|████████▍ | 4735/5616 [1:00:16<10:51,  1.35it/s, gpt_loss=0.0246, lr=7.86e-7]
+
Train step of epoch 0:  84%|████████▍ | 4735/5616 [1:00:17<10:51,  1.35it/s, gpt_loss=0.146, lr=7.84e-7] 
+
Train step of epoch 0:  84%|████████▍ | 4736/5616 [1:00:17<12:02,  1.22it/s, gpt_loss=0.146, lr=7.84e-7]
+
Train step of epoch 0:  84%|████████▍ | 4736/5616 [1:00:18<12:02,  1.22it/s, gpt_loss=0.00618, lr=7.84e-7]
+
Train step of epoch 0:  84%|████████▍ | 4737/5616 [1:00:18<11:25,  1.28it/s, gpt_loss=0.00618, lr=7.84e-7]
+
Train step of epoch 0:  84%|████████▍ | 4737/5616 [1:00:18<11:25,  1.28it/s, gpt_loss=0.0523, lr=7.84e-7] 
+
Train step of epoch 0:  84%|████████▍ | 4738/5616 [1:00:18<10:49,  1.35it/s, gpt_loss=0.0523, lr=7.84e-7]
+
Train step of epoch 0:  84%|████████▍ | 4738/5616 [1:00:19<10:49,  1.35it/s, gpt_loss=0.00217, lr=7.84e-7]
+
Train step of epoch 0:  84%|████████▍ | 4739/5616 [1:00:19<10:55,  1.34it/s, gpt_loss=0.00217, lr=7.84e-7]
+
Train step of epoch 0:  84%|████████▍ | 4739/5616 [1:00:20<10:55,  1.34it/s, gpt_loss=0.0572, lr=7.81e-7] 
+
Train step of epoch 0:  84%|████████▍ | 4740/5616 [1:00:20<11:33,  1.26it/s, gpt_loss=0.0572, lr=7.81e-7]
+
Train step of epoch 0:  84%|████████▍ | 4740/5616 [1:00:21<11:33,  1.26it/s, gpt_loss=0.00923, lr=7.81e-7]
+
Train step of epoch 0:  84%|████████▍ | 4741/5616 [1:00:21<11:20,  1.29it/s, gpt_loss=0.00923, lr=7.81e-7]
+
Train step of epoch 0:  84%|████████▍ | 4741/5616 [1:00:21<11:20,  1.29it/s, gpt_loss=0.00272, lr=7.81e-7]
+
Train step of epoch 0:  84%|████████▍ | 4742/5616 [1:00:21<10:53,  1.34it/s, gpt_loss=0.00272, lr=7.81e-7]
+
Train step of epoch 0:  84%|████████▍ | 4742/5616 [1:00:22<10:53,  1.34it/s, gpt_loss=0.0859, lr=7.81e-7] 
+
Train step of epoch 0:  84%|████████▍ | 4743/5616 [1:00:22<11:16,  1.29it/s, gpt_loss=0.0859, lr=7.81e-7]
+
Train step of epoch 0:  84%|████████▍ | 4743/5616 [1:00:23<11:16,  1.29it/s, gpt_loss=0.00444, lr=7.79e-7]
+
Train step of epoch 0:  84%|████████▍ | 4744/5616 [1:00:23<11:18,  1.28it/s, gpt_loss=0.00444, lr=7.79e-7]
+
Train step of epoch 0:  84%|████████▍ | 4744/5616 [1:00:24<11:18,  1.28it/s, gpt_loss=0.00622, lr=7.79e-7]
+
Train step of epoch 0:  84%|████████▍ | 4745/5616 [1:00:24<10:58,  1.32it/s, gpt_loss=0.00622, lr=7.79e-7]
+
Train step of epoch 0:  84%|████████▍ | 4745/5616 [1:00:24<10:58,  1.32it/s, gpt_loss=0.0109, lr=7.79e-7] 
+
Train step of epoch 0:  85%|████████▍ | 4746/5616 [1:00:24<10:31,  1.38it/s, gpt_loss=0.0109, lr=7.79e-7]
+
Train step of epoch 0:  85%|████████▍ | 4746/5616 [1:00:25<10:31,  1.38it/s, gpt_loss=0.0083, lr=7.79e-7]
+
Train step of epoch 0:  85%|████████▍ | 4747/5616 [1:00:25<10:47,  1.34it/s, gpt_loss=0.0083, lr=7.79e-7]
+
Train step of epoch 0:  85%|████████▍ | 4747/5616 [1:00:26<10:47,  1.34it/s, gpt_loss=0.0276, lr=7.76e-7]
+
Train step of epoch 0:  85%|████████▍ | 4748/5616 [1:00:26<11:16,  1.28it/s, gpt_loss=0.0276, lr=7.76e-7]
+
Train step of epoch 0:  85%|████████▍ | 4748/5616 [1:00:27<11:16,  1.28it/s, gpt_loss=0.0794, lr=7.76e-7]
+
Train step of epoch 0:  85%|████████▍ | 4749/5616 [1:00:27<11:09,  1.30it/s, gpt_loss=0.0794, lr=7.76e-7]
+
Train step of epoch 0:  85%|████████▍ | 4749/5616 [1:00:27<11:09,  1.30it/s, gpt_loss=0.0413, lr=7.76e-7]
+
Train step of epoch 0:  85%|████████▍ | 4750/5616 [1:00:27<10:36,  1.36it/s, gpt_loss=0.0413, lr=7.76e-7]
+
Train step of epoch 0:  85%|████████▍ | 4750/5616 [1:00:28<10:36,  1.36it/s, gpt_loss=0.00812, lr=7.76e-7]
+
Train step of epoch 0:  85%|████████▍ | 4751/5616 [1:00:28<11:04,  1.30it/s, gpt_loss=0.00812, lr=7.76e-7]
+
Train step of epoch 0:  85%|████████▍ | 4751/5616 [1:00:29<11:04,  1.30it/s, gpt_loss=0.0923, lr=7.74e-7] 
+
Train step of epoch 0:  85%|████████▍ | 4752/5616 [1:00:29<11:00,  1.31it/s, gpt_loss=0.0923, lr=7.74e-7]
+
Train step of epoch 0:  85%|████████▍ | 4752/5616 [1:00:30<11:00,  1.31it/s, gpt_loss=0.0198, lr=7.74e-7]
+
Train step of epoch 0:  85%|████████▍ | 4753/5616 [1:00:30<10:44,  1.34it/s, gpt_loss=0.0198, lr=7.74e-7]
+
Train step of epoch 0:  85%|████████▍ | 4753/5616 [1:00:31<10:44,  1.34it/s, gpt_loss=0.0928, lr=7.74e-7]
+
Train step of epoch 0:  85%|████████▍ | 4754/5616 [1:00:31<10:51,  1.32it/s, gpt_loss=0.0928, lr=7.74e-7]
+
Train step of epoch 0:  85%|████████▍ | 4754/5616 [1:00:31<10:51,  1.32it/s, gpt_loss=0.00618, lr=7.74e-7]
+
Train step of epoch 0:  85%|████████▍ | 4755/5616 [1:00:31<10:48,  1.33it/s, gpt_loss=0.00618, lr=7.74e-7]
+
Train step of epoch 0:  85%|████████▍ | 4755/5616 [1:00:32<10:48,  1.33it/s, gpt_loss=0.0122, lr=7.71e-7] 
+
Train step of epoch 0:  85%|████████▍ | 4756/5616 [1:00:32<11:44,  1.22it/s, gpt_loss=0.0122, lr=7.71e-7]
+
Train step of epoch 0:  85%|████████▍ | 4756/5616 [1:00:33<11:44,  1.22it/s, gpt_loss=0.0107, lr=7.71e-7]
+
Train step of epoch 0:  85%|████████▍ | 4757/5616 [1:00:33<13:03,  1.10it/s, gpt_loss=0.0107, lr=7.71e-7]
+
Train step of epoch 0:  85%|████████▍ | 4757/5616 [1:00:34<13:03,  1.10it/s, gpt_loss=0.0593, lr=7.71e-7]
+
Train step of epoch 0:  85%|████████▍ | 4758/5616 [1:00:34<12:01,  1.19it/s, gpt_loss=0.0593, lr=7.71e-7]
+
Train step of epoch 0:  85%|████████▍ | 4758/5616 [1:00:35<12:01,  1.19it/s, gpt_loss=0.0302, lr=7.71e-7]
+
Train step of epoch 0:  85%|████████▍ | 4759/5616 [1:00:35<11:18,  1.26it/s, gpt_loss=0.0302, lr=7.71e-7]
+
Train step of epoch 0:  85%|████████▍ | 4759/5616 [1:00:36<11:18,  1.26it/s, gpt_loss=0.00513, lr=7.69e-7]
+
Train step of epoch 0:  85%|████████▍ | 4760/5616 [1:00:36<12:00,  1.19it/s, gpt_loss=0.00513, lr=7.69e-7]
+
Train step of epoch 0:  85%|████████▍ | 4760/5616 [1:00:36<12:00,  1.19it/s, gpt_loss=0.0195, lr=7.69e-7] 
+
Train step of epoch 0:  85%|████████▍ | 4761/5616 [1:00:36<11:24,  1.25it/s, gpt_loss=0.0195, lr=7.69e-7]
+
Train step of epoch 0:  85%|████████▍ | 4761/5616 [1:00:37<11:24,  1.25it/s, gpt_loss=0.0619, lr=7.69e-7]
+
Train step of epoch 0:  85%|████████▍ | 4762/5616 [1:00:37<10:50,  1.31it/s, gpt_loss=0.0619, lr=7.69e-7]
+
Train step of epoch 0:  85%|████████▍ | 4762/5616 [1:00:38<10:50,  1.31it/s, gpt_loss=0.0222, lr=7.69e-7]
+
Train step of epoch 0:  85%|████████▍ | 4763/5616 [1:00:38<10:34,  1.34it/s, gpt_loss=0.0222, lr=7.69e-7]
+
Train step of epoch 0:  85%|████████▍ | 4763/5616 [1:00:39<10:34,  1.34it/s, gpt_loss=0.0022, lr=7.67e-7]
+
Train step of epoch 0:  85%|████████▍ | 4764/5616 [1:00:39<10:55,  1.30it/s, gpt_loss=0.0022, lr=7.67e-7]
+
Train step of epoch 0:  85%|████████▍ | 4764/5616 [1:00:39<10:55,  1.30it/s, gpt_loss=0.0134, lr=7.67e-7]
+
Train step of epoch 0:  85%|████████▍ | 4765/5616 [1:00:39<10:56,  1.30it/s, gpt_loss=0.0134, lr=7.67e-7]
+
Train step of epoch 0:  85%|████████▍ | 4765/5616 [1:00:40<10:56,  1.30it/s, gpt_loss=0.0217, lr=7.67e-7]
+
Train step of epoch 0:  85%|████████▍ | 4766/5616 [1:00:40<11:02,  1.28it/s, gpt_loss=0.0217, lr=7.67e-7]
+
Train step of epoch 0:  85%|████████▍ | 4766/5616 [1:00:41<11:02,  1.28it/s, gpt_loss=0.0259, lr=7.67e-7]
+
Train step of epoch 0:  85%|████████▍ | 4767/5616 [1:00:41<11:12,  1.26it/s, gpt_loss=0.0259, lr=7.67e-7]
+
Train step of epoch 0:  85%|████████▍ | 4767/5616 [1:00:42<11:12,  1.26it/s, gpt_loss=0.0155, lr=7.64e-7]
+
Train step of epoch 0:  85%|████████▍ | 4768/5616 [1:00:42<11:14,  1.26it/s, gpt_loss=0.0155, lr=7.64e-7]
+
Train step of epoch 0:  85%|████████▍ | 4768/5616 [1:00:43<11:14,  1.26it/s, gpt_loss=0.0322, lr=7.64e-7]
+
Train step of epoch 0:  85%|████████▍ | 4769/5616 [1:00:43<11:02,  1.28it/s, gpt_loss=0.0322, lr=7.64e-7]
+
Train step of epoch 0:  85%|████████▍ | 4769/5616 [1:00:43<11:02,  1.28it/s, gpt_loss=0.0138, lr=7.64e-7]
+
Train step of epoch 0:  85%|████████▍ | 4770/5616 [1:00:43<10:45,  1.31it/s, gpt_loss=0.0138, lr=7.64e-7]
+
Train step of epoch 0:  85%|████████▍ | 4770/5616 [1:00:44<10:45,  1.31it/s, gpt_loss=0.0714, lr=7.64e-7]
+
Train step of epoch 0:  85%|████████▍ | 4771/5616 [1:00:44<10:48,  1.30it/s, gpt_loss=0.0714, lr=7.64e-7]
+
Train step of epoch 0:  85%|████████▍ | 4771/5616 [1:00:45<10:48,  1.30it/s, gpt_loss=0.0154, lr=7.62e-7]
+
Train step of epoch 0:  85%|████████▍ | 4772/5616 [1:00:45<10:40,  1.32it/s, gpt_loss=0.0154, lr=7.62e-7]
+
Train step of epoch 0:  85%|████████▍ | 4772/5616 [1:00:45<10:40,  1.32it/s, gpt_loss=0.0078, lr=7.62e-7]
+
Train step of epoch 0:  85%|████████▍ | 4773/5616 [1:00:45<10:28,  1.34it/s, gpt_loss=0.0078, lr=7.62e-7]
+
Train step of epoch 0:  85%|████████▍ | 4773/5616 [1:00:46<10:28,  1.34it/s, gpt_loss=0.0104, lr=7.62e-7]
+
Train step of epoch 0:  85%|████████▌ | 4774/5616 [1:00:46<10:15,  1.37it/s, gpt_loss=0.0104, lr=7.62e-7]
+
Train step of epoch 0:  85%|████████▌ | 4774/5616 [1:00:47<10:15,  1.37it/s, gpt_loss=0.00797, lr=7.62e-7]
+
Train step of epoch 0:  85%|████████▌ | 4775/5616 [1:00:47<10:03,  1.39it/s, gpt_loss=0.00797, lr=7.62e-7]
+
Train step of epoch 0:  85%|████████▌ | 4775/5616 [1:00:48<10:03,  1.39it/s, gpt_loss=0.0273, lr=7.59e-7] 
+
Train step of epoch 0:  85%|████████▌ | 4776/5616 [1:00:48<10:15,  1.36it/s, gpt_loss=0.0273, lr=7.59e-7]
+
Train step of epoch 0:  85%|████████▌ | 4776/5616 [1:00:49<10:15,  1.36it/s, gpt_loss=0.00214, lr=7.59e-7]
+
Train step of epoch 0:  85%|████████▌ | 4777/5616 [1:00:49<10:45,  1.30it/s, gpt_loss=0.00214, lr=7.59e-7]
+
Train step of epoch 0:  85%|████████▌ | 4777/5616 [1:00:49<10:45,  1.30it/s, gpt_loss=0.00226, lr=7.59e-7]
+
Train step of epoch 0:  85%|████████▌ | 4778/5616 [1:00:49<10:25,  1.34it/s, gpt_loss=0.00226, lr=7.59e-7]
+
Train step of epoch 0:  85%|████████▌ | 4778/5616 [1:00:50<10:25,  1.34it/s, gpt_loss=0.0311, lr=7.59e-7] 
+
Train step of epoch 0:  85%|████████▌ | 4779/5616 [1:00:50<10:17,  1.36it/s, gpt_loss=0.0311, lr=7.59e-7]
+
Train step of epoch 0:  85%|████████▌ | 4779/5616 [1:00:51<10:17,  1.36it/s, gpt_loss=0.13, lr=7.57e-7]  
+
Train step of epoch 0:  85%|████████▌ | 4780/5616 [1:00:51<10:22,  1.34it/s, gpt_loss=0.13, lr=7.57e-7]
+
Train step of epoch 0:  85%|████████▌ | 4780/5616 [1:00:51<10:22,  1.34it/s, gpt_loss=0.0374, lr=7.57e-7]
+
Train step of epoch 0:  85%|████████▌ | 4781/5616 [1:00:51<10:18,  1.35it/s, gpt_loss=0.0374, lr=7.57e-7]
+
Train step of epoch 0:  85%|████████▌ | 4781/5616 [1:00:52<10:18,  1.35it/s, gpt_loss=0.0147, lr=7.57e-7]
+
Train step of epoch 0:  85%|████████▌ | 4782/5616 [1:00:52<10:19,  1.35it/s, gpt_loss=0.0147, lr=7.57e-7]
+
Train step of epoch 0:  85%|████████▌ | 4782/5616 [1:00:53<10:19,  1.35it/s, gpt_loss=0.0333, lr=7.57e-7]
+
Train step of epoch 0:  85%|████████▌ | 4783/5616 [1:00:53<10:06,  1.37it/s, gpt_loss=0.0333, lr=7.57e-7]
+
Train step of epoch 0:  85%|████████▌ | 4783/5616 [1:00:54<10:06,  1.37it/s, gpt_loss=0.0523, lr=7.54e-7]
+
Train step of epoch 0:  85%|████████▌ | 4784/5616 [1:00:54<10:19,  1.34it/s, gpt_loss=0.0523, lr=7.54e-7]
+
Train step of epoch 0:  85%|████████▌ | 4784/5616 [1:00:54<10:19,  1.34it/s, gpt_loss=0.02, lr=7.54e-7]  
+
Train step of epoch 0:  85%|████████▌ | 4785/5616 [1:00:54<10:36,  1.30it/s, gpt_loss=0.02, lr=7.54e-7]
+
Train step of epoch 0:  85%|████████▌ | 4785/5616 [1:00:55<10:36,  1.30it/s, gpt_loss=0.0032, lr=7.54e-7]
+
Train step of epoch 0:  85%|████████▌ | 4786/5616 [1:00:55<10:42,  1.29it/s, gpt_loss=0.0032, lr=7.54e-7]
+
Train step of epoch 0:  85%|████████▌ | 4786/5616 [1:00:56<10:42,  1.29it/s, gpt_loss=0.0379, lr=7.54e-7]
+
Train step of epoch 0:  85%|████████▌ | 4787/5616 [1:00:56<10:09,  1.36it/s, gpt_loss=0.0379, lr=7.54e-7]
+
Train step of epoch 0:  85%|████████▌ | 4787/5616 [1:00:57<10:09,  1.36it/s, gpt_loss=0.0935, lr=7.52e-7]
+
Train step of epoch 0:  85%|████████▌ | 4788/5616 [1:00:57<10:21,  1.33it/s, gpt_loss=0.0935, lr=7.52e-7]
+
Train step of epoch 0:  85%|████████▌ | 4788/5616 [1:00:57<10:21,  1.33it/s, gpt_loss=0.0049, lr=7.52e-7]
+
Train step of epoch 0:  85%|████████▌ | 4789/5616 [1:00:57<09:57,  1.38it/s, gpt_loss=0.0049, lr=7.52e-7]
+
Train step of epoch 0:  85%|████████▌ | 4789/5616 [1:00:58<09:57,  1.38it/s, gpt_loss=0.0152, lr=7.52e-7]
+
Train step of epoch 0:  85%|████████▌ | 4790/5616 [1:00:58<09:58,  1.38it/s, gpt_loss=0.0152, lr=7.52e-7]
+
Train step of epoch 0:  85%|████████▌ | 4790/5616 [1:00:59<09:58,  1.38it/s, gpt_loss=0.0114, lr=7.52e-7]
+
Train step of epoch 0:  85%|████████▌ | 4791/5616 [1:00:59<09:35,  1.43it/s, gpt_loss=0.0114, lr=7.52e-7]
+
Train step of epoch 0:  85%|████████▌ | 4791/5616 [1:01:00<09:35,  1.43it/s, gpt_loss=0.0104, lr=7.5e-7] 
+
Train step of epoch 0:  85%|████████▌ | 4792/5616 [1:01:00<10:10,  1.35it/s, gpt_loss=0.0104, lr=7.5e-7]
+
Train step of epoch 0:  85%|████████▌ | 4792/5616 [1:01:00<10:10,  1.35it/s, gpt_loss=0.00273, lr=7.5e-7]
+
Train step of epoch 0:  85%|████████▌ | 4793/5616 [1:01:00<10:02,  1.37it/s, gpt_loss=0.00273, lr=7.5e-7]
+
Train step of epoch 0:  85%|████████▌ | 4793/5616 [1:01:01<10:02,  1.37it/s, gpt_loss=0.00139, lr=7.5e-7]
+
Train step of epoch 0:  85%|████████▌ | 4794/5616 [1:01:01<10:09,  1.35it/s, gpt_loss=0.00139, lr=7.5e-7]
+
Train step of epoch 0:  85%|████████▌ | 4794/5616 [1:01:02<10:09,  1.35it/s, gpt_loss=0.0318, lr=7.5e-7] 
+
Train step of epoch 0:  85%|████████▌ | 4795/5616 [1:01:02<10:22,  1.32it/s, gpt_loss=0.0318, lr=7.5e-7]
+
Train step of epoch 0:  85%|████████▌ | 4795/5616 [1:01:03<10:22,  1.32it/s, gpt_loss=0.0114, lr=7.47e-7]
+
Train step of epoch 0:  85%|████████▌ | 4796/5616 [1:01:03<11:11,  1.22it/s, gpt_loss=0.0114, lr=7.47e-7]
+
Train step of epoch 0:  85%|████████▌ | 4796/5616 [1:01:03<11:11,  1.22it/s, gpt_loss=0.0166, lr=7.47e-7]
+
Train step of epoch 0:  85%|████████▌ | 4797/5616 [1:01:03<10:33,  1.29it/s, gpt_loss=0.0166, lr=7.47e-7]
+
Train step of epoch 0:  85%|████████▌ | 4797/5616 [1:01:04<10:33,  1.29it/s, gpt_loss=0.0146, lr=7.47e-7]
+
Train step of epoch 0:  85%|████████▌ | 4798/5616 [1:01:04<10:12,  1.34it/s, gpt_loss=0.0146, lr=7.47e-7]
+
Train step of epoch 0:  85%|████████▌ | 4798/5616 [1:01:05<10:12,  1.34it/s, gpt_loss=0.0125, lr=7.47e-7]
+
Train step of epoch 0:  85%|████████▌ | 4799/5616 [1:01:05<10:09,  1.34it/s, gpt_loss=0.0125, lr=7.47e-7][2025-06-30 07:39:14,668] [INFO] [logging.py:128:log_dist] [Rank 0] step=1200, skipped=0, lr=[7.448817592983713e-07, 7.448817592983713e-07], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 07:39:14,669] [INFO] [timer.py:264:stop] epoch=0/micro_step=4800/global_step=1200, RunningAvgSamplesPerSec=42.439344887933785, CurrSamplesPerSec=45.35052927486839, MemAllocated=15.92GB, MaxMemAllocated=54.86GB
+
+
Train step of epoch 0:  85%|████████▌ | 4799/5616 [1:01:06<10:09,  1.34it/s, gpt_loss=0.00835, lr=7.45e-7]
+
Train step of epoch 0:  85%|████████▌ | 4800/5616 [1:01:06<10:13,  1.33it/s, gpt_loss=0.00835, lr=7.45e-7]
+
Train step of epoch 0:  85%|████████▌ | 4800/5616 [1:01:06<10:13,  1.33it/s, gpt_loss=0.0512, lr=7.45e-7] 
+
Train step of epoch 0:  85%|████████▌ | 4801/5616 [1:01:06<10:02,  1.35it/s, gpt_loss=0.0512, lr=7.45e-7]
+
Train step of epoch 0:  85%|████████▌ | 4801/5616 [1:01:07<10:02,  1.35it/s, gpt_loss=0.0077, lr=7.45e-7]
+
Train step of epoch 0:  86%|████████▌ | 4802/5616 [1:01:07<09:47,  1.38it/s, gpt_loss=0.0077, lr=7.45e-7]
+
Train step of epoch 0:  86%|████████▌ | 4802/5616 [1:01:08<09:47,  1.38it/s, gpt_loss=0.00688, lr=7.45e-7]
+
Train step of epoch 0:  86%|████████▌ | 4803/5616 [1:01:08<09:24,  1.44it/s, gpt_loss=0.00688, lr=7.45e-7]
+
Train step of epoch 0:  86%|████████▌ | 4803/5616 [1:01:08<09:24,  1.44it/s, gpt_loss=0.00169, lr=7.43e-7]
+
Train step of epoch 0:  86%|████████▌ | 4804/5616 [1:01:08<09:52,  1.37it/s, gpt_loss=0.00169, lr=7.43e-7]
+
Train step of epoch 0:  86%|████████▌ | 4804/5616 [1:01:09<09:52,  1.37it/s, gpt_loss=0.0129, lr=7.43e-7] 
+
Train step of epoch 0:  86%|████████▌ | 4805/5616 [1:01:09<09:49,  1.38it/s, gpt_loss=0.0129, lr=7.43e-7]
+
Train step of epoch 0:  86%|████████▌ | 4805/5616 [1:01:10<09:49,  1.38it/s, gpt_loss=0.00775, lr=7.43e-7]
+
Train step of epoch 0:  86%|████████▌ | 4806/5616 [1:01:10<09:49,  1.38it/s, gpt_loss=0.00775, lr=7.43e-7]
+
Train step of epoch 0:  86%|████████▌ | 4806/5616 [1:01:11<09:49,  1.38it/s, gpt_loss=0.00258, lr=7.43e-7]
+
Train step of epoch 0:  86%|████████▌ | 4807/5616 [1:01:11<09:29,  1.42it/s, gpt_loss=0.00258, lr=7.43e-7]
+
Train step of epoch 0:  86%|████████▌ | 4807/5616 [1:01:11<09:29,  1.42it/s, gpt_loss=0.0233, lr=7.4e-7]  
+
Train step of epoch 0:  86%|████████▌ | 4808/5616 [1:01:11<09:35,  1.40it/s, gpt_loss=0.0233, lr=7.4e-7]
+
Train step of epoch 0:  86%|████████▌ | 4808/5616 [1:01:12<09:35,  1.40it/s, gpt_loss=0.00223, lr=7.4e-7]
+
Train step of epoch 0:  86%|████████▌ | 4809/5616 [1:01:12<09:18,  1.45it/s, gpt_loss=0.00223, lr=7.4e-7]
+
Train step of epoch 0:  86%|████████▌ | 4809/5616 [1:01:13<09:18,  1.45it/s, gpt_loss=0.053, lr=7.4e-7]  
+
Train step of epoch 0:  86%|████████▌ | 4810/5616 [1:01:13<09:43,  1.38it/s, gpt_loss=0.053, lr=7.4e-7]
+
Train step of epoch 0:  86%|████████▌ | 4810/5616 [1:01:14<09:43,  1.38it/s, gpt_loss=0.0548, lr=7.4e-7]
+
Train step of epoch 0:  86%|████████▌ | 4811/5616 [1:01:14<10:17,  1.30it/s, gpt_loss=0.0548, lr=7.4e-7]
+
Train step of epoch 0:  86%|████████▌ | 4811/5616 [1:01:14<10:17,  1.30it/s, gpt_loss=0.00605, lr=7.38e-7]
+
Train step of epoch 0:  86%|████████▌ | 4812/5616 [1:01:14<10:29,  1.28it/s, gpt_loss=0.00605, lr=7.38e-7]
+
Train step of epoch 0:  86%|████████▌ | 4812/5616 [1:01:15<10:29,  1.28it/s, gpt_loss=0.00948, lr=7.38e-7]
+
Train step of epoch 0:  86%|████████▌ | 4813/5616 [1:01:15<10:04,  1.33it/s, gpt_loss=0.00948, lr=7.38e-7]
+
Train step of epoch 0:  86%|████████▌ | 4813/5616 [1:01:16<10:04,  1.33it/s, gpt_loss=0.0335, lr=7.38e-7] 
+
Train step of epoch 0:  86%|████████▌ | 4814/5616 [1:01:16<09:51,  1.36it/s, gpt_loss=0.0335, lr=7.38e-7]
+
Train step of epoch 0:  86%|████████▌ | 4814/5616 [1:01:17<09:51,  1.36it/s, gpt_loss=0.00722, lr=7.38e-7]
+
Train step of epoch 0:  86%|████████▌ | 4815/5616 [1:01:17<10:14,  1.30it/s, gpt_loss=0.00722, lr=7.38e-7]
+
Train step of epoch 0:  86%|████████▌ | 4815/5616 [1:01:17<10:14,  1.30it/s, gpt_loss=0.0183, lr=7.36e-7] 
+
Train step of epoch 0:  86%|████████▌ | 4816/5616 [1:01:17<10:26,  1.28it/s, gpt_loss=0.0183, lr=7.36e-7]
+
Train step of epoch 0:  86%|████████▌ | 4816/5616 [1:01:18<10:26,  1.28it/s, gpt_loss=0.012, lr=7.36e-7] 
+
Train step of epoch 0:  86%|████████▌ | 4817/5616 [1:01:18<10:07,  1.32it/s, gpt_loss=0.012, lr=7.36e-7]
+
Train step of epoch 0:  86%|████████▌ | 4817/5616 [1:01:19<10:07,  1.32it/s, gpt_loss=0.0182, lr=7.36e-7]
+
Train step of epoch 0:  86%|████████▌ | 4818/5616 [1:01:19<10:08,  1.31it/s, gpt_loss=0.0182, lr=7.36e-7]
+
Train step of epoch 0:  86%|████████▌ | 4818/5616 [1:01:20<10:08,  1.31it/s, gpt_loss=0.0126, lr=7.36e-7]
+
Train step of epoch 0:  86%|████████▌ | 4819/5616 [1:01:20<09:39,  1.38it/s, gpt_loss=0.0126, lr=7.36e-7]
+
Train step of epoch 0:  86%|████████▌ | 4819/5616 [1:01:21<09:39,  1.38it/s, gpt_loss=0.0823, lr=7.33e-7]
+
Train step of epoch 0:  86%|████████▌ | 4820/5616 [1:01:21<10:32,  1.26it/s, gpt_loss=0.0823, lr=7.33e-7]
+
Train step of epoch 0:  86%|████████▌ | 4820/5616 [1:01:21<10:32,  1.26it/s, gpt_loss=0.0471, lr=7.33e-7]
+
Train step of epoch 0:  86%|████████▌ | 4821/5616 [1:01:21<10:07,  1.31it/s, gpt_loss=0.0471, lr=7.33e-7]
+
Train step of epoch 0:  86%|████████▌ | 4821/5616 [1:01:22<10:07,  1.31it/s, gpt_loss=0.0211, lr=7.33e-7]
+
Train step of epoch 0:  86%|████████▌ | 4822/5616 [1:01:22<10:02,  1.32it/s, gpt_loss=0.0211, lr=7.33e-7]
+
Train step of epoch 0:  86%|████████▌ | 4822/5616 [1:01:23<10:02,  1.32it/s, gpt_loss=0.0178, lr=7.33e-7]
+
Train step of epoch 0:  86%|████████▌ | 4823/5616 [1:01:23<09:41,  1.36it/s, gpt_loss=0.0178, lr=7.33e-7]
+
Train step of epoch 0:  86%|████████▌ | 4823/5616 [1:01:23<09:41,  1.36it/s, gpt_loss=0.0896, lr=7.31e-7]
+
Train step of epoch 0:  86%|████████▌ | 4824/5616 [1:01:23<10:06,  1.31it/s, gpt_loss=0.0896, lr=7.31e-7]
+
Train step of epoch 0:  86%|████████▌ | 4824/5616 [1:01:24<10:06,  1.31it/s, gpt_loss=0.0692, lr=7.31e-7]
+
Train step of epoch 0:  86%|████████▌ | 4825/5616 [1:01:24<09:49,  1.34it/s, gpt_loss=0.0692, lr=7.31e-7]
+
Train step of epoch 0:  86%|████████▌ | 4825/5616 [1:01:25<09:49,  1.34it/s, gpt_loss=0.00639, lr=7.31e-7]
+
Train step of epoch 0:  86%|██████���█▌ | 4826/5616 [1:01:25<09:39,  1.36it/s, gpt_loss=0.00639, lr=7.31e-7]
+
Train step of epoch 0:  86%|████████▌ | 4826/5616 [1:01:26<09:39,  1.36it/s, gpt_loss=0.0117, lr=7.31e-7] 
+
Train step of epoch 0:  86%|████████▌ | 4827/5616 [1:01:26<09:51,  1.33it/s, gpt_loss=0.0117, lr=7.31e-7]
+
Train step of epoch 0:  86%|████████▌ | 4827/5616 [1:01:27<09:51,  1.33it/s, gpt_loss=0.0482, lr=7.29e-7]
+
Train step of epoch 0:  86%|████████▌ | 4828/5616 [1:01:27<13:53,  1.06s/it, gpt_loss=0.0482, lr=7.29e-7]
+
Train step of epoch 0:  86%|████████▌ | 4828/5616 [1:01:28<13:53,  1.06s/it, gpt_loss=0.039, lr=7.29e-7] 
+
Train step of epoch 0:  86%|████████▌ | 4829/5616 [1:01:28<13:23,  1.02s/it, gpt_loss=0.039, lr=7.29e-7]
+
Train step of epoch 0:  86%|████████▌ | 4829/5616 [1:01:29<13:23,  1.02s/it, gpt_loss=0.00939, lr=7.29e-7]
+
Train step of epoch 0:  86%|████████▌ | 4830/5616 [1:01:29<11:52,  1.10it/s, gpt_loss=0.00939, lr=7.29e-7]
+
Train step of epoch 0:  86%|████████▌ | 4830/5616 [1:01:30<11:52,  1.10it/s, gpt_loss=0.0378, lr=7.29e-7] 
+
Train step of epoch 0:  86%|████████▌ | 4831/5616 [1:01:30<11:13,  1.17it/s, gpt_loss=0.0378, lr=7.29e-7]
+
Train step of epoch 0:  86%|████████▌ | 4831/5616 [1:01:31<11:13,  1.17it/s, gpt_loss=0.119, lr=7.26e-7] 
+
Train step of epoch 0:  86%|████████▌ | 4832/5616 [1:01:31<10:49,  1.21it/s, gpt_loss=0.119, lr=7.26e-7]
+
Train step of epoch 0:  86%|████████▌ | 4832/5616 [1:01:31<10:49,  1.21it/s, gpt_loss=0.0108, lr=7.26e-7]
+
Train step of epoch 0:  86%|████████▌ | 4833/5616 [1:01:31<10:24,  1.25it/s, gpt_loss=0.0108, lr=7.26e-7]
+
Train step of epoch 0:  86%|████████▌ | 4833/5616 [1:01:32<10:24,  1.25it/s, gpt_loss=0.0094, lr=7.26e-7]
+
Train step of epoch 0:  86%|████████▌ | 4834/5616 [1:01:32<10:02,  1.30it/s, gpt_loss=0.0094, lr=7.26e-7]
+
Train step of epoch 0:  86%|████████▌ | 4834/5616 [1:01:33<10:02,  1.30it/s, gpt_loss=0.00756, lr=7.26e-7]
+
Train step of epoch 0:  86%|████████▌ | 4835/5616 [1:01:33<11:09,  1.17it/s, gpt_loss=0.00756, lr=7.26e-7]
+
Train step of epoch 0:  86%|████████▌ | 4835/5616 [1:01:34<11:09,  1.17it/s, gpt_loss=0.0131, lr=7.24e-7] 
+
Train step of epoch 0:  86%|████████▌ | 4836/5616 [1:01:34<11:00,  1.18it/s, gpt_loss=0.0131, lr=7.24e-7]
+
Train step of epoch 0:  86%|████████▌ | 4836/5616 [1:01:35<11:00,  1.18it/s, gpt_loss=0.00257, lr=7.24e-7]
+
Train step of epoch 0:  86%|████████▌ | 4837/5616 [1:01:35<10:31,  1.23it/s, gpt_loss=0.00257, lr=7.24e-7]
+
Train step of epoch 0:  86%|████████▌ | 4837/5616 [1:01:35<10:31,  1.23it/s, gpt_loss=0.0116, lr=7.24e-7] 
+
Train step of epoch 0:  86%|████████▌ | 4838/5616 [1:01:35<09:48,  1.32it/s, gpt_loss=0.0116, lr=7.24e-7]
+
Train step of epoch 0:  86%|████████▌ | 4838/5616 [1:01:36<09:48,  1.32it/s, gpt_loss=0.0322, lr=7.24e-7]
+
Train step of epoch 0:  86%|████████▌ | 4839/5616 [1:01:36<09:36,  1.35it/s, gpt_loss=0.0322, lr=7.24e-7]
+
Train step of epoch 0:  86%|████████▌ | 4839/5616 [1:01:37<09:36,  1.35it/s, gpt_loss=0.0308, lr=7.22e-7]
+
Train step of epoch 0:  86%|████████▌ | 4840/5616 [1:01:37<11:04,  1.17it/s, gpt_loss=0.0308, lr=7.22e-7]
+
Train step of epoch 0:  86%|████████▌ | 4840/5616 [1:01:38<11:04,  1.17it/s, gpt_loss=0.017, lr=7.22e-7] 
+
Train step of epoch 0:  86%|████████▌ | 4841/5616 [1:01:38<10:21,  1.25it/s, gpt_loss=0.017, lr=7.22e-7]
+
Train step of epoch 0:  86%|████████▌ | 4841/5616 [1:01:39<10:21,  1.25it/s, gpt_loss=0.0501, lr=7.22e-7]
+
Train step of epoch 0:  86%|████████▌ | 4842/5616 [1:01:39<10:17,  1.25it/s, gpt_loss=0.0501, lr=7.22e-7]
+
Train step of epoch 0:  86%|████████▌ | 4842/5616 [1:01:39<10:17,  1.25it/s, gpt_loss=0.0132, lr=7.22e-7]
+
Train step of epoch 0:  86%|████████▌ | 4843/5616 [1:01:39<10:33,  1.22it/s, gpt_loss=0.0132, lr=7.22e-7]
+
Train step of epoch 0:  86%|████████▌ | 4843/5616 [1:01:40<10:33,  1.22it/s, gpt_loss=0.0198, lr=7.2e-7] 
+
Train step of epoch 0:  86%|████████▋ | 4844/5616 [1:01:40<10:40,  1.21it/s, gpt_loss=0.0198, lr=7.2e-7]
+
Train step of epoch 0:  86%|████████▋ | 4844/5616 [1:01:41<10:40,  1.21it/s, gpt_loss=0.0637, lr=7.2e-7]
+
Train step of epoch 0:  86%|████████▋ | 4845/5616 [1:01:41<10:02,  1.28it/s, gpt_loss=0.0637, lr=7.2e-7]
+
Train step of epoch 0:  86%|████████▋ | 4845/5616 [1:01:42<10:02,  1.28it/s, gpt_loss=0.014, lr=7.2e-7] 
+
Train step of epoch 0:  86%|████████▋ | 4846/5616 [1:01:42<10:17,  1.25it/s, gpt_loss=0.014, lr=7.2e-7]
+
Train step of epoch 0:  86%|████████▋ | 4846/5616 [1:01:42<10:17,  1.25it/s, gpt_loss=0.0137, lr=7.2e-7]
+
Train step of epoch 0:  86%|████████▋ | 4847/5616 [1:01:42<10:01,  1.28it/s, gpt_loss=0.0137, lr=7.2e-7]
+
Train step of epoch 0:  86%|████████▋ | 4847/5616 [1:01:43<10:01,  1.28it/s, gpt_loss=0.0336, lr=7.17e-7]
+
Train step of epoch 0:  86%|████████▋ | 4848/5616 [1:01:43<09:59,  1.28it/s, gpt_loss=0.0336, lr=7.17e-7]
+
Train step of epoch 0:  86%|████████▋ | 4848/5616 [1:01:44<09:59,  1.28it/s, gpt_loss=0.0472, lr=7.17e-7]
+
Train step of epoch 0:  86%|████████▋ | 4849/5616 [1:01:44<09:28,  1.35it/s, gpt_loss=0.0472, lr=7.17e-7]
+
Train step of epoch 0:  86%|████████▋ | 4849/5616 [1:01:45<09:28,  1.35it/s, gpt_loss=0.0682, lr=7.17e-7]
+
Train step of epoch 0:  86%|████████▋ | 4850/5616 [1:01:45<10:02,  1.27it/s, gpt_loss=0.0682, lr=7.17e-7]
+
Train step of epoch 0:  86%|████████▋ | 4850/5616 [1:01:45<10:02,  1.27it/s, gpt_loss=0.0756, lr=7.17e-7]
+
Train step of epoch 0:  86%|████████▋ | 4851/5616 [1:01:45<09:39,  1.32it/s, gpt_loss=0.0756, lr=7.17e-7]
+
Train step of epoch 0:  86%|████████▋ | 4851/5616 [1:01:46<09:39,  1.32it/s, gpt_loss=0.0555, lr=7.15e-7]
+
Train step of epoch 0:  86%|████████▋ | 4852/5616 [1:01:46<09:54,  1.28it/s, gpt_loss=0.0555, lr=7.15e-7]
+
Train step of epoch 0:  86%|████████▋ | 4852/5616 [1:01:47<09:54,  1.28it/s, gpt_loss=0.0466, lr=7.15e-7]
+
Train step of epoch 0:  86%|████████▋ | 4853/5616 [1:01:47<10:06,  1.26it/s, gpt_loss=0.0466, lr=7.15e-7]
+
Train step of epoch 0:  86%|████████▋ | 4853/5616 [1:01:48<10:06,  1.26it/s, gpt_loss=0.00506, lr=7.15e-7]
+
Train step of epoch 0:  86%|████████▋ | 4854/5616 [1:01:48<09:43,  1.31it/s, gpt_loss=0.00506, lr=7.15e-7]
+
Train step of epoch 0:  86%|████████▋ | 4854/5616 [1:01:49<09:43,  1.31it/s, gpt_loss=0.0262, lr=7.15e-7] 
+
Train step of epoch 0:  86%|████████▋ | 4855/5616 [1:01:49<09:22,  1.35it/s, gpt_loss=0.0262, lr=7.15e-7]
+
Train step of epoch 0:  86%|████████▋ | 4855/5616 [1:01:49<09:22,  1.35it/s, gpt_loss=0.0321, lr=7.13e-7]
+
Train step of epoch 0:  86%|████████▋ | 4856/5616 [1:01:49<09:45,  1.30it/s, gpt_loss=0.0321, lr=7.13e-7]
+
Train step of epoch 0:  86%|████████▋ | 4856/5616 [1:01:50<09:45,  1.30it/s, gpt_loss=0.0655, lr=7.13e-7]
+
Train step of epoch 0:  86%|████████▋ | 4857/5616 [1:01:50<09:21,  1.35it/s, gpt_loss=0.0655, lr=7.13e-7]
+
Train step of epoch 0:  86%|████████▋ | 4857/5616 [1:01:51<09:21,  1.35it/s, gpt_loss=0.0153, lr=7.13e-7]
+
Train step of epoch 0:  87%|████████▋ | 4858/5616 [1:01:51<09:03,  1.39it/s, gpt_loss=0.0153, lr=7.13e-7]
+
Train step of epoch 0:  87%|████████▋ | 4858/5616 [1:01:52<09:03,  1.39it/s, gpt_loss=0.0136, lr=7.13e-7]
+
Train step of epoch 0:  87%|████████▋ | 4859/5616 [1:01:52<09:30,  1.33it/s, gpt_loss=0.0136, lr=7.13e-7]
+
Train step of epoch 0:  87%|████████▋ | 4859/5616 [1:01:52<09:30,  1.33it/s, gpt_loss=0.0174, lr=7.11e-7]
+
Train step of epoch 0:  87%|████████▋ | 4860/5616 [1:01:52<10:02,  1.26it/s, gpt_loss=0.0174, lr=7.11e-7]
+
Train step of epoch 0:  87%|████████▋ | 4860/5616 [1:01:53<10:02,  1.26it/s, gpt_loss=0.0658, lr=7.11e-7]
+
Train step of epoch 0:  87%|████████▋ | 4861/5616 [1:01:53<10:13,  1.23it/s, gpt_loss=0.0658, lr=7.11e-7]
+
Train step of epoch 0:  87%|████████▋ | 4861/5616 [1:01:54<10:13,  1.23it/s, gpt_loss=0.00637, lr=7.11e-7]
+
Train step of epoch 0:  87%|████████▋ | 4862/5616 [1:01:54<10:11,  1.23it/s, gpt_loss=0.00637, lr=7.11e-7]
+
Train step of epoch 0:  87%|████████▋ | 4862/5616 [1:01:55<10:11,  1.23it/s, gpt_loss=0.00597, lr=7.11e-7]
+
Train step of epoch 0:  87%|████████▋ | 4863/5616 [1:01:55<09:36,  1.31it/s, gpt_loss=0.00597, lr=7.11e-7]
+
Train step of epoch 0:  87%|████████▋ | 4863/5616 [1:01:56<09:36,  1.31it/s, gpt_loss=0.0473, lr=7.09e-7] 
+
Train step of epoch 0:  87%|████████▋ | 4864/5616 [1:01:56<11:16,  1.11it/s, gpt_loss=0.0473, lr=7.09e-7]
+
Train step of epoch 0:  87%|████████▋ | 4864/5616 [1:01:57<11:16,  1.11it/s, gpt_loss=0.0241, lr=7.09e-7]
+
Train step of epoch 0:  87%|████████▋ | 4865/5616 [1:01:57<11:10,  1.12it/s, gpt_loss=0.0241, lr=7.09e-7]
+
Train step of epoch 0:  87%|████████▋ | 4865/5616 [1:01:58<11:10,  1.12it/s, gpt_loss=0.0611, lr=7.09e-7]
+
Train step of epoch 0:  87%|████████▋ | 4866/5616 [1:01:58<10:22,  1.20it/s, gpt_loss=0.0611, lr=7.09e-7]
+
Train step of epoch 0:  87%|████████▋ | 4866/5616 [1:01:58<10:22,  1.20it/s, gpt_loss=0.0418, lr=7.09e-7]
+
Train step of epoch 0:  87%|████████▋ | 4867/5616 [1:01:58<09:53,  1.26it/s, gpt_loss=0.0418, lr=7.09e-7]
+
Train step of epoch 0:  87%|████████▋ | 4867/5616 [1:01:59<09:53,  1.26it/s, gpt_loss=0.00575, lr=7.06e-7]
+
Train step of epoch 0:  87%|████████▋ | 4868/5616 [1:01:59<09:55,  1.26it/s, gpt_loss=0.00575, lr=7.06e-7]
+
Train step of epoch 0:  87%|████████▋ | 4868/5616 [1:02:00<09:55,  1.26it/s, gpt_loss=0.0353, lr=7.06e-7] 
+
Train step of epoch 0:  87%|████████▋ | 4869/5616 [1:02:00<09:49,  1.27it/s, gpt_loss=0.0353, lr=7.06e-7]
+
Train step of epoch 0:  87%|████████▋ | 4869/5616 [1:02:00<09:49,  1.27it/s, gpt_loss=0.111, lr=7.06e-7] 
+
Train step of epoch 0:  87%|████████▋ | 4870/5616 [1:02:00<09:17,  1.34it/s, gpt_loss=0.111, lr=7.06e-7]
+
Train step of epoch 0:  87%|████████▋ | 4870/5616 [1:02:01<09:17,  1.34it/s, gpt_loss=0.0182, lr=7.06e-7]
+
Train step of epoch 0:  87%|████████▋ | 4871/5616 [1:02:01<09:06,  1.36it/s, gpt_loss=0.0182, lr=7.06e-7]
+
Train step of epoch 0:  87%|████████▋ | 4871/5616 [1:02:02<09:06,  1.36it/s, gpt_loss=0.0143, lr=7.04e-7]
+
Train step of epoch 0:  87%|████████▋ | 4872/5616 [1:02:02<09:12,  1.35it/s, gpt_loss=0.0143, lr=7.04e-7]
+
Train step of epoch 0:  87%|████████▋ | 4872/5616 [1:02:03<09:12,  1.35it/s, gpt_loss=0.0276, lr=7.04e-7]
+
Train step of epoch 0:  87%|████████▋ | 4873/5616 [1:02:03<09:08,  1.36it/s, gpt_loss=0.0276, lr=7.04e-7]
+
Train step of epoch 0:  87%|████████▋ | 4873/5616 [1:02:03<09:08,  1.36it/s, gpt_loss=0.0196, lr=7.04e-7]
+
Train step of epoch 0:  87%|████████▋ | 4874/5616 [1:02:03<09:16,  1.33it/s, gpt_loss=0.0196, lr=7.04e-7]
+
Train step of epoch 0:  87%|████████▋ | 4874/5616 [1:02:04<09:16,  1.33it/s, gpt_loss=0.00329, lr=7.04e-7]
+
Train step of epoch 0:  87%|████████▋ | 4875/5616 [1:02:04<09:00,  1.37it/s, gpt_loss=0.00329, lr=7.04e-7]
+
Train step of epoch 0:  87%|████████▋ | 4875/5616 [1:02:05<09:00,  1.37it/s, gpt_loss=0.0178, lr=7.02e-7] 
+
Train step of epoch 0:  87%|████████▋ | 4876/5616 [1:02:05<09:04,  1.36it/s, gpt_loss=0.0178, lr=7.02e-7]
+
Train step of epoch 0:  87%|████████▋ | 4876/5616 [1:02:06<09:04,  1.36it/s, gpt_loss=0.00152, lr=7.02e-7]
+
Train step of epoch 0:  87%|████████▋ | 4877/5616 [1:02:06<09:14,  1.33it/s, gpt_loss=0.00152, lr=7.02e-7]
+
Train step of epoch 0:  87%|████████▋ | 4877/5616 [1:02:06<09:14,  1.33it/s, gpt_loss=0.0704, lr=7.02e-7] 
+
Train step of epoch 0:  87%|████████▋ | 4878/5616 [1:02:06<09:03,  1.36it/s, gpt_loss=0.0704, lr=7.02e-7]
+
Train step of epoch 0:  87%|████████▋ | 4878/5616 [1:02:07<09:03,  1.36it/s, gpt_loss=0.0112, lr=7.02e-7]
+
Train step of epoch 0:  87%|████████▋ | 4879/5616 [1:02:07<09:06,  1.35it/s, gpt_loss=0.0112, lr=7.02e-7]
+
Train step of epoch 0:  87%|████████▋ | 4879/5616 [1:02:08<09:06,  1.35it/s, gpt_loss=0.0309, lr=7e-7]   
+
Train step of epoch 0:  87%|████████▋ | 4880/5616 [1:02:08<09:24,  1.30it/s, gpt_loss=0.0309, lr=7e-7]
+
Train step of epoch 0:  87%|████████▋ | 4880/5616 [1:02:09<09:24,  1.30it/s, gpt_loss=0.0314, lr=7e-7]
+
Train step of epoch 0:  87%|████████▋ | 4881/5616 [1:02:09<09:06,  1.34it/s, gpt_loss=0.0314, lr=7e-7]
+
Train step of epoch 0:  87%|████████▋ | 4881/5616 [1:02:09<09:06,  1.34it/s, gpt_loss=0.00222, lr=7e-7]
+
Train step of epoch 0:  87%|████████▋ | 4882/5616 [1:02:09<08:45,  1.40it/s, gpt_loss=0.00222, lr=7e-7]
+
Train step of epoch 0:  87%|████████▋ | 4882/5616 [1:02:10<08:45,  1.40it/s, gpt_loss=0.0384, lr=7e-7] 
+
Train step of epoch 0:  87%|████████▋ | 4883/5616 [1:02:10<08:42,  1.40it/s, gpt_loss=0.0384, lr=7e-7]
+
Train step of epoch 0:  87%|████████▋ | 4883/5616 [1:02:11<08:42,  1.40it/s, gpt_loss=0.0132, lr=6.98e-7]
+
Train step of epoch 0:  87%|████████▋ | 4884/5616 [1:02:11<09:00,  1.35it/s, gpt_loss=0.0132, lr=6.98e-7]
+
Train step of epoch 0:  87%|████████▋ | 4884/5616 [1:02:11<09:00,  1.35it/s, gpt_loss=0.0154, lr=6.98e-7]
+
Train step of epoch 0:  87%|████████▋ | 4885/5616 [1:02:11<08:47,  1.38it/s, gpt_loss=0.0154, lr=6.98e-7]
+
Train step of epoch 0:  87%|████████▋ | 4885/5616 [1:02:12<08:47,  1.38it/s, gpt_loss=0.00135, lr=6.98e-7]
+
Train step of epoch 0:  87%|████████▋ | 4886/5616 [1:02:12<08:34,  1.42it/s, gpt_loss=0.00135, lr=6.98e-7]
+
Train step of epoch 0:  87%|████████▋ | 4886/5616 [1:02:13<08:34,  1.42it/s, gpt_loss=0.067, lr=6.98e-7]  
+
Train step of epoch 0:  87%|████████▋ | 4887/5616 [1:02:13<08:32,  1.42it/s, gpt_loss=0.067, lr=6.98e-7]
+
Train step of epoch 0:  87%|████████▋ | 4887/5616 [1:02:14<08:32,  1.42it/s, gpt_loss=0.0213, lr=6.96e-7]
+
Train step of epoch 0:  87%|████████▋ | 4888/5616 [1:02:14<08:46,  1.38it/s, gpt_loss=0.0213, lr=6.96e-7]
+
Train step of epoch 0:  87%|████████▋ | 4888/5616 [1:02:14<08:46,  1.38it/s, gpt_loss=0.000699, lr=6.96e-7]
+
Train step of epoch 0:  87%|████████▋ | 4889/5616 [1:02:14<09:12,  1.32it/s, gpt_loss=0.000699, lr=6.96e-7]
+
Train step of epoch 0:  87%|████████▋ | 4889/5616 [1:02:15<09:12,  1.32it/s, gpt_loss=0.0229, lr=6.96e-7]  
+
Train step of epoch 0:  87%|████████▋ | 4890/5616 [1:02:15<09:09,  1.32it/s, gpt_loss=0.0229, lr=6.96e-7]
+
Train step of epoch 0:  87%|████████▋ | 4890/5616 [1:02:16<09:09,  1.32it/s, gpt_loss=0.0301, lr=6.96e-7]
+
Train step of epoch 0:  87%|████████▋ | 4891/5616 [1:02:16<08:54,  1.36it/s, gpt_loss=0.0301, lr=6.96e-7]
+
Train step of epoch 0:  87%|████████▋ | 4891/5616 [1:02:17<08:54,  1.36it/s, gpt_loss=0.0441, lr=6.94e-7]
+
Train step of epoch 0:  87%|████████▋ | 4892/5616 [1:02:17<09:05,  1.33it/s, gpt_loss=0.0441, lr=6.94e-7]
+
Train step of epoch 0:  87%|████████▋ | 4892/5616 [1:02:17<09:05,  1.33it/s, gpt_loss=0.00325, lr=6.94e-7]
+
Train step of epoch 0:  87%|████████▋ | 4893/5616 [1:02:17<08:49,  1.37it/s, gpt_loss=0.00325, lr=6.94e-7]
+
Train step of epoch 0:  87%|████████▋ | 4893/5616 [1:02:18<08:49,  1.37it/s, gpt_loss=0.019, lr=6.94e-7]  
+
Train step of epoch 0:  87%|████████▋ | 4894/5616 [1:02:18<09:15,  1.30it/s, gpt_loss=0.019, lr=6.94e-7]
+
Train step of epoch 0:  87%|████████▋ | 4894/5616 [1:02:19<09:15,  1.30it/s, gpt_loss=0.00959, lr=6.94e-7]
+
Train step of epoch 0:  87%|████████▋ | 4895/5616 [1:02:19<09:07,  1.32it/s, gpt_loss=0.00959, lr=6.94e-7]
+
Train step of epoch 0:  87%|████████▋ | 4895/5616 [1:02:20<09:07,  1.32it/s, gpt_loss=0.00549, lr=6.91e-7]
+
Train step of epoch 0:  87%|████████▋ | 4896/5616 [1:02:20<09:32,  1.26it/s, gpt_loss=0.00549, lr=6.91e-7]
+
Train step of epoch 0:  87%|████████▋ | 4896/5616 [1:02:21<09:32,  1.26it/s, gpt_loss=0.0406, lr=6.91e-7] 
+
Train step of epoch 0:  87%|████████▋ | 4897/5616 [1:02:21<09:30,  1.26it/s, gpt_loss=0.0406, lr=6.91e-7]
+
Train step of epoch 0:  87%|████████▋ | 4897/5616 [1:02:21<09:30,  1.26it/s, gpt_loss=0.113, lr=6.91e-7] 
+
Train step of epoch 0:  87%|████████▋ | 4898/5616 [1:02:21<09:19,  1.28it/s, gpt_loss=0.113, lr=6.91e-7]
+
Train step of epoch 0:  87%|████████▋ | 4898/5616 [1:02:22<09:19,  1.28it/s, gpt_loss=0.0458, lr=6.91e-7]
+
Train step of epoch 0:  87%|████████▋ | 4899/5616 [1:02:22<09:12,  1.30it/s, gpt_loss=0.0458, lr=6.91e-7]
+
Train step of epoch 0:  87%|████████▋ | 4899/5616 [1:02:23<09:12,  1.30it/s, gpt_loss=0.00837, lr=6.89e-7]
+
Train step of epoch 0:  87%|████████▋ | 4900/5616 [1:02:23<09:30,  1.26it/s, gpt_loss=0.00837, lr=6.89e-7]
+
Train step of epoch 0:  87%|████████▋ | 4900/5616 [1:02:24<09:30,  1.26it/s, gpt_loss=0.00613, lr=6.89e-7]
+
Train step of epoch 0:  87%|████████▋ | 4901/5616 [1:02:24<09:05,  1.31it/s, gpt_loss=0.00613, lr=6.89e-7]
+
Train step of epoch 0:  87%|████████▋ | 4901/5616 [1:02:24<09:05,  1.31it/s, gpt_loss=0.0531, lr=6.89e-7] 
+
Train step of epoch 0:  87%|████████▋ | 4902/5616 [1:02:24<08:57,  1.33it/s, gpt_loss=0.0531, lr=6.89e-7]
+
Train step of epoch 0:  87%|████████▋ | 4902/5616 [1:02:25<08:57,  1.33it/s, gpt_loss=0.0307, lr=6.89e-7]
+
Train step of epoch 0:  87%|████████▋ | 4903/5616 [1:02:25<08:54,  1.33it/s, gpt_loss=0.0307, lr=6.89e-7]
+
Train step of epoch 0:  87%|████████▋ | 4903/5616 [1:02:26<08:54,  1.33it/s, gpt_loss=0.000983, lr=6.87e-7]
+
Train step of epoch 0:  87%|████████▋ | 4904/5616 [1:02:26<09:00,  1.32it/s, gpt_loss=0.000983, lr=6.87e-7]
+
Train step of epoch 0:  87%|████████▋ | 4904/5616 [1:02:27<09:00,  1.32it/s, gpt_loss=0.0784, lr=6.87e-7]  
+
Train step of epoch 0:  87%|████████▋ | 4905/5616 [1:02:27<08:44,  1.36it/s, gpt_loss=0.0784, lr=6.87e-7]
+
Train step of epoch 0:  87%|████████▋ | 4905/5616 [1:02:27<08:44,  1.36it/s, gpt_loss=0.000407, lr=6.87e-7]
+
Train step of epoch 0:  87%|████████▋ | 4906/5616 [1:02:27<08:40,  1.37it/s, gpt_loss=0.000407, lr=6.87e-7]
+
Train step of epoch 0:  87%|████████▋ | 4906/5616 [1:02:28<08:40,  1.37it/s, gpt_loss=0.0755, lr=6.87e-7]  
+
Train step of epoch 0:  87%|████████▋ | 4907/5616 [1:02:28<08:41,  1.36it/s, gpt_loss=0.0755, lr=6.87e-7]
+
Train step of epoch 0:  87%|████████▋ | 4907/5616 [1:02:29<08:41,  1.36it/s, gpt_loss=0.00129, lr=6.85e-7]
+
Train step of epoch 0:  87%|████████▋ | 4908/5616 [1:02:29<09:04,  1.30it/s, gpt_loss=0.00129, lr=6.85e-7]
+
Train step of epoch 0:  87%|████████▋ | 4908/5616 [1:02:30<09:04,  1.30it/s, gpt_loss=0.0668, lr=6.85e-7] 
+
Train step of epoch 0:  87%|████████▋ | 4909/5616 [1:02:30<09:24,  1.25it/s, gpt_loss=0.0668, lr=6.85e-7]
+
Train step of epoch 0:  87%|████████▋ | 4909/5616 [1:02:30<09:24,  1.25it/s, gpt_loss=0.00368, lr=6.85e-7]
+
Train step of epoch 0:  87%|████████▋ | 4910/5616 [1:02:30<09:07,  1.29it/s, gpt_loss=0.00368, lr=6.85e-7]
+
Train step of epoch 0:  87%|████████▋ | 4910/5616 [1:02:31<09:07,  1.29it/s, gpt_loss=0.0523, lr=6.85e-7] 
+
Train step of epoch 0:  87%|████████▋ | 4911/5616 [1:02:31<08:36,  1.37it/s, gpt_loss=0.0523, lr=6.85e-7]
+
Train step of epoch 0:  87%|████████▋ | 4911/5616 [1:02:32<08:36,  1.37it/s, gpt_loss=0.0314, lr=6.83e-7]
+
Train step of epoch 0:  87%|████████▋ | 4912/5616 [1:02:32<08:50,  1.33it/s, gpt_loss=0.0314, lr=6.83e-7]
+
Train step of epoch 0:  87%|████████▋ | 4912/5616 [1:02:33<08:50,  1.33it/s, gpt_loss=0.0668, lr=6.83e-7]
+
Train step of epoch 0:  87%|████████▋ | 4913/5616 [1:02:33<08:36,  1.36it/s, gpt_loss=0.0668, lr=6.83e-7]
+
Train step of epoch 0:  87%|████████▋ | 4913/5616 [1:02:33<08:36,  1.36it/s, gpt_loss=0.0332, lr=6.83e-7]
+
Train step of epoch 0:  88%|████████▊ | 4914/5616 [1:02:33<08:18,  1.41it/s, gpt_loss=0.0332, lr=6.83e-7]
+
Train step of epoch 0:  88%|████████▊ | 4914/5616 [1:02:34<08:18,  1.41it/s, gpt_loss=0.0349, lr=6.83e-7]
+
Train step of epoch 0:  88%|████████▊ | 4915/5616 [1:02:34<08:49,  1.32it/s, gpt_loss=0.0349, lr=6.83e-7]
+
Train step of epoch 0:  88%|████████▊ | 4915/5616 [1:02:35<08:49,  1.32it/s, gpt_loss=0.0287, lr=6.81e-7]
+
Train step of epoch 0:  88%|████████▊ | 4916/5616 [1:02:35<08:47,  1.33it/s, gpt_loss=0.0287, lr=6.81e-7]
+
Train step of epoch 0:  88%|████████▊ | 4916/5616 [1:02:36<08:47,  1.33it/s, gpt_loss=0.0194, lr=6.81e-7]
+
Train step of epoch 0:  88%|████████▊ | 4917/5616 [1:02:36<08:39,  1.35it/s, gpt_loss=0.0194, lr=6.81e-7]
+
Train step of epoch 0:  88%|████████▊ | 4917/5616 [1:02:36<08:39,  1.35it/s, gpt_loss=0.113, lr=6.81e-7] 
+
Train step of epoch 0:  88%|████████▊ | 4918/5616 [1:02:36<08:48,  1.32it/s, gpt_loss=0.113, lr=6.81e-7]
+
Train step of epoch 0:  88%|████████▊ | 4918/5616 [1:02:37<08:48,  1.32it/s, gpt_loss=0.0429, lr=6.81e-7]
+
Train step of epoch 0:  88%|████████▊ | 4919/5616 [1:02:37<08:31,  1.36it/s, gpt_loss=0.0429, lr=6.81e-7]
+
Train step of epoch 0:  88%|████████▊ | 4919/5616 [1:02:38<08:31,  1.36it/s, gpt_loss=0.0556, lr=6.79e-7]
+
Train step of epoch 0:  88%|████████▊ | 4920/5616 [1:02:38<08:34,  1.35it/s, gpt_loss=0.0556, lr=6.79e-7]
+
Train step of epoch 0:  88%|████████▊ | 4920/5616 [1:02:39<08:34,  1.35it/s, gpt_loss=0.0187, lr=6.79e-7]
+
Train step of epoch 0:  88%|████████▊ | 4921/5616 [1:02:39<08:22,  1.38it/s, gpt_loss=0.0187, lr=6.79e-7]
+
Train step of epoch 0:  88%|████████▊ | 4921/5616 [1:02:39<08:22,  1.38it/s, gpt_loss=0.00981, lr=6.79e-7]
+
Train step of epoch 0:  88%|████████▊ | 4922/5616 [1:02:39<08:04,  1.43it/s, gpt_loss=0.00981, lr=6.79e-7]
+
Train step of epoch 0:  88%|████████▊ | 4922/5616 [1:02:40<08:04,  1.43it/s, gpt_loss=0.00419, lr=6.79e-7]
+
Train step of epoch 0:  88%|████████▊ | 4923/5616 [1:02:40<07:50,  1.47it/s, gpt_loss=0.00419, lr=6.79e-7]
+
Train step of epoch 0:  88%|████████▊ | 4923/5616 [1:02:41<07:50,  1.47it/s, gpt_loss=0.0334, lr=6.77e-7] 
+
Train step of epoch 0:  88%|████████▊ | 4924/5616 [1:02:41<08:09,  1.41it/s, gpt_loss=0.0334, lr=6.77e-7]
+
Train step of epoch 0:  88%|████████▊ | 4924/5616 [1:02:41<08:09,  1.41it/s, gpt_loss=0.0166, lr=6.77e-7]
+
Train step of epoch 0:  88%|████████▊ | 4925/5616 [1:02:41<08:14,  1.40it/s, gpt_loss=0.0166, lr=6.77e-7]
+
Train step of epoch 0:  88%|████████▊ | 4925/5616 [1:02:42<08:14,  1.40it/s, gpt_loss=0.0723, lr=6.77e-7]
+
Train step of epoch 0:  88%|████████▊ | 4926/5616 [1:02:42<08:11,  1.40it/s, gpt_loss=0.0723, lr=6.77e-7]
+
Train step of epoch 0:  88%|████████▊ | 4926/5616 [1:02:43<08:11,  1.40it/s, gpt_loss=0.0113, lr=6.77e-7]
+
Train step of epoch 0:  88%|████████▊ | 4927/5616 [1:02:43<08:06,  1.42it/s, gpt_loss=0.0113, lr=6.77e-7]
+
Train step of epoch 0:  88%|████████▊ | 4927/5616 [1:02:43<08:06,  1.42it/s, gpt_loss=0.0149, lr=6.75e-7]
+
Train step of epoch 0:  88%|████████▊ | 4928/5616 [1:02:43<08:21,  1.37it/s, gpt_loss=0.0149, lr=6.75e-7]
+
Train step of epoch 0:  88%|████████▊ | 4928/5616 [1:02:44<08:21,  1.37it/s, gpt_loss=0.0215, lr=6.75e-7]
+
Train step of epoch 0:  88%|████████▊ | 4929/5616 [1:02:44<08:59,  1.27it/s, gpt_loss=0.0215, lr=6.75e-7]
+
Train step of epoch 0:  88%|████████▊ | 4929/5616 [1:02:45<08:59,  1.27it/s, gpt_loss=0.0569, lr=6.75e-7]
+
Train step of epoch 0:  88%|████████▊ | 4930/5616 [1:02:45<08:51,  1.29it/s, gpt_loss=0.0569, lr=6.75e-7]
+
Train step of epoch 0:  88%|████████▊ | 4930/5616 [1:02:46<08:51,  1.29it/s, gpt_loss=0.0388, lr=6.75e-7]
+
Train step of epoch 0:  88%|████████▊ | 4931/5616 [1:02:46<08:29,  1.35it/s, gpt_loss=0.0388, lr=6.75e-7]
+
Train step of epoch 0:  88%|████████▊ | 4931/5616 [1:02:47<08:29,  1.35it/s, gpt_loss=0.0357, lr=6.73e-7]
+
Train step of epoch 0:  88%|████████▊ | 4932/5616 [1:02:47<08:49,  1.29it/s, gpt_loss=0.0357, lr=6.73e-7]
+
Train step of epoch 0:  88%|████████▊ | 4932/5616 [1:02:48<08:49,  1.29it/s, gpt_loss=0.0048, lr=6.73e-7]
+
Train step of epoch 0:  88%|████████▊ | 4933/5616 [1:02:48<09:22,  1.21it/s, gpt_loss=0.0048, lr=6.73e-7]
+
Train step of epoch 0:  88%|████████▊ | 4933/5616 [1:02:48<09:22,  1.21it/s, gpt_loss=0.00108, lr=6.73e-7]
+
Train step of epoch 0:  88%|████████▊ | 4934/5616 [1:02:48<09:22,  1.21it/s, gpt_loss=0.00108, lr=6.73e-7]
+
Train step of epoch 0:  88%|████████▊ | 4934/5616 [1:02:49<09:22,  1.21it/s, gpt_loss=0.0452, lr=6.73e-7] 
+
Train step of epoch 0:  88%|████████▊ | 4935/5616 [1:02:49<09:26,  1.20it/s, gpt_loss=0.0452, lr=6.73e-7]
+
Train step of epoch 0:  88%|████████▊ | 4935/5616 [1:02:50<09:26,  1.20it/s, gpt_loss=0.0156, lr=6.71e-7]
+
Train step of epoch 0:  88%|████████▊ | 4936/5616 [1:02:50<09:33,  1.19it/s, gpt_loss=0.0156, lr=6.71e-7]
+
Train step of epoch 0:  88%|████████▊ | 4936/5616 [1:02:51<09:33,  1.19it/s, gpt_loss=0.0111, lr=6.71e-7]
+
Train step of epoch 0:  88%|████████▊ | 4937/5616 [1:02:51<09:27,  1.20it/s, gpt_loss=0.0111, lr=6.71e-7]
+
Train step of epoch 0:  88%|████████▊ | 4937/5616 [1:02:52<09:27,  1.20it/s, gpt_loss=0.062, lr=6.71e-7] 
+
Train step of epoch 0:  88%|████████▊ | 4938/5616 [1:02:52<09:33,  1.18it/s, gpt_loss=0.062, lr=6.71e-7]
+
Train step of epoch 0:  88%|████████▊ | 4938/5616 [1:02:52<09:33,  1.18it/s, gpt_loss=0.0261, lr=6.71e-7]
+
Train step of epoch 0:  88%|████████▊ | 4939/5616 [1:02:52<08:53,  1.27it/s, gpt_loss=0.0261, lr=6.71e-7]
+
Train step of epoch 0:  88%|████████▊ | 4939/5616 [1:02:53<08:53,  1.27it/s, gpt_loss=0.0181, lr=6.69e-7]
+
Train step of epoch 0:  88%|████████▊ | 4940/5616 [1:02:53<08:40,  1.30it/s, gpt_loss=0.0181, lr=6.69e-7]
+
Train step of epoch 0:  88%|████████▊ | 4940/5616 [1:02:54<08:40,  1.30it/s, gpt_loss=0.00179, lr=6.69e-7]
+
Train step of epoch 0:  88%|████████▊ | 4941/5616 [1:02:54<08:28,  1.33it/s, gpt_loss=0.00179, lr=6.69e-7]
+
Train step of epoch 0:  88%|████████▊ | 4941/5616 [1:02:55<08:28,  1.33it/s, gpt_loss=0.0232, lr=6.69e-7] 
+
Train step of epoch 0:  88%|████████▊ | 4942/5616 [1:02:55<08:23,  1.34it/s, gpt_loss=0.0232, lr=6.69e-7]
+
Train step of epoch 0:  88%|████████▊ | 4942/5616 [1:02:55<08:23,  1.34it/s, gpt_loss=0.0569, lr=6.69e-7]
+
Train step of epoch 0:  88%|████████▊ | 4943/5616 [1:02:55<08:29,  1.32it/s, gpt_loss=0.0569, lr=6.69e-7]
+
Train step of epoch 0:  88%|████████▊ | 4943/5616 [1:02:56<08:29,  1.32it/s, gpt_loss=0.00783, lr=6.67e-7]
+
Train step of epoch 0:  88%|████████▊ | 4944/5616 [1:02:56<08:33,  1.31it/s, gpt_loss=0.00783, lr=6.67e-7]
+
Train step of epoch 0:  88%|████████▊ | 4944/5616 [1:02:57<08:33,  1.31it/s, gpt_loss=0.0414, lr=6.67e-7] 
+
Train step of epoch 0:  88%|████████▊ | 4945/5616 [1:02:57<08:36,  1.30it/s, gpt_loss=0.0414, lr=6.67e-7]
+
Train step of epoch 0:  88%|████████▊ | 4945/5616 [1:02:58<08:36,  1.30it/s, gpt_loss=0.00491, lr=6.67e-7]
+
Train step of epoch 0:  88%|████████▊ | 4946/5616 [1:02:58<08:17,  1.35it/s, gpt_loss=0.00491, lr=6.67e-7]
+
Train step of epoch 0:  88%|████████▊ | 4946/5616 [1:02:58<08:17,  1.35it/s, gpt_loss=0.00563, lr=6.67e-7]
+
Train step of epoch 0:  88%|████████▊ | 4947/5616 [1:02:58<08:33,  1.30it/s, gpt_loss=0.00563, lr=6.67e-7]
+
Train step of epoch 0:  88%|████████▊ | 4947/5616 [1:03:00<08:33,  1.30it/s, gpt_loss=0.0888, lr=6.65e-7] 
+
Train step of epoch 0:  88%|████████▊ | 4948/5616 [1:03:00<10:24,  1.07it/s, gpt_loss=0.0888, lr=6.65e-7]
+
Train step of epoch 0:  88%|████████▊ | 4948/5616 [1:03:01<10:24,  1.07it/s, gpt_loss=0.0355, lr=6.65e-7]
+
Train step of epoch 0:  88%|████████▊ | 4949/5616 [1:03:01<09:46,  1.14it/s, gpt_loss=0.0355, lr=6.65e-7]
+
Train step of epoch 0:  88%|████████▊ | 4949/5616 [1:03:01<09:46,  1.14it/s, gpt_loss=0.0316, lr=6.65e-7]
+
Train step of epoch 0:  88%|████████▊ | 4950/5616 [1:03:01<09:08,  1.22it/s, gpt_loss=0.0316, lr=6.65e-7]
+
Train step of epoch 0:  88%|████████▊ | 4950/5616 [1:03:02<09:08,  1.22it/s, gpt_loss=0.00746, lr=6.65e-7]
+
Train step of epoch 0:  88%|████████▊ | 4951/5616 [1:03:02<10:00,  1.11it/s, gpt_loss=0.00746, lr=6.65e-7]
+
Train step of epoch 0:  88%|████████▊ | 4951/5616 [1:03:03<10:00,  1.11it/s, gpt_loss=0.0103, lr=6.63e-7] 
+
Train step of epoch 0:  88%|████████▊ | 4952/5616 [1:03:03<09:28,  1.17it/s, gpt_loss=0.0103, lr=6.63e-7]
+
Train step of epoch 0:  88%|████████▊ | 4952/5616 [1:03:04<09:28,  1.17it/s, gpt_loss=0.057, lr=6.63e-7] 
+
Train step of epoch 0:  88%|████████▊ | 4953/5616 [1:03:04<10:12,  1.08it/s, gpt_loss=0.057, lr=6.63e-7]
+
Train step of epoch 0:  88%|████████▊ | 4953/5616 [1:03:05<10:12,  1.08it/s, gpt_loss=0.0379, lr=6.63e-7]
+
Train step of epoch 0:  88%|████████▊ | 4954/5616 [1:03:05<09:27,  1.17it/s, gpt_loss=0.0379, lr=6.63e-7]
+
Train step of epoch 0:  88%|████████▊ | 4954/5616 [1:03:06<09:27,  1.17it/s, gpt_loss=0.00245, lr=6.63e-7]
+
Train step of epoch 0:  88%|████████▊ | 4955/5616 [1:03:06<08:52,  1.24it/s, gpt_loss=0.00245, lr=6.63e-7]
+
Train step of epoch 0:  88%|████████▊ | 4955/5616 [1:03:06<08:52,  1.24it/s, gpt_loss=0.0273, lr=6.61e-7] 
+
Train step of epoch 0:  88%|████████▊ | 4956/5616 [1:03:06<08:40,  1.27it/s, gpt_loss=0.0273, lr=6.61e-7]
+
Train step of epoch 0:  88%|████████▊ | 4956/5616 [1:03:07<08:40,  1.27it/s, gpt_loss=0.0623, lr=6.61e-7]
+
Train step of epoch 0:  88%|████████▊ | 4957/5616 [1:03:07<08:18,  1.32it/s, gpt_loss=0.0623, lr=6.61e-7]
+
Train step of epoch 0:  88%|████████▊ | 4957/5616 [1:03:08<08:18,  1.32it/s, gpt_loss=0.0122, lr=6.61e-7]
+
Train step of epoch 0:  88%|████████▊ | 4958/5616 [1:03:08<08:06,  1.35it/s, gpt_loss=0.0122, lr=6.61e-7]
+
Train step of epoch 0:  88%|████████▊ | 4958/5616 [1:03:08<08:06,  1.35it/s, gpt_loss=0.0368, lr=6.61e-7]
+
Train step of epoch 0:  88%|████████▊ | 4959/5616 [1:03:08<07:50,  1.40it/s, gpt_loss=0.0368, lr=6.61e-7]
+
Train step of epoch 0:  88%|████████▊ | 4959/5616 [1:03:09<07:50,  1.40it/s, gpt_loss=0.0441, lr=6.59e-7]
+
Train step of epoch 0:  88%|████████▊ | 4960/5616 [1:03:09<08:11,  1.33it/s, gpt_loss=0.0441, lr=6.59e-7]
+
Train step of epoch 0:  88%|████████▊ | 4960/5616 [1:03:10<08:11,  1.33it/s, gpt_loss=0.0373, lr=6.59e-7]
+
Train step of epoch 0:  88%|████████▊ | 4961/5616 [1:03:10<08:29,  1.29it/s, gpt_loss=0.0373, lr=6.59e-7]
+
Train step of epoch 0:  88%|████████▊ | 4961/5616 [1:03:11<08:29,  1.29it/s, gpt_loss=0.00938, lr=6.59e-7]
+
Train step of epoch 0:  88%|████████▊ | 4962/5616 [1:03:11<08:12,  1.33it/s, gpt_loss=0.00938, lr=6.59e-7]
+
Train step of epoch 0:  88%|████████▊ | 4962/5616 [1:03:11<08:12,  1.33it/s, gpt_loss=0.0436, lr=6.59e-7] 
+
Train step of epoch 0:  88%|████████▊ | 4963/5616 [1:03:11<07:54,  1.38it/s, gpt_loss=0.0436, lr=6.59e-7]
+
Train step of epoch 0:  88%|████████▊ | 4963/5616 [1:03:12<07:54,  1.38it/s, gpt_loss=0.022, lr=6.57e-7] 
+
Train step of epoch 0:  88%|████████▊ | 4964/5616 [1:03:12<08:26,  1.29it/s, gpt_loss=0.022, lr=6.57e-7]
+
Train step of epoch 0:  88%|████████▊ | 4964/5616 [1:03:13<08:26,  1.29it/s, gpt_loss=0.0442, lr=6.57e-7]
+
Train step of epoch 0:  88%|████████▊ | 4965/5616 [1:03:13<08:11,  1.32it/s, gpt_loss=0.0442, lr=6.57e-7]
+
Train step of epoch 0:  88%|████████▊ | 4965/5616 [1:03:14<08:11,  1.32it/s, gpt_loss=0.0117, lr=6.57e-7]
+
Train step of epoch 0:  88%|████████▊ | 4966/5616 [1:03:14<08:10,  1.32it/s, gpt_loss=0.0117, lr=6.57e-7]
+
Train step of epoch 0:  88%|████████▊ | 4966/5616 [1:03:14<08:10,  1.32it/s, gpt_loss=0.0146, lr=6.57e-7]
+
Train step of epoch 0:  88%|████████▊ | 4967/5616 [1:03:14<07:58,  1.36it/s, gpt_loss=0.0146, lr=6.57e-7]
+
Train step of epoch 0:  88%|████████▊ | 4967/5616 [1:03:15<07:58,  1.36it/s, gpt_loss=0.0593, lr=6.55e-7]
+
Train step of epoch 0:  88%|████████▊ | 4968/5616 [1:03:15<08:03,  1.34it/s, gpt_loss=0.0593, lr=6.55e-7]
+
Train step of epoch 0:  88%|████████▊ | 4968/5616 [1:03:16<08:03,  1.34it/s, gpt_loss=0.116, lr=6.55e-7] 
+
Train step of epoch 0:  88%|████████▊ | 4969/5616 [1:03:16<08:04,  1.34it/s, gpt_loss=0.116, lr=6.55e-7]
+
Train step of epoch 0:  88%|████████▊ | 4969/5616 [1:03:17<08:04,  1.34it/s, gpt_loss=0.0502, lr=6.55e-7]
+
Train step of epoch 0:  88%|████████▊ | 4970/5616 [1:03:17<08:07,  1.32it/s, gpt_loss=0.0502, lr=6.55e-7]
+
Train step of epoch 0:  88%|████████▊ | 4970/5616 [1:03:17<08:07,  1.32it/s, gpt_loss=0.0707, lr=6.55e-7]
+
Train step of epoch 0:  89%|████████▊ | 4971/5616 [1:03:17<07:47,  1.38it/s, gpt_loss=0.0707, lr=6.55e-7]
+
Train step of epoch 0:  89%|████████▊ | 4971/5616 [1:03:18<07:47,  1.38it/s, gpt_loss=0.0295, lr=6.54e-7]
+
Train step of epoch 0:  89%|████████▊ | 4972/5616 [1:03:18<07:47,  1.38it/s, gpt_loss=0.0295, lr=6.54e-7]
+
Train step of epoch 0:  89%|████████▊ | 4972/5616 [1:03:19<07:47,  1.38it/s, gpt_loss=0.057, lr=6.54e-7] 
+
Train step of epoch 0:  89%|████████▊ | 4973/5616 [1:03:19<08:03,  1.33it/s, gpt_loss=0.057, lr=6.54e-7]
+
Train step of epoch 0:  89%|████████▊ | 4973/5616 [1:03:20<08:03,  1.33it/s, gpt_loss=0.0791, lr=6.54e-7]
+
Train step of epoch 0:  89%|████████▊ | 4974/5616 [1:03:20<07:51,  1.36it/s, gpt_loss=0.0791, lr=6.54e-7]
+
Train step of epoch 0:  89%|████████▊ | 4974/5616 [1:03:20<07:51,  1.36it/s, gpt_loss=0.043, lr=6.54e-7] 
+
Train step of epoch 0:  89%|████████▊ | 4975/5616 [1:03:20<07:56,  1.34it/s, gpt_loss=0.043, lr=6.54e-7]
+
Train step of epoch 0:  89%|████████▊ | 4975/5616 [1:03:21<07:56,  1.34it/s, gpt_loss=0.000477, lr=6.52e-7]
+
Train step of epoch 0:  89%|████████▊ | 4976/5616 [1:03:21<07:58,  1.34it/s, gpt_loss=0.000477, lr=6.52e-7]
+
Train step of epoch 0:  89%|████████▊ | 4976/5616 [1:03:22<07:58,  1.34it/s, gpt_loss=0.00724, lr=6.52e-7] 
+
Train step of epoch 0:  89%|████████▊ | 4977/5616 [1:03:22<07:57,  1.34it/s, gpt_loss=0.00724, lr=6.52e-7]
+
Train step of epoch 0:  89%|████████▊ | 4977/5616 [1:03:23<07:57,  1.34it/s, gpt_loss=0.0349, lr=6.52e-7] 
+
Train step of epoch 0:  89%|████████▊ | 4978/5616 [1:03:23<07:52,  1.35it/s, gpt_loss=0.0349, lr=6.52e-7]
+
Train step of epoch 0:  89%|████████▊ | 4978/5616 [1:03:23<07:52,  1.35it/s, gpt_loss=0.0205, lr=6.52e-7]
+
Train step of epoch 0:  89%|████████▊ | 4979/5616 [1:03:23<08:03,  1.32it/s, gpt_loss=0.0205, lr=6.52e-7]
+
Train step of epoch 0:  89%|████████▊ | 4979/5616 [1:03:24<08:03,  1.32it/s, gpt_loss=0.107, lr=6.5e-7]  
+
Train step of epoch 0:  89%|████████▊ | 4980/5616 [1:03:24<08:10,  1.30it/s, gpt_loss=0.107, lr=6.5e-7]
+
Train step of epoch 0:  89%|████████▊ | 4980/5616 [1:03:25<08:10,  1.30it/s, gpt_loss=0.017, lr=6.5e-7]
+
Train step of epoch 0:  89%|████████▊ | 4981/5616 [1:03:25<08:24,  1.26it/s, gpt_loss=0.017, lr=6.5e-7]
+
Train step of epoch 0:  89%|████████▊ | 4981/5616 [1:03:26<08:24,  1.26it/s, gpt_loss=0.0403, lr=6.5e-7]
+
Train step of epoch 0:  89%|████████▊ | 4982/5616 [1:03:26<09:03,  1.17it/s, gpt_loss=0.0403, lr=6.5e-7]
+
Train step of epoch 0:  89%|████████▊ | 4982/5616 [1:03:27<09:03,  1.17it/s, gpt_loss=0.0129, lr=6.5e-7]
+
Train step of epoch 0:  89%|████████▊ | 4983/5616 [1:03:27<09:11,  1.15it/s, gpt_loss=0.0129, lr=6.5e-7]
+
Train step of epoch 0:  89%|████████▊ | 4983/5616 [1:03:28<09:11,  1.15it/s, gpt_loss=0.0336, lr=6.48e-7]
+
Train step of epoch 0:  89%|████████▊ | 4984/5616 [1:03:28<08:50,  1.19it/s, gpt_loss=0.0336, lr=6.48e-7]
+
Train step of epoch 0:  89%|████████▊ | 4984/5616 [1:03:29<08:50,  1.19it/s, gpt_loss=0.116, lr=6.48e-7] 
+
Train step of epoch 0:  89%|████████▉ | 4985/5616 [1:03:29<08:52,  1.19it/s, gpt_loss=0.116, lr=6.48e-7]
+
Train step of epoch 0:  89%|████████▉ | 4985/5616 [1:03:29<08:52,  1.19it/s, gpt_loss=0.0827, lr=6.48e-7]
+
Train step of epoch 0:  89%|████████▉ | 4986/5616 [1:03:29<08:29,  1.24it/s, gpt_loss=0.0827, lr=6.48e-7]
+
Train step of epoch 0:  89%|████████▉ | 4986/5616 [1:03:30<08:29,  1.24it/s, gpt_loss=0.0229, lr=6.48e-7]
+
Train step of epoch 0:  89%|████████▉ | 4987/5616 [1:03:30<08:10,  1.28it/s, gpt_loss=0.0229, lr=6.48e-7]
+
Train step of epoch 0:  89%|████████▉ | 4987/5616 [1:03:31<08:10,  1.28it/s, gpt_loss=0.0533, lr=6.46e-7]
+
Train step of epoch 0:  89%|████████▉ | 4988/5616 [1:03:31<07:58,  1.31it/s, gpt_loss=0.0533, lr=6.46e-7]
+
Train step of epoch 0:  89%|████████▉ | 4988/5616 [1:03:32<07:58,  1.31it/s, gpt_loss=0.0886, lr=6.46e-7]
+
Train step of epoch 0:  89%|████████▉ | 4989/5616 [1:03:32<07:56,  1.32it/s, gpt_loss=0.0886, lr=6.46e-7]
+
Train step of epoch 0:  89%|████████▉ | 4989/5616 [1:03:32<07:56,  1.32it/s, gpt_loss=0.00867, lr=6.46e-7]
+
Train step of epoch 0:  89%|████████▉ | 4990/5616 [1:03:32<08:09,  1.28it/s, gpt_loss=0.00867, lr=6.46e-7]
+
Train step of epoch 0:  89%|████████▉ | 4990/5616 [1:03:33<08:09,  1.28it/s, gpt_loss=0.0864, lr=6.46e-7] 
+
Train step of epoch 0:  89%|████████▉ | 4991/5616 [1:03:33<07:44,  1.35it/s, gpt_loss=0.0864, lr=6.46e-7]
+
Train step of epoch 0:  89%|████████▉ | 4991/5616 [1:03:34<07:44,  1.35it/s, gpt_loss=0.0328, lr=6.44e-7]
+
Train step of epoch 0:  89%|████████▉ | 4992/5616 [1:03:34<07:47,  1.34it/s, gpt_loss=0.0328, lr=6.44e-7]
+
Train step of epoch 0:  89%|████████▉ | 4992/5616 [1:03:34<07:47,  1.34it/s, gpt_loss=0.152, lr=6.44e-7] 
+
Train step of epoch 0:  89%|████████▉ | 4993/5616 [1:03:34<07:39,  1.36it/s, gpt_loss=0.152, lr=6.44e-7]
+
Train step of epoch 0:  89%|████████▉ | 4993/5616 [1:03:35<07:39,  1.36it/s, gpt_loss=0.0386, lr=6.44e-7]
+
Train step of epoch 0:  89%|████████▉ | 4994/5616 [1:03:35<07:45,  1.34it/s, gpt_loss=0.0386, lr=6.44e-7]
+
Train step of epoch 0:  89%|████████▉ | 4994/5616 [1:03:36<07:45,  1.34it/s, gpt_loss=0.0271, lr=6.44e-7]
+
Train step of epoch 0:  89%|████████▉ | 4995/5616 [1:03:36<07:46,  1.33it/s, gpt_loss=0.0271, lr=6.44e-7]
+
Train step of epoch 0:  89%|████████▉ | 4995/5616 [1:03:37<07:46,  1.33it/s, gpt_loss=0.0768, lr=6.42e-7]
+
Train step of epoch 0:  89%|████████▉ | 4996/5616 [1:03:37<08:09,  1.27it/s, gpt_loss=0.0768, lr=6.42e-7]
+
Train step of epoch 0:  89%|████████▉ | 4996/5616 [1:03:38<08:09,  1.27it/s, gpt_loss=0.00368, lr=6.42e-7]
+
Train step of epoch 0:  89%|████████▉ | 4997/5616 [1:03:38<08:15,  1.25it/s, gpt_loss=0.00368, lr=6.42e-7]
+
Train step of epoch 0:  89%|████████▉ | 4997/5616 [1:03:38<08:15,  1.25it/s, gpt_loss=0.0143, lr=6.42e-7] 
+
Train step of epoch 0:  89%|████████▉ | 4998/5616 [1:03:38<07:58,  1.29it/s, gpt_loss=0.0143, lr=6.42e-7]
+
Train step of epoch 0:  89%|████████▉ | 4998/5616 [1:03:39<07:58,  1.29it/s, gpt_loss=0.0275, lr=6.42e-7]
+
Train step of epoch 0:  89%|████████▉ | 4999/5616 [1:03:39<08:16,  1.24it/s, gpt_loss=0.0275, lr=6.42e-7]
+
Train step of epoch 0:  89%|████████▉ | 4999/5616 [1:03:40<08:16,  1.24it/s, gpt_loss=0.019, lr=6.41e-7] 
+
Train step of epoch 0:  89%|████████▉ | 5000/5616 [1:03:40<08:28,  1.21it/s, gpt_loss=0.019, lr=6.41e-7]
+
Train step of epoch 0:  89%|████████▉ | 5000/5616 [1:03:41<08:28,  1.21it/s, gpt_loss=0.00771, lr=6.41e-7]
+
Train step of epoch 0:  89%|████████▉ | 5001/5616 [1:03:41<08:53,  1.15it/s, gpt_loss=0.00771, lr=6.41e-7]
+
Train step of epoch 0:  89%|████████▉ | 5001/5616 [1:03:42<08:53,  1.15it/s, gpt_loss=0.0325, lr=6.41e-7] 
+
Train step of epoch 0:  89%|████████▉ | 5002/5616 [1:03:42<08:22,  1.22it/s, gpt_loss=0.0325, lr=6.41e-7]
+
Train step of epoch 0:  89%|████████▉ | 5002/5616 [1:03:43<08:22,  1.22it/s, gpt_loss=0.0178, lr=6.41e-7]
+
Train step of epoch 0:  89%|████████▉ | 5003/5616 [1:03:43<07:58,  1.28it/s, gpt_loss=0.0178, lr=6.41e-7]
+
Train step of epoch 0:  89%|████████▉ | 5003/5616 [1:03:43<07:58,  1.28it/s, gpt_loss=0.0173, lr=6.39e-7]
+
Train step of epoch 0:  89%|████████▉ | 5004/5616 [1:03:43<07:58,  1.28it/s, gpt_loss=0.0173, lr=6.39e-7]
+
Train step of epoch 0:  89%|████████▉ | 5004/5616 [1:03:44<07:58,  1.28it/s, gpt_loss=0.0165, lr=6.39e-7]
+
Train step of epoch 0:  89%|████████▉ | 5005/5616 [1:03:44<08:04,  1.26it/s, gpt_loss=0.0165, lr=6.39e-7]
+
Train step of epoch 0:  89%|████████▉ | 5005/5616 [1:03:45<08:04,  1.26it/s, gpt_loss=0.0193, lr=6.39e-7]
+
Train step of epoch 0:  89%|████████▉ | 5006/5616 [1:03:45<07:51,  1.29it/s, gpt_loss=0.0193, lr=6.39e-7]
+
Train step of epoch 0:  89%|████████▉ | 5006/5616 [1:03:46<07:51,  1.29it/s, gpt_loss=0.00527, lr=6.39e-7]
+
Train step of epoch 0:  89%|████████▉ | 5007/5616 [1:03:46<07:31,  1.35it/s, gpt_loss=0.00527, lr=6.39e-7]
+
Train step of epoch 0:  89%|████████▉ | 5007/5616 [1:03:46<07:31,  1.35it/s, gpt_loss=0.0525, lr=6.37e-7] 
+
Train step of epoch 0:  89%|████████▉ | 5008/5616 [1:03:46<07:59,  1.27it/s, gpt_loss=0.0525, lr=6.37e-7]
+
Train step of epoch 0:  89%|████████▉ | 5008/5616 [1:03:47<07:59,  1.27it/s, gpt_loss=0.0276, lr=6.37e-7]
+
Train step of epoch 0:  89%|████████▉ | 5009/5616 [1:03:47<07:30,  1.35it/s, gpt_loss=0.0276, lr=6.37e-7]
+
Train step of epoch 0:  89%|████████▉ | 5009/5616 [1:03:48<07:30,  1.35it/s, gpt_loss=0.0088, lr=6.37e-7]
+
Train step of epoch 0:  89%|████████▉ | 5010/5616 [1:03:48<07:22,  1.37it/s, gpt_loss=0.0088, lr=6.37e-7]
+
Train step of epoch 0:  89%|████████▉ | 5010/5616 [1:03:49<07:22,  1.37it/s, gpt_loss=0.0422, lr=6.37e-7]
+
Train step of epoch 0:  89%|████████▉ | 5011/5616 [1:03:49<07:37,  1.32it/s, gpt_loss=0.0422, lr=6.37e-7]
+
Train step of epoch 0:  89%|████████▉ | 5011/5616 [1:03:49<07:37,  1.32it/s, gpt_loss=0.106, lr=6.35e-7] 
+
Train step of epoch 0:  89%|████████▉ | 5012/5616 [1:03:49<07:41,  1.31it/s, gpt_loss=0.106, lr=6.35e-7]
+
Train step of epoch 0:  89%|████████▉ | 5012/5616 [1:03:50<07:41,  1.31it/s, gpt_loss=0.0434, lr=6.35e-7]
+
Train step of epoch 0:  89%|████████▉ | 5013/5616 [1:03:50<07:32,  1.33it/s, gpt_loss=0.0434, lr=6.35e-7]
+
Train step of epoch 0:  89%|████████▉ | 5013/5616 [1:03:51<07:32,  1.33it/s, gpt_loss=0.0141, lr=6.35e-7]
+
Train step of epoch 0:  89%|████████▉ | 5014/5616 [1:03:51<07:12,  1.39it/s, gpt_loss=0.0141, lr=6.35e-7]
+
Train step of epoch 0:  89%|████████▉ | 5014/5616 [1:03:51<07:12,  1.39it/s, gpt_loss=0.0386, lr=6.35e-7]
+
Train step of epoch 0:  89%|████████▉ | 5015/5616 [1:03:51<07:13,  1.39it/s, gpt_loss=0.0386, lr=6.35e-7]
+
Train step of epoch 0:  89%|████████▉ | 5015/5616 [1:03:52<07:13,  1.39it/s, gpt_loss=0.0104, lr=6.34e-7]
+
Train step of epoch 0:  89%|████████▉ | 5016/5616 [1:03:52<07:29,  1.33it/s, gpt_loss=0.0104, lr=6.34e-7]
+
Train step of epoch 0:  89%|████████▉ | 5016/5616 [1:03:53<07:29,  1.33it/s, gpt_loss=0.0174, lr=6.34e-7]
+
Train step of epoch 0:  89%|████████▉ | 5017/5616 [1:03:53<07:15,  1.38it/s, gpt_loss=0.0174, lr=6.34e-7]
+
Train step of epoch 0:  89%|████████▉ | 5017/5616 [1:03:54<07:15,  1.38it/s, gpt_loss=0.0308, lr=6.34e-7]
+
Train step of epoch 0:  89%|████████▉ | 5018/5616 [1:03:54<07:08,  1.40it/s, gpt_loss=0.0308, lr=6.34e-7]
+
Train step of epoch 0:  89%|████████▉ | 5018/5616 [1:03:54<07:08,  1.40it/s, gpt_loss=0.0041, lr=6.34e-7]
+
Train step of epoch 0:  89%|████████▉ | 5019/5616 [1:03:54<07:08,  1.39it/s, gpt_loss=0.0041, lr=6.34e-7]
+
Train step of epoch 0:  89%|████████▉ | 5019/5616 [1:03:55<07:08,  1.39it/s, gpt_loss=0.0324, lr=6.32e-7]
+
Train step of epoch 0:  89%|████████▉ | 5020/5616 [1:03:55<07:16,  1.37it/s, gpt_loss=0.0324, lr=6.32e-7]
+
Train step of epoch 0:  89%|████████▉ | 5020/5616 [1:03:56<07:16,  1.37it/s, gpt_loss=0.0046, lr=6.32e-7]
+
Train step of epoch 0:  89%|████████▉ | 5021/5616 [1:03:56<07:27,  1.33it/s, gpt_loss=0.0046, lr=6.32e-7]
+
Train step of epoch 0:  89%|████████▉ | 5021/5616 [1:03:57<07:27,  1.33it/s, gpt_loss=0.0107, lr=6.32e-7]
+
Train step of epoch 0:  89%|████████▉ | 5022/5616 [1:03:57<07:23,  1.34it/s, gpt_loss=0.0107, lr=6.32e-7]
+
Train step of epoch 0:  89%|████████▉ | 5022/5616 [1:03:57<07:23,  1.34it/s, gpt_loss=0.0538, lr=6.32e-7]
+
Train step of epoch 0:  89%|████████▉ | 5023/5616 [1:03:57<07:03,  1.40it/s, gpt_loss=0.0538, lr=6.32e-7]
+
Train step of epoch 0:  89%|████████▉ | 5023/5616 [1:03:58<07:03,  1.40it/s, gpt_loss=0.0112, lr=6.3e-7] 
+
Train step of epoch 0:  89%|████████▉ | 5024/5616 [1:03:58<07:26,  1.32it/s, gpt_loss=0.0112, lr=6.3e-7]
+
Train step of epoch 0:  89%|████████▉ | 5024/5616 [1:03:59<07:26,  1.32it/s, gpt_loss=0.0261, lr=6.3e-7]
+
Train step of epoch 0:  89%|████████▉ | 5025/5616 [1:03:59<07:26,  1.32it/s, gpt_loss=0.0261, lr=6.3e-7]
+
Train step of epoch 0:  89%|████████▉ | 5025/5616 [1:04:00<07:26,  1.32it/s, gpt_loss=0.0219, lr=6.3e-7]
+
Train step of epoch 0:  89%|████████▉ | 5026/5616 [1:04:00<07:28,  1.32it/s, gpt_loss=0.0219, lr=6.3e-7]
+
Train step of epoch 0:  89%|████████▉ | 5026/5616 [1:04:00<07:28,  1.32it/s, gpt_loss=0.013, lr=6.3e-7] 
+
Train step of epoch 0:  90%|████████▉ | 5027/5616 [1:04:00<07:21,  1.33it/s, gpt_loss=0.013, lr=6.3e-7]
+
Train step of epoch 0:  90%|████████▉ | 5027/5616 [1:04:01<07:21,  1.33it/s, gpt_loss=0.0217, lr=6.28e-7]
+
Train step of epoch 0:  90%|████████▉ | 5028/5616 [1:04:01<07:43,  1.27it/s, gpt_loss=0.0217, lr=6.28e-7]
+
Train step of epoch 0:  90%|████████▉ | 5028/5616 [1:04:02<07:43,  1.27it/s, gpt_loss=0.0527, lr=6.28e-7]
+
Train step of epoch 0:  90%|████████▉ | 5029/5616 [1:04:02<07:54,  1.24it/s, gpt_loss=0.0527, lr=6.28e-7]
+
Train step of epoch 0:  90%|████████▉ | 5029/5616 [1:04:03<07:54,  1.24it/s, gpt_loss=0.026, lr=6.28e-7] 
+
Train step of epoch 0:  90%|████████▉ | 5030/5616 [1:04:03<07:32,  1.30it/s, gpt_loss=0.026, lr=6.28e-7]
+
Train step of epoch 0:  90%|████████▉ | 5030/5616 [1:04:04<07:32,  1.30it/s, gpt_loss=0.0314, lr=6.28e-7]
+
Train step of epoch 0:  90%|████████▉ | 5031/5616 [1:04:04<07:22,  1.32it/s, gpt_loss=0.0314, lr=6.28e-7]
+
Train step of epoch 0:  90%|████████▉ | 5031/5616 [1:04:04<07:22,  1.32it/s, gpt_loss=0.037, lr=6.27e-7] 
+
Train step of epoch 0:  90%|████████▉ | 5032/5616 [1:04:04<07:40,  1.27it/s, gpt_loss=0.037, lr=6.27e-7]
+
Train step of epoch 0:  90%|████████▉ | 5032/5616 [1:04:05<07:40,  1.27it/s, gpt_loss=0.02, lr=6.27e-7] 
+
Train step of epoch 0:  90%|████████▉ | 5033/5616 [1:04:05<07:28,  1.30it/s, gpt_loss=0.02, lr=6.27e-7]
+
Train step of epoch 0:  90%|████████▉ | 5033/5616 [1:04:06<07:28,  1.30it/s, gpt_loss=0.00915, lr=6.27e-7]
+
Train step of epoch 0:  90%|████████▉ | 5034/5616 [1:04:06<08:02,  1.21it/s, gpt_loss=0.00915, lr=6.27e-7]
+
Train step of epoch 0:  90%|████████▉ | 5034/5616 [1:04:07<08:02,  1.21it/s, gpt_loss=0.0163, lr=6.27e-7] 
+
Train step of epoch 0:  90%|████████▉ | 5035/5616 [1:04:07<07:47,  1.24it/s, gpt_loss=0.0163, lr=6.27e-7]
+
Train step of epoch 0:  90%|████████▉ | 5035/5616 [1:04:08<07:47,  1.24it/s, gpt_loss=0.0756, lr=6.25e-7]
+
Train step of epoch 0:  90%|████████▉ | 5036/5616 [1:04:08<07:57,  1.21it/s, gpt_loss=0.0756, lr=6.25e-7]
+
Train step of epoch 0:  90%|████████▉ | 5036/5616 [1:04:08<07:57,  1.21it/s, gpt_loss=0.0905, lr=6.25e-7]
+
Train step of epoch 0:  90%|████████▉ | 5037/5616 [1:04:08<07:35,  1.27it/s, gpt_loss=0.0905, lr=6.25e-7]
+
Train step of epoch 0:  90%|████████▉ | 5037/5616 [1:04:09<07:35,  1.27it/s, gpt_loss=0.0204, lr=6.25e-7]
+
Train step of epoch 0:  90%|████████▉ | 5038/5616 [1:04:09<07:19,  1.31it/s, gpt_loss=0.0204, lr=6.25e-7]
+
Train step of epoch 0:  90%|████████▉ | 5038/5616 [1:04:10<07:19,  1.31it/s, gpt_loss=0.0643, lr=6.25e-7]
+
Train step of epoch 0:  90%|████████▉ | 5039/5616 [1:04:10<07:19,  1.31it/s, gpt_loss=0.0643, lr=6.25e-7]
+
Train step of epoch 0:  90%|████████▉ | 5039/5616 [1:04:11<07:19,  1.31it/s, gpt_loss=0.0163, lr=6.23e-7]
+
Train step of epoch 0:  90%|████████▉ | 5040/5616 [1:04:11<07:26,  1.29it/s, gpt_loss=0.0163, lr=6.23e-7]
+
Train step of epoch 0:  90%|████████▉ | 5040/5616 [1:04:11<07:26,  1.29it/s, gpt_loss=0.00588, lr=6.23e-7]
+
Train step of epoch 0:  90%|████████▉ | 5041/5616 [1:04:11<07:25,  1.29it/s, gpt_loss=0.00588, lr=6.23e-7]
+
Train step of epoch 0:  90%|████████▉ | 5041/5616 [1:04:12<07:25,  1.29it/s, gpt_loss=0.00155, lr=6.23e-7]
+
Train step of epoch 0:  90%|████████▉ | 5042/5616 [1:04:12<07:10,  1.33it/s, gpt_loss=0.00155, lr=6.23e-7]
+
Train step of epoch 0:  90%|████████▉ | 5042/5616 [1:04:13<07:10,  1.33it/s, gpt_loss=0.013, lr=6.23e-7]  
+
Train step of epoch 0:  90%|████████▉ | 5043/5616 [1:04:13<07:04,  1.35it/s, gpt_loss=0.013, lr=6.23e-7]
+
Train step of epoch 0:  90%|████████▉ | 5043/5616 [1:04:14<07:04,  1.35it/s, gpt_loss=0.0396, lr=6.21e-7]
+
Train step of epoch 0:  90%|████████▉ | 5044/5616 [1:04:14<07:10,  1.33it/s, gpt_loss=0.0396, lr=6.21e-7]
+
Train step of epoch 0:  90%|████████▉ | 5044/5616 [1:04:14<07:10,  1.33it/s, gpt_loss=0.0457, lr=6.21e-7]
+
Train step of epoch 0:  90%|████████▉ | 5045/5616 [1:04:14<06:57,  1.37it/s, gpt_loss=0.0457, lr=6.21e-7]
+
Train step of epoch 0:  90%|████████▉ | 5045/5616 [1:04:15<06:57,  1.37it/s, gpt_loss=0.0475, lr=6.21e-7]
+
Train step of epoch 0:  90%|████████▉ | 5046/5616 [1:04:15<06:49,  1.39it/s, gpt_loss=0.0475, lr=6.21e-7]
+
Train step of epoch 0:  90%|████████▉ | 5046/5616 [1:04:16<06:49,  1.39it/s, gpt_loss=0.0124, lr=6.21e-7]
+
Train step of epoch 0:  90%|████████▉ | 5047/5616 [1:04:16<07:05,  1.34it/s, gpt_loss=0.0124, lr=6.21e-7]
+
Train step of epoch 0:  90%|████████▉ | 5047/5616 [1:04:17<07:05,  1.34it/s, gpt_loss=0.0391, lr=6.2e-7] 
+
Train step of epoch 0:  90%|████████▉ | 5048/5616 [1:04:17<08:02,  1.18it/s, gpt_loss=0.0391, lr=6.2e-7]
+
Train step of epoch 0:  90%|████████▉ | 5048/5616 [1:04:18<08:02,  1.18it/s, gpt_loss=0.00334, lr=6.2e-7]
+
Train step of epoch 0:  90%|████████▉ | 5049/5616 [1:04:18<07:24,  1.28it/s, gpt_loss=0.00334, lr=6.2e-7]
+
Train step of epoch 0:  90%|████████▉ | 5049/5616 [1:04:18<07:24,  1.28it/s, gpt_loss=0.0227, lr=6.2e-7] 
+
Train step of epoch 0:  90%|████████▉ | 5050/5616 [1:04:18<07:39,  1.23it/s, gpt_loss=0.0227, lr=6.2e-7]
+
Train step of epoch 0:  90%|████████▉ | 5050/5616 [1:04:19<07:39,  1.23it/s, gpt_loss=0.0483, lr=6.2e-7]
+
Train step of epoch 0:  90%|████████▉ | 5051/5616 [1:04:19<07:16,  1.29it/s, gpt_loss=0.0483, lr=6.2e-7]
+
Train step of epoch 0:  90%|████████▉ | 5051/5616 [1:04:20<07:16,  1.29it/s, gpt_loss=0.0983, lr=6.18e-7]
+
Train step of epoch 0:  90%|████████▉ | 5052/5616 [1:04:20<07:20,  1.28it/s, gpt_loss=0.0983, lr=6.18e-7]
+
Train step of epoch 0:  90%|████████▉ | 5052/5616 [1:04:21<07:20,  1.28it/s, gpt_loss=0.132, lr=6.18e-7] 
+
Train step of epoch 0:  90%|████████▉ | 5053/5616 [1:04:21<07:01,  1.34it/s, gpt_loss=0.132, lr=6.18e-7]
+
Train step of epoch 0:  90%|████████▉ | 5053/5616 [1:04:21<07:01,  1.34it/s, gpt_loss=0.0135, lr=6.18e-7]
+
Train step of epoch 0:  90%|████████▉ | 5054/5616 [1:04:21<06:46,  1.38it/s, gpt_loss=0.0135, lr=6.18e-7]
+
Train step of epoch 0:  90%|████████▉ | 5054/5616 [1:04:22<06:46,  1.38it/s, gpt_loss=0.0185, lr=6.18e-7]
+
Train step of epoch 0:  90%|█████████ | 5055/5616 [1:04:22<06:56,  1.35it/s, gpt_loss=0.0185, lr=6.18e-7]
+
Train step of epoch 0:  90%|█████████ | 5055/5616 [1:04:23<06:56,  1.35it/s, gpt_loss=0.00804, lr=6.16e-7]
+
Train step of epoch 0:  90%|█████████ | 5056/5616 [1:04:23<06:58,  1.34it/s, gpt_loss=0.00804, lr=6.16e-7]
+
Train step of epoch 0:  90%|█████████ | 5056/5616 [1:04:24<06:58,  1.34it/s, gpt_loss=0.00518, lr=6.16e-7]
+
Train step of epoch 0:  90%|█████████ | 5057/5616 [1:04:24<06:53,  1.35it/s, gpt_loss=0.00518, lr=6.16e-7]
+
Train step of epoch 0:  90%|█████████ | 5057/5616 [1:04:24<06:53,  1.35it/s, gpt_loss=0.0508, lr=6.16e-7] 
+
Train step of epoch 0:  90%|█████████ | 5058/5616 [1:04:24<06:57,  1.34it/s, gpt_loss=0.0508, lr=6.16e-7]
+
Train step of epoch 0:  90%|█████████ | 5058/5616 [1:04:25<06:57,  1.34it/s, gpt_loss=0.0192, lr=6.16e-7]
+
Train step of epoch 0:  90%|█████████ | 5059/5616 [1:04:25<07:11,  1.29it/s, gpt_loss=0.0192, lr=6.16e-7]
+
Train step of epoch 0:  90%|█████████ | 5059/5616 [1:04:26<07:11,  1.29it/s, gpt_loss=0.0774, lr=6.15e-7]
+
Train step of epoch 0:  90%|█████████ | 5060/5616 [1:04:26<07:01,  1.32it/s, gpt_loss=0.0774, lr=6.15e-7]
+
Train step of epoch 0:  90%|█████████ | 5060/5616 [1:04:27<07:01,  1.32it/s, gpt_loss=0.0581, lr=6.15e-7]
+
Train step of epoch 0:  90%|█████████ | 5061/5616 [1:04:27<08:19,  1.11it/s, gpt_loss=0.0581, lr=6.15e-7]
+
Train step of epoch 0:  90%|█████████ | 5061/5616 [1:04:28<08:19,  1.11it/s, gpt_loss=0.0448, lr=6.15e-7]
+
Train step of epoch 0:  90%|█████████ | 5062/5616 [1:04:28<07:40,  1.20it/s, gpt_loss=0.0448, lr=6.15e-7]
+
Train step of epoch 0:  90%|█████████ | 5062/5616 [1:04:29<07:40,  1.20it/s, gpt_loss=0.0342, lr=6.15e-7]
+
Train step of epoch 0:  90%|█████████ | 5063/5616 [1:04:29<07:30,  1.23it/s, gpt_loss=0.0342, lr=6.15e-7]
+
Train step of epoch 0:  90%|█████████ | 5063/5616 [1:04:29<07:30,  1.23it/s, gpt_loss=0.0432, lr=6.13e-7]
+
Train step of epoch 0:  90%|█████████ | 5064/5616 [1:04:29<07:18,  1.26it/s, gpt_loss=0.0432, lr=6.13e-7]
+
Train step of epoch 0:  90%|█████████ | 5064/5616 [1:04:30<07:18,  1.26it/s, gpt_loss=0.00991, lr=6.13e-7]
+
Train step of epoch 0:  90%|█████████ | 5065/5616 [1:04:30<07:15,  1.27it/s, gpt_loss=0.00991, lr=6.13e-7]
+
Train step of epoch 0:  90%|█████████ | 5065/5616 [1:04:31<07:15,  1.27it/s, gpt_loss=0.00614, lr=6.13e-7]
+
Train step of epoch 0:  90%|█████████ | 5066/5616 [1:04:31<07:55,  1.16it/s, gpt_loss=0.00614, lr=6.13e-7]
+
Train step of epoch 0:  90%|█████████ | 5066/5616 [1:04:32<07:55,  1.16it/s, gpt_loss=0.168, lr=6.13e-7]  
+
Train step of epoch 0:  90%|█████████ | 5067/5616 [1:04:32<07:35,  1.20it/s, gpt_loss=0.168, lr=6.13e-7]
+
Train step of epoch 0:  90%|█████████ | 5067/5616 [1:04:33<07:35,  1.20it/s, gpt_loss=0.0197, lr=6.12e-7]
+
Train step of epoch 0:  90%|█████████ | 5068/5616 [1:04:33<07:24,  1.23it/s, gpt_loss=0.0197, lr=6.12e-7]
+
Train step of epoch 0:  90%|█████████ | 5068/5616 [1:04:33<07:24,  1.23it/s, gpt_loss=0.0733, lr=6.12e-7]
+
Train step of epoch 0:  90%|█████████ | 5069/5616 [1:04:33<07:28,  1.22it/s, gpt_loss=0.0733, lr=6.12e-7]
+
Train step of epoch 0:  90%|█████████ | 5069/5616 [1:04:34<07:28,  1.22it/s, gpt_loss=0.0433, lr=6.12e-7]
+
Train step of epoch 0:  90%|█████████ | 5070/5616 [1:04:34<06:56,  1.31it/s, gpt_loss=0.0433, lr=6.12e-7]
+
Train step of epoch 0:  90%|█████████ | 5070/5616 [1:04:35<06:56,  1.31it/s, gpt_loss=0.0491, lr=6.12e-7]
+
Train step of epoch 0:  90%|█████████ | 5071/5616 [1:04:35<07:03,  1.29it/s, gpt_loss=0.0491, lr=6.12e-7]
+
Train step of epoch 0:  90%|█████████ | 5071/5616 [1:04:36<07:03,  1.29it/s, gpt_loss=0.0249, lr=6.1e-7] 
+
Train step of epoch 0:  90%|█████████ | 5072/5616 [1:04:36<07:01,  1.29it/s, gpt_loss=0.0249, lr=6.1e-7]
+
Train step of epoch 0:  90%|█████████ | 5072/5616 [1:04:37<07:01,  1.29it/s, gpt_loss=0.00586, lr=6.1e-7]
+
Train step of epoch 0:  90%|█████████ | 5073/5616 [1:04:37<08:08,  1.11it/s, gpt_loss=0.00586, lr=6.1e-7]
+
Train step of epoch 0:  90%|█████████ | 5073/5616 [1:04:38<08:08,  1.11it/s, gpt_loss=0.023, lr=6.1e-7]  
+
Train step of epoch 0:  90%|█████████ | 5074/5616 [1:04:38<07:48,  1.16it/s, gpt_loss=0.023, lr=6.1e-7]
+
Train step of epoch 0:  90%|█████████ | 5074/5616 [1:04:38<07:48,  1.16it/s, gpt_loss=0.0145, lr=6.1e-7]
+
Train step of epoch 0:  90%|█████████ | 5075/5616 [1:04:38<07:15,  1.24it/s, gpt_loss=0.0145, lr=6.1e-7]
+
Train step of epoch 0:  90%|█████████ | 5075/5616 [1:04:39<07:15,  1.24it/s, gpt_loss=0.0107, lr=6.08e-7]
+
Train step of epoch 0:  90%|█████████ | 5076/5616 [1:04:39<07:02,  1.28it/s, gpt_loss=0.0107, lr=6.08e-7]
+
Train step of epoch 0:  90%|█████████ | 5076/5616 [1:04:40<07:02,  1.28it/s, gpt_loss=0.00309, lr=6.08e-7]
+
Train step of epoch 0:  90%|█████████ | 5077/5616 [1:04:40<06:45,  1.33it/s, gpt_loss=0.00309, lr=6.08e-7]
+
Train step of epoch 0:  90%|█████████ | 5077/5616 [1:04:40<06:45,  1.33it/s, gpt_loss=0.0159, lr=6.08e-7] 
+
Train step of epoch 0:  90%|█████████ | 5078/5616 [1:04:40<06:31,  1.37it/s, gpt_loss=0.0159, lr=6.08e-7]
+
Train step of epoch 0:  90%|█████████ | 5078/5616 [1:04:41<06:31,  1.37it/s, gpt_loss=0.0423, lr=6.08e-7]
+
Train step of epoch 0:  90%|█████████ | 5079/5616 [1:04:41<06:33,  1.36it/s, gpt_loss=0.0423, lr=6.08e-7]
+
Train step of epoch 0:  90%|█████████ | 5079/5616 [1:04:42<06:33,  1.36it/s, gpt_loss=0.0046, lr=6.07e-7]
+
Train step of epoch 0:  90%|█████████ | 5080/5616 [1:04:42<06:39,  1.34it/s, gpt_loss=0.0046, lr=6.07e-7]
+
Train step of epoch 0:  90%|█████████ | 5080/5616 [1:04:43<06:39,  1.34it/s, gpt_loss=0.00611, lr=6.07e-7]
+
Train step of epoch 0:  90%|█████████ | 5081/5616 [1:04:43<06:39,  1.34it/s, gpt_loss=0.00611, lr=6.07e-7]
+
Train step of epoch 0:  90%|█████████ | 5081/5616 [1:04:43<06:39,  1.34it/s, gpt_loss=0.055, lr=6.07e-7]  
+
Train step of epoch 0:  90%|█████████ | 5082/5616 [1:04:43<06:29,  1.37it/s, gpt_loss=0.055, lr=6.07e-7]
+
Train step of epoch 0:  90%|█████████ | 5082/5616 [1:04:44<06:29,  1.37it/s, gpt_loss=0.0867, lr=6.07e-7]
+
Train step of epoch 0:  91%|█████████ | 5083/5616 [1:04:44<06:19,  1.41it/s, gpt_loss=0.0867, lr=6.07e-7]
+
Train step of epoch 0:  91%|█████████ | 5083/5616 [1:04:45<06:19,  1.41it/s, gpt_loss=0.00959, lr=6.05e-7]
+
Train step of epoch 0:  91%|█████████ | 5084/5616 [1:04:45<06:36,  1.34it/s, gpt_loss=0.00959, lr=6.05e-7]
+
Train step of epoch 0:  91%|█████████ | 5084/5616 [1:04:46<06:36,  1.34it/s, gpt_loss=0.0801, lr=6.05e-7] 
+
Train step of epoch 0:  91%|█████████ | 5085/5616 [1:04:46<06:44,  1.31it/s, gpt_loss=0.0801, lr=6.05e-7]
+
Train step of epoch 0:  91%|█████████ | 5085/5616 [1:04:46<06:44,  1.31it/s, gpt_loss=0.102, lr=6.05e-7] 
+
Train step of epoch 0:  91%|█████████ | 5086/5616 [1:04:46<06:35,  1.34it/s, gpt_loss=0.102, lr=6.05e-7]
+
Train step of epoch 0:  91%|█████████ | 5086/5616 [1:04:47<06:35,  1.34it/s, gpt_loss=0.0886, lr=6.05e-7]
+
Train step of epoch 0:  91%|█████████ | 5087/5616 [1:04:47<06:32,  1.35it/s, gpt_loss=0.0886, lr=6.05e-7]
+
Train step of epoch 0:  91%|█████████ | 5087/5616 [1:04:48<06:32,  1.35it/s, gpt_loss=0.0807, lr=6.04e-7]
+
Train step of epoch 0:  91%|█████████ | 5088/5616 [1:04:48<06:51,  1.28it/s, gpt_loss=0.0807, lr=6.04e-7]
+
Train step of epoch 0:  91%|█████████ | 5088/5616 [1:04:49<06:51,  1.28it/s, gpt_loss=0.0472, lr=6.04e-7]
+
Train step of epoch 0:  91%|█████████ | 5089/5616 [1:04:49<07:05,  1.24it/s, gpt_loss=0.0472, lr=6.04e-7]
+
Train step of epoch 0:  91%|█████████ | 5089/5616 [1:04:49<07:05,  1.24it/s, gpt_loss=0.0088, lr=6.04e-7]
+
Train step of epoch 0:  91%|█████████ | 5090/5616 [1:04:49<06:43,  1.30it/s, gpt_loss=0.0088, lr=6.04e-7]
+
Train step of epoch 0:  91%|█████████ | 5090/5616 [1:04:50<06:43,  1.30it/s, gpt_loss=0.081, lr=6.04e-7] 
+
Train step of epoch 0:  91%|█████████ | 5091/5616 [1:04:50<06:44,  1.30it/s, gpt_loss=0.081, lr=6.04e-7]
+
Train step of epoch 0:  91%|█████████ | 5091/5616 [1:04:51<06:44,  1.30it/s, gpt_loss=0.0156, lr=6.02e-7]
+
Train step of epoch 0:  91%|█████████ | 5092/5616 [1:04:51<06:52,  1.27it/s, gpt_loss=0.0156, lr=6.02e-7]
+
Train step of epoch 0:  91%|█████████ | 5092/5616 [1:04:52<06:52,  1.27it/s, gpt_loss=0.16, lr=6.02e-7]  
+
Train step of epoch 0:  91%|█████████ | 5093/5616 [1:04:52<06:34,  1.32it/s, gpt_loss=0.16, lr=6.02e-7]
+
Train step of epoch 0:  91%|█████████ | 5093/5616 [1:04:53<06:34,  1.32it/s, gpt_loss=0.00231, lr=6.02e-7]
+
Train step of epoch 0:  91%|█████████ | 5094/5616 [1:04:53<06:39,  1.31it/s, gpt_loss=0.00231, lr=6.02e-7]
+
Train step of epoch 0:  91%|█████████ | 5094/5616 [1:04:53<06:39,  1.31it/s, gpt_loss=0.00857, lr=6.02e-7]
+
Train step of epoch 0:  91%|█████████ | 5095/5616 [1:04:53<06:34,  1.32it/s, gpt_loss=0.00857, lr=6.02e-7]
+
Train step of epoch 0:  91%|█████████ | 5095/5616 [1:04:54<06:34,  1.32it/s, gpt_loss=0.0322, lr=6.01e-7] 
+
Train step of epoch 0:  91%|█████████ | 5096/5616 [1:04:54<06:33,  1.32it/s, gpt_loss=0.0322, lr=6.01e-7]
+
Train step of epoch 0:  91%|█████████ | 5096/5616 [1:04:55<06:33,  1.32it/s, gpt_loss=0.0405, lr=6.01e-7]
+
Train step of epoch 0:  91%|█████████ | 5097/5616 [1:04:55<06:25,  1.35it/s, gpt_loss=0.0405, lr=6.01e-7]
+
Train step of epoch 0:  91%|█████████ | 5097/5616 [1:04:55<06:25,  1.35it/s, gpt_loss=0.0376, lr=6.01e-7]
+
Train step of epoch 0:  91%|█████████ | 5098/5616 [1:04:55<06:26,  1.34it/s, gpt_loss=0.0376, lr=6.01e-7]
+
Train step of epoch 0:  91%|█████████ | 5098/5616 [1:04:56<06:26,  1.34it/s, gpt_loss=0.0245, lr=6.01e-7]
+
Train step of epoch 0:  91%|█████████ | 5099/5616 [1:04:56<06:18,  1.37it/s, gpt_loss=0.0245, lr=6.01e-7]
+
Train step of epoch 0:  91%|█████████ | 5099/5616 [1:04:57<06:18,  1.37it/s, gpt_loss=0.0384, lr=5.99e-7]
+
Train step of epoch 0:  91%|█████████ | 5100/5616 [1:04:57<06:40,  1.29it/s, gpt_loss=0.0384, lr=5.99e-7]
+
Train step of epoch 0:  91%|█████████ | 5100/5616 [1:04:58<06:40,  1.29it/s, gpt_loss=0.0345, lr=5.99e-7]
+
Train step of epoch 0:  91%|█████████ | 5101/5616 [1:04:58<06:30,  1.32it/s, gpt_loss=0.0345, lr=5.99e-7]
+
Train step of epoch 0:  91%|█████████ | 5101/5616 [1:04:58<06:30,  1.32it/s, gpt_loss=0.0167, lr=5.99e-7]
+
Train step of epoch 0:  91%|█████████ | 5102/5616 [1:04:58<06:21,  1.35it/s, gpt_loss=0.0167, lr=5.99e-7]
+
Train step of epoch 0:  91%|█████████ | 5102/5616 [1:04:59<06:21,  1.35it/s, gpt_loss=0.0115, lr=5.99e-7]
+
Train step of epoch 0:  91%|█████████ | 5103/5616 [1:04:59<06:43,  1.27it/s, gpt_loss=0.0115, lr=5.99e-7]
+
Train step of epoch 0:  91%|█████████ | 5103/5616 [1:05:00<06:43,  1.27it/s, gpt_loss=0.0612, lr=5.97e-7]
+
Train step of epoch 0:  91%|█████████ | 5104/5616 [1:05:00<06:44,  1.27it/s, gpt_loss=0.0612, lr=5.97e-7]
+
Train step of epoch 0:  91%|█████████ | 5104/5616 [1:05:01<06:44,  1.27it/s, gpt_loss=0.0234, lr=5.97e-7]
+
Train step of epoch 0:  91%|█████████ | 5105/5616 [1:05:01<06:39,  1.28it/s, gpt_loss=0.0234, lr=5.97e-7]
+
Train step of epoch 0:  91%|█████████ | 5105/5616 [1:05:02<06:39,  1.28it/s, gpt_loss=0.0123, lr=5.97e-7]
+
Train step of epoch 0:  91%|█████████ | 5106/5616 [1:05:02<06:37,  1.28it/s, gpt_loss=0.0123, lr=5.97e-7]
+
Train step of epoch 0:  91%|█████████ | 5106/5616 [1:05:02<06:37,  1.28it/s, gpt_loss=0.0838, lr=5.97e-7]
+
Train step of epoch 0:  91%|█████████ | 5107/5616 [1:05:02<06:28,  1.31it/s, gpt_loss=0.0838, lr=5.97e-7]
+
Train step of epoch 0:  91%|█████████ | 5107/5616 [1:05:03<06:28,  1.31it/s, gpt_loss=0.0165, lr=5.96e-7]
+
Train step of epoch 0:  91%|█████████ | 5108/5616 [1:05:03<06:28,  1.31it/s, gpt_loss=0.0165, lr=5.96e-7]
+
Train step of epoch 0:  91%|█████████ | 5108/5616 [1:05:04<06:28,  1.31it/s, gpt_loss=0.0334, lr=5.96e-7]
+
Train step of epoch 0:  91%|█████████ | 5109/5616 [1:05:04<06:23,  1.32it/s, gpt_loss=0.0334, lr=5.96e-7]
+
Train step of epoch 0:  91%|█████████ | 5109/5616 [1:05:05<06:23,  1.32it/s, gpt_loss=0.016, lr=5.96e-7] 
+
Train step of epoch 0:  91%|█████████ | 5110/5616 [1:05:05<06:22,  1.32it/s, gpt_loss=0.016, lr=5.96e-7]
+
Train step of epoch 0:  91%|█████████ | 5110/5616 [1:05:05<06:22,  1.32it/s, gpt_loss=0.00959, lr=5.96e-7]
+
Train step of epoch 0:  91%|█████████ | 5111/5616 [1:05:05<06:12,  1.36it/s, gpt_loss=0.00959, lr=5.96e-7]
+
Train step of epoch 0:  91%|█████████ | 5111/5616 [1:05:06<06:12,  1.36it/s, gpt_loss=0.00488, lr=5.94e-7]
+
Train step of epoch 0:  91%|█████████ | 5112/5616 [1:05:06<06:37,  1.27it/s, gpt_loss=0.00488, lr=5.94e-7]
+
Train step of epoch 0:  91%|█████████ | 5112/5616 [1:05:07<06:37,  1.27it/s, gpt_loss=0.00263, lr=5.94e-7]
+
Train step of epoch 0:  91%|█████████ | 5113/5616 [1:05:07<06:20,  1.32it/s, gpt_loss=0.00263, lr=5.94e-7]
+
Train step of epoch 0:  91%|█████████ | 5113/5616 [1:05:08<06:20,  1.32it/s, gpt_loss=0.00457, lr=5.94e-7]
+
Train step of epoch 0:  91%|█████████ | 5114/5616 [1:05:08<06:04,  1.38it/s, gpt_loss=0.00457, lr=5.94e-7]
+
Train step of epoch 0:  91%|█████████ | 5114/5616 [1:05:08<06:04,  1.38it/s, gpt_loss=0.0515, lr=5.94e-7] 
+
Train step of epoch 0:  91%|█████████ | 5115/5616 [1:05:08<06:01,  1.39it/s, gpt_loss=0.0515, lr=5.94e-7]
+
Train step of epoch 0:  91%|█████████ | 5115/5616 [1:05:09<06:01,  1.39it/s, gpt_loss=0.00742, lr=5.93e-7]
+
Train step of epoch 0:  91%|█████████ | 5116/5616 [1:05:09<06:07,  1.36it/s, gpt_loss=0.00742, lr=5.93e-7]
+
Train step of epoch 0:  91%|█████████ | 5116/5616 [1:05:10<06:07,  1.36it/s, gpt_loss=0.0168, lr=5.93e-7] 
+
Train step of epoch 0:  91%|█████████ | 5117/5616 [1:05:10<06:06,  1.36it/s, gpt_loss=0.0168, lr=5.93e-7]
+
Train step of epoch 0:  91%|█████████ | 5117/5616 [1:05:11<06:06,  1.36it/s, gpt_loss=0.00266, lr=5.93e-7]
+
Train step of epoch 0:  91%|█████████ | 5118/5616 [1:05:11<06:03,  1.37it/s, gpt_loss=0.00266, lr=5.93e-7]
+
Train step of epoch 0:  91%|█████████ | 5118/5616 [1:05:11<06:03,  1.37it/s, gpt_loss=0.00661, lr=5.93e-7]
+
Train step of epoch 0:  91%|█████████ | 5119/5616 [1:05:11<05:49,  1.42it/s, gpt_loss=0.00661, lr=5.93e-7]
+
Train step of epoch 0:  91%|█████████ | 5119/5616 [1:05:12<05:49,  1.42it/s, gpt_loss=0.0298, lr=5.92e-7] 
+
Train step of epoch 0:  91%|█████████ | 5120/5616 [1:05:12<05:50,  1.41it/s, gpt_loss=0.0298, lr=5.92e-7]
+
Train step of epoch 0:  91%|█████████ | 5120/5616 [1:05:13<05:50,  1.41it/s, gpt_loss=0.0163, lr=5.92e-7]
+
Train step of epoch 0:  91%|█████████ | 5121/5616 [1:05:13<05:54,  1.40it/s, gpt_loss=0.0163, lr=5.92e-7]
+
Train step of epoch 0:  91%|█████████ | 5121/5616 [1:05:14<05:54,  1.40it/s, gpt_loss=0.0526, lr=5.92e-7]
+
Train step of epoch 0:  91%|█████████ | 5122/5616 [1:05:14<06:18,  1.31it/s, gpt_loss=0.0526, lr=5.92e-7]
+
Train step of epoch 0:  91%|█████████ | 5122/5616 [1:05:14<06:18,  1.31it/s, gpt_loss=0.0129, lr=5.92e-7]
+
Train step of epoch 0:  91%|█████████ | 5123/5616 [1:05:14<05:59,  1.37it/s, gpt_loss=0.0129, lr=5.92e-7]
+
Train step of epoch 0:  91%|█████████ | 5123/5616 [1:05:15<05:59,  1.37it/s, gpt_loss=0.0043, lr=5.9e-7] 
+
Train step of epoch 0:  91%|█████████ | 5124/5616 [1:05:15<06:00,  1.37it/s, gpt_loss=0.0043, lr=5.9e-7]
+
Train step of epoch 0:  91%|█████████ | 5124/5616 [1:05:16<06:00,  1.37it/s, gpt_loss=0.0369, lr=5.9e-7]
+
Train step of epoch 0:  91%|█████████▏| 5125/5616 [1:05:16<06:53,  1.19it/s, gpt_loss=0.0369, lr=5.9e-7]
+
Train step of epoch 0:  91%|█████████▏| 5125/5616 [1:05:17<06:53,  1.19it/s, gpt_loss=0.0101, lr=5.9e-7]
+
Train step of epoch 0:  91%|█████████▏| 5126/5616 [1:05:17<06:37,  1.23it/s, gpt_loss=0.0101, lr=5.9e-7]
+
Train step of epoch 0:  91%|█████████▏| 5126/5616 [1:05:18<06:37,  1.23it/s, gpt_loss=0.14, lr=5.9e-7]  
+
Train step of epoch 0:  91%|█████████▏| 5127/5616 [1:05:18<06:37,  1.23it/s, gpt_loss=0.14, lr=5.9e-7]
+
Train step of epoch 0:  91%|█████████▏| 5127/5616 [1:05:18<06:37,  1.23it/s, gpt_loss=0.0502, lr=5.89e-7]
+
Train step of epoch 0:  91%|█████████▏| 5128/5616 [1:05:18<06:39,  1.22it/s, gpt_loss=0.0502, lr=5.89e-7]
+
Train step of epoch 0:  91%|█████████▏| 5128/5616 [1:05:19<06:39,  1.22it/s, gpt_loss=0.00666, lr=5.89e-7]
+
Train step of epoch 0:  91%|█████████▏| 5129/5616 [1:05:19<06:31,  1.24it/s, gpt_loss=0.00666, lr=5.89e-7]
+
Train step of epoch 0:  91%|█████████▏| 5129/5616 [1:05:20<06:31,  1.24it/s, gpt_loss=0.0393, lr=5.89e-7] 
+
Train step of epoch 0:  91%|█████████▏| 5130/5616 [1:05:20<06:30,  1.25it/s, gpt_loss=0.0393, lr=5.89e-7]
+
Train step of epoch 0:  91%|█████████▏| 5130/5616 [1:05:21<06:30,  1.25it/s, gpt_loss=0.0256, lr=5.89e-7]
+
Train step of epoch 0:  91%|█████████▏| 5131/5616 [1:05:21<06:13,  1.30it/s, gpt_loss=0.0256, lr=5.89e-7]
+
Train step of epoch 0:  91%|█████████▏| 5131/5616 [1:05:21<06:13,  1.30it/s, gpt_loss=0.0208, lr=5.87e-7]
+
Train step of epoch 0:  91%|█████████▏| 5132/5616 [1:05:21<06:12,  1.30it/s, gpt_loss=0.0208, lr=5.87e-7]
+
Train step of epoch 0:  91%|█████████▏| 5132/5616 [1:05:22<06:12,  1.30it/s, gpt_loss=0.0383, lr=5.87e-7]
+
Train step of epoch 0:  91%|█████████▏| 5133/5616 [1:05:22<06:04,  1.32it/s, gpt_loss=0.0383, lr=5.87e-7]
+
Train step of epoch 0:  91%|█████████▏| 5133/5616 [1:05:23<06:04,  1.32it/s, gpt_loss=0.0736, lr=5.87e-7]
+
Train step of epoch 0:  91%|█████████▏| 5134/5616 [1:05:23<06:02,  1.33it/s, gpt_loss=0.0736, lr=5.87e-7]
+
Train step of epoch 0:  91%|█████████▏| 5134/5616 [1:05:24<06:02,  1.33it/s, gpt_loss=0.0185, lr=5.87e-7]
+
Train step of epoch 0:  91%|█████████▏| 5135/5616 [1:05:24<05:53,  1.36it/s, gpt_loss=0.0185, lr=5.87e-7]
+
Train step of epoch 0:  91%|█████████▏| 5135/5616 [1:05:24<05:53,  1.36it/s, gpt_loss=0.0643, lr=5.86e-7]
+
Train step of epoch 0:  91%|█████████▏| 5136/5616 [1:05:24<06:11,  1.29it/s, gpt_loss=0.0643, lr=5.86e-7]
+
Train step of epoch 0:  91%|█████████▏| 5136/5616 [1:05:25<06:11,  1.29it/s, gpt_loss=0.0558, lr=5.86e-7]
+
Train step of epoch 0:  91%|█████████▏| 5137/5616 [1:05:25<05:55,  1.35it/s, gpt_loss=0.0558, lr=5.86e-7]
+
Train step of epoch 0:  91%|█████████▏| 5137/5616 [1:05:26<05:55,  1.35it/s, gpt_loss=0.0025, lr=5.86e-7]
+
Train step of epoch 0:  91%|█████████▏| 5138/5616 [1:05:26<05:55,  1.34it/s, gpt_loss=0.0025, lr=5.86e-7]
+
Train step of epoch 0:  91%|█████████▏| 5138/5616 [1:05:27<05:55,  1.34it/s, gpt_loss=0.0854, lr=5.86e-7]
+
Train step of epoch 0:  92%|█████████▏| 5139/5616 [1:05:27<06:02,  1.32it/s, gpt_loss=0.0854, lr=5.86e-7]
+
Train step of epoch 0:  92%|█████████▏| 5139/5616 [1:05:28<06:02,  1.32it/s, gpt_loss=0.0232, lr=5.84e-7]
+
Train step of epoch 0:  92%|█████████▏| 5140/5616 [1:05:28<06:27,  1.23it/s, gpt_loss=0.0232, lr=5.84e-7]
+
Train step of epoch 0:  92%|█████████▏| 5140/5616 [1:05:28<06:27,  1.23it/s, gpt_loss=0.00716, lr=5.84e-7]
+
Train step of epoch 0:  92%|█████████▏| 5141/5616 [1:05:28<06:05,  1.30it/s, gpt_loss=0.00716, lr=5.84e-7]
+
Train step of epoch 0:  92%|█████████▏| 5141/5616 [1:05:29<06:05,  1.30it/s, gpt_loss=0.0478, lr=5.84e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5142/5616 [1:05:29<06:06,  1.29it/s, gpt_loss=0.0478, lr=5.84e-7]
+
Train step of epoch 0:  92%|█████████▏| 5142/5616 [1:05:30<06:06,  1.29it/s, gpt_loss=0.0362, lr=5.84e-7]
+
Train step of epoch 0:  92%|█���███████▏| 5143/5616 [1:05:30<06:12,  1.27it/s, gpt_loss=0.0362, lr=5.84e-7]
+
Train step of epoch 0:  92%|█████████▏| 5143/5616 [1:05:31<06:12,  1.27it/s, gpt_loss=0.00488, lr=5.83e-7]
+
Train step of epoch 0:  92%|█████████▏| 5144/5616 [1:05:31<06:03,  1.30it/s, gpt_loss=0.00488, lr=5.83e-7]
+
Train step of epoch 0:  92%|█████████▏| 5144/5616 [1:05:31<06:03,  1.30it/s, gpt_loss=0.0112, lr=5.83e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5145/5616 [1:05:31<05:53,  1.33it/s, gpt_loss=0.0112, lr=5.83e-7]
+
Train step of epoch 0:  92%|█████████▏| 5145/5616 [1:05:32<05:53,  1.33it/s, gpt_loss=0.0277, lr=5.83e-7]
+
Train step of epoch 0:  92%|█████████▏| 5146/5616 [1:05:32<06:21,  1.23it/s, gpt_loss=0.0277, lr=5.83e-7]
+
Train step of epoch 0:  92%|█████████▏| 5146/5616 [1:05:33<06:21,  1.23it/s, gpt_loss=0.168, lr=5.83e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5147/5616 [1:05:33<06:17,  1.24it/s, gpt_loss=0.168, lr=5.83e-7]
+
Train step of epoch 0:  92%|█████████▏| 5147/5616 [1:05:34<06:17,  1.24it/s, gpt_loss=0.0119, lr=5.82e-7]
+
Train step of epoch 0:  92%|█████████▏| 5148/5616 [1:05:34<06:12,  1.26it/s, gpt_loss=0.0119, lr=5.82e-7]
+
Train step of epoch 0:  92%|█████████▏| 5148/5616 [1:05:35<06:12,  1.26it/s, gpt_loss=0.0869, lr=5.82e-7]
+
Train step of epoch 0:  92%|█████████▏| 5149/5616 [1:05:35<05:59,  1.30it/s, gpt_loss=0.0869, lr=5.82e-7]
+
Train step of epoch 0:  92%|█████████▏| 5149/5616 [1:05:35<05:59,  1.30it/s, gpt_loss=0.00746, lr=5.82e-7]
+
Train step of epoch 0:  92%|█████████▏| 5150/5616 [1:05:35<06:00,  1.29it/s, gpt_loss=0.00746, lr=5.82e-7]
+
Train step of epoch 0:  92%|█████████▏| 5150/5616 [1:05:36<06:00,  1.29it/s, gpt_loss=0.0374, lr=5.82e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5151/5616 [1:05:36<05:41,  1.36it/s, gpt_loss=0.0374, lr=5.82e-7]
+
Train step of epoch 0:  92%|█████████▏| 5151/5616 [1:05:37<05:41,  1.36it/s, gpt_loss=0.0232, lr=5.8e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5152/5616 [1:05:37<05:42,  1.36it/s, gpt_loss=0.0232, lr=5.8e-7]
+
Train step of epoch 0:  92%|█████████▏| 5152/5616 [1:05:37<05:42,  1.36it/s, gpt_loss=0.0536, lr=5.8e-7]
+
Train step of epoch 0:  92%|█████████▏| 5153/5616 [1:05:37<05:40,  1.36it/s, gpt_loss=0.0536, lr=5.8e-7]
+
Train step of epoch 0:  92%|█████████▏| 5153/5616 [1:05:38<05:40,  1.36it/s, gpt_loss=0.0311, lr=5.8e-7]
+
Train step of epoch 0:  92%|█████████▏| 5154/5616 [1:05:38<05:41,  1.35it/s, gpt_loss=0.0311, lr=5.8e-7]
+
Train step of epoch 0:  92%|█████████▏| 5154/5616 [1:05:39<05:41,  1.35it/s, gpt_loss=0.0592, lr=5.8e-7]
+
Train step of epoch 0:  92%|█████████▏| 5155/5616 [1:05:39<05:58,  1.29it/s, gpt_loss=0.0592, lr=5.8e-7]
+
Train step of epoch 0:  92%|█████████▏| 5155/5616 [1:05:40<05:58,  1.29it/s, gpt_loss=0.00922, lr=5.79e-7]
+
Train step of epoch 0:  92%|█████████▏| 5156/5616 [1:05:40<05:57,  1.29it/s, gpt_loss=0.00922, lr=5.79e-7]
+
Train step of epoch 0:  92%|█████████▏| 5156/5616 [1:05:41<05:57,  1.29it/s, gpt_loss=0.0331, lr=5.79e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5157/5616 [1:05:41<05:51,  1.31it/s, gpt_loss=0.0331, lr=5.79e-7]
+
Train step of epoch 0:  92%|█████████▏| 5157/5616 [1:05:41<05:51,  1.31it/s, gpt_loss=0.046, lr=5.79e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5158/5616 [1:05:41<05:39,  1.35it/s, gpt_loss=0.046, lr=5.79e-7]
+
Train step of epoch 0:  92%|█████████▏| 5158/5616 [1:05:42<05:39,  1.35it/s, gpt_loss=0.00465, lr=5.79e-7]
+
Train step of epoch 0:  92%|█████████▏| 5159/5616 [1:05:42<05:23,  1.41it/s, gpt_loss=0.00465, lr=5.79e-7]
+
Train step of epoch 0:  92%|█████████▏| 5159/5616 [1:05:43<05:23,  1.41it/s, gpt_loss=0.0702, lr=5.77e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5160/5616 [1:05:43<06:03,  1.25it/s, gpt_loss=0.0702, lr=5.77e-7]
+
Train step of epoch 0:  92%|█████████▏| 5160/5616 [1:05:44<06:03,  1.25it/s, gpt_loss=0.00621, lr=5.77e-7]
+
Train step of epoch 0:  92%|█████████▏| 5161/5616 [1:05:44<05:46,  1.31it/s, gpt_loss=0.00621, lr=5.77e-7]
+
Train step of epoch 0:  92%|█████████▏| 5161/5616 [1:05:44<05:46,  1.31it/s, gpt_loss=0.0217, lr=5.77e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5162/5616 [1:05:44<05:48,  1.30it/s, gpt_loss=0.0217, lr=5.77e-7]
+
Train step of epoch 0:  92%|█████████▏| 5162/5616 [1:05:45<05:48,  1.30it/s, gpt_loss=0.00257, lr=5.77e-7]
+
Train step of epoch 0:  92%|█████████▏| 5163/5616 [1:05:45<05:34,  1.35it/s, gpt_loss=0.00257, lr=5.77e-7]
+
Train step of epoch 0:  92%|█████████▏| 5163/5616 [1:05:46<05:34,  1.35it/s, gpt_loss=0.0224, lr=5.76e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5164/5616 [1:05:46<05:34,  1.35it/s, gpt_loss=0.0224, lr=5.76e-7]
+
Train step of epoch 0:  92%|█████████▏| 5164/5616 [1:05:46<05:34,  1.35it/s, gpt_loss=0.0693, lr=5.76e-7]
+
Train step of epoch 0:  92%|█████████▏| 5165/5616 [1:05:46<05:23,  1.39it/s, gpt_loss=0.0693, lr=5.76e-7]
+
Train step of epoch 0:  92%|█████████▏| 5165/5616 [1:05:47<05:23,  1.39it/s, gpt_loss=0.0623, lr=5.76e-7]
+
Train step of epoch 0:  92%|█████████▏| 5166/5616 [1:05:47<05:32,  1.35it/s, gpt_loss=0.0623, lr=5.76e-7]
+
Train step of epoch 0:  92%|█████████▏| 5166/5616 [1:05:48<05:32,  1.35it/s, gpt_loss=0.122, lr=5.76e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5167/5616 [1:05:48<05:25,  1.38it/s, gpt_loss=0.122, lr=5.76e-7]
+
Train step of epoch 0:  92%|█████████▏| 5167/5616 [1:05:49<05:25,  1.38it/s, gpt_loss=0.021, lr=5.75e-7]
+
Train step of epoch 0:  92%|█████████▏| 5168/5616 [1:05:49<05:32,  1.35it/s, gpt_loss=0.021, lr=5.75e-7]
+
Train step of epoch 0:  92%|█████████▏| 5168/5616 [1:05:49<05:32,  1.35it/s, gpt_loss=0.0147, lr=5.75e-7]
+
Train step of epoch 0:  92%|█████████▏| 5169/5616 [1:05:49<05:27,  1.37it/s, gpt_loss=0.0147, lr=5.75e-7]
+
Train step of epoch 0:  92%|█████████▏| 5169/5616 [1:05:50<05:27,  1.37it/s, gpt_loss=0.0138, lr=5.75e-7]
+
Train step of epoch 0:  92%|█████████▏| 5170/5616 [1:05:50<05:27,  1.36it/s, gpt_loss=0.0138, lr=5.75e-7]
+
Train step of epoch 0:  92%|█████████▏| 5170/5616 [1:05:51<05:27,  1.36it/s, gpt_loss=0.0155, lr=5.75e-7]
+
Train step of epoch 0:  92%|█████████▏| 5171/5616 [1:05:51<05:38,  1.32it/s, gpt_loss=0.0155, lr=5.75e-7]
+
Train step of epoch 0:  92%|█████████▏| 5171/5616 [1:05:52<05:38,  1.32it/s, gpt_loss=0.05, lr=5.73e-7]  
+
Train step of epoch 0:  92%|█████████▏| 5172/5616 [1:05:52<05:43,  1.29it/s, gpt_loss=0.05, lr=5.73e-7]
+
Train step of epoch 0:  92%|█████████▏| 5172/5616 [1:05:52<05:43,  1.29it/s, gpt_loss=0.0276, lr=5.73e-7]
+
Train step of epoch 0:  92%|█████████▏| 5173/5616 [1:05:52<05:30,  1.34it/s, gpt_loss=0.0276, lr=5.73e-7]
+
Train step of epoch 0:  92%|█████████▏| 5173/5616 [1:05:54<05:30,  1.34it/s, gpt_loss=0.0582, lr=5.73e-7]
+
Train step of epoch 0:  92%|█████████▏| 5174/5616 [1:05:54<06:13,  1.18it/s, gpt_loss=0.0582, lr=5.73e-7]
+
Train step of epoch 0:  92%|█████████▏| 5174/5616 [1:05:54<06:13,  1.18it/s, gpt_loss=0.0525, lr=5.73e-7]
+
Train step of epoch 0:  92%|█████████▏| 5175/5616 [1:05:54<05:56,  1.24it/s, gpt_loss=0.0525, lr=5.73e-7]
+
Train step of epoch 0:  92%|█████████▏| 5175/5616 [1:05:55<05:56,  1.24it/s, gpt_loss=0.0209, lr=5.72e-7]
+
Train step of epoch 0:  92%|█████████▏| 5176/5616 [1:05:55<05:47,  1.27it/s, gpt_loss=0.0209, lr=5.72e-7]
+
Train step of epoch 0:  92%|█████████▏| 5176/5616 [1:05:56<05:47,  1.27it/s, gpt_loss=0.0306, lr=5.72e-7]
+
Train step of epoch 0:  92%|█████████▏| 5177/5616 [1:05:56<05:32,  1.32it/s, gpt_loss=0.0306, lr=5.72e-7]
+
Train step of epoch 0:  92%|█████████▏| 5177/5616 [1:05:56<05:32,  1.32it/s, gpt_loss=0.0847, lr=5.72e-7]
+
Train step of epoch 0:  92%|█████████▏| 5178/5616 [1:05:56<05:26,  1.34it/s, gpt_loss=0.0847, lr=5.72e-7]
+
Train step of epoch 0:  92%|█████████▏| 5178/5616 [1:05:57<05:26,  1.34it/s, gpt_loss=0.0254, lr=5.72e-7]
+
Train step of epoch 0:  92%|█████████▏| 5179/5616 [1:05:57<05:54,  1.23it/s, gpt_loss=0.0254, lr=5.72e-7]
+
Train step of epoch 0:  92%|█████████▏| 5179/5616 [1:05:58<05:54,  1.23it/s, gpt_loss=0.0505, lr=5.71e-7]
+
Train step of epoch 0:  92%|█████████▏| 5180/5616 [1:05:58<06:14,  1.17it/s, gpt_loss=0.0505, lr=5.71e-7]
+
Train step of epoch 0:  92%|█████████▏| 5180/5616 [1:05:59<06:14,  1.17it/s, gpt_loss=0.0409, lr=5.71e-7]
+
Train step of epoch 0:  92%|█████████▏| 5181/5616 [1:05:59<05:58,  1.21it/s, gpt_loss=0.0409, lr=5.71e-7]
+
Train step of epoch 0:  92%|█████████▏| 5181/5616 [1:06:00<05:58,  1.21it/s, gpt_loss=0.0114, lr=5.71e-7]
+
Train step of epoch 0:  92%|███████���█▏| 5182/5616 [1:06:00<05:44,  1.26it/s, gpt_loss=0.0114, lr=5.71e-7]
+
Train step of epoch 0:  92%|█████████▏| 5182/5616 [1:06:01<05:44,  1.26it/s, gpt_loss=0.0117, lr=5.71e-7]
+
Train step of epoch 0:  92%|█████████▏| 5183/5616 [1:06:01<05:33,  1.30it/s, gpt_loss=0.0117, lr=5.71e-7]
+
Train step of epoch 0:  92%|█████████▏| 5183/5616 [1:06:01<05:33,  1.30it/s, gpt_loss=0.00824, lr=5.7e-7]
+
Train step of epoch 0:  92%|█████████▏| 5184/5616 [1:06:01<05:43,  1.26it/s, gpt_loss=0.00824, lr=5.7e-7]
+
Train step of epoch 0:  92%|█████████▏| 5184/5616 [1:06:02<05:43,  1.26it/s, gpt_loss=0.0133, lr=5.7e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5185/5616 [1:06:02<05:45,  1.25it/s, gpt_loss=0.0133, lr=5.7e-7]
+
Train step of epoch 0:  92%|█████████▏| 5185/5616 [1:06:03<05:45,  1.25it/s, gpt_loss=0.00694, lr=5.7e-7]
+
Train step of epoch 0:  92%|█████████▏| 5186/5616 [1:06:03<05:52,  1.22it/s, gpt_loss=0.00694, lr=5.7e-7]
+
Train step of epoch 0:  92%|█████████▏| 5186/5616 [1:06:04<05:52,  1.22it/s, gpt_loss=0.00573, lr=5.7e-7]
+
Train step of epoch 0:  92%|█████████▏| 5187/5616 [1:06:04<05:30,  1.30it/s, gpt_loss=0.00573, lr=5.7e-7]
+
Train step of epoch 0:  92%|█████████▏| 5187/5616 [1:06:05<05:30,  1.30it/s, gpt_loss=0.0119, lr=5.68e-7]
+
Train step of epoch 0:  92%|█████████▏| 5188/5616 [1:06:05<05:33,  1.28it/s, gpt_loss=0.0119, lr=5.68e-7]
+
Train step of epoch 0:  92%|█████████▏| 5188/5616 [1:06:05<05:33,  1.28it/s, gpt_loss=0.0133, lr=5.68e-7]
+
Train step of epoch 0:  92%|█████████▏| 5189/5616 [1:06:05<05:19,  1.34it/s, gpt_loss=0.0133, lr=5.68e-7]
+
Train step of epoch 0:  92%|█████████▏| 5189/5616 [1:06:06<05:19,  1.34it/s, gpt_loss=0.0122, lr=5.68e-7]
+
Train step of epoch 0:  92%|█████████▏| 5190/5616 [1:06:06<05:10,  1.37it/s, gpt_loss=0.0122, lr=5.68e-7]
+
Train step of epoch 0:  92%|█████████▏| 5190/5616 [1:06:07<05:10,  1.37it/s, gpt_loss=0.00333, lr=5.68e-7]
+
Train step of epoch 0:  92%|█████████▏| 5191/5616 [1:06:07<05:14,  1.35it/s, gpt_loss=0.00333, lr=5.68e-7]
+
Train step of epoch 0:  92%|█████████▏| 5191/5616 [1:06:07<05:14,  1.35it/s, gpt_loss=0.0529, lr=5.67e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5192/5616 [1:06:07<05:14,  1.35it/s, gpt_loss=0.0529, lr=5.67e-7]
+
Train step of epoch 0:  92%|█████████▏| 5192/5616 [1:06:08<05:14,  1.35it/s, gpt_loss=0.179, lr=5.67e-7] 
+
Train step of epoch 0:  92%|█████████▏| 5193/5616 [1:06:08<05:33,  1.27it/s, gpt_loss=0.179, lr=5.67e-7]
+
Train step of epoch 0:  92%|█████████▏| 5193/5616 [1:06:09<05:33,  1.27it/s, gpt_loss=0.0199, lr=5.67e-7]
+
Train step of epoch 0:  92%|█████████▏| 5194/5616 [1:06:09<05:20,  1.32it/s, gpt_loss=0.0199, lr=5.67e-7]
+
Train step of epoch 0:  92%|█████████▏| 5194/5616 [1:06:10<05:20,  1.32it/s, gpt_loss=0.0176, lr=5.67e-7]
+
Train step of epoch 0:  93%|█████████▎| 5195/5616 [1:06:10<05:06,  1.38it/s, gpt_loss=0.0176, lr=5.67e-7]
+
Train step of epoch 0:  93%|█████████▎| 5195/5616 [1:06:11<05:06,  1.38it/s, gpt_loss=0.0095, lr=5.66e-7]
+
Train step of epoch 0:  93%|█████████▎| 5196/5616 [1:06:11<06:04,  1.15it/s, gpt_loss=0.0095, lr=5.66e-7]
+
Train step of epoch 0:  93%|█████████▎| 5196/5616 [1:06:12<06:04,  1.15it/s, gpt_loss=0.0364, lr=5.66e-7]
+
Train step of epoch 0:  93%|█████████▎| 5197/5616 [1:06:12<05:57,  1.17it/s, gpt_loss=0.0364, lr=5.66e-7]
+
Train step of epoch 0:  93%|█████████▎| 5197/5616 [1:06:12<05:57,  1.17it/s, gpt_loss=0.0458, lr=5.66e-7]
+
Train step of epoch 0:  93%|█████████▎| 5198/5616 [1:06:12<05:35,  1.24it/s, gpt_loss=0.0458, lr=5.66e-7]
+
Train step of epoch 0:  93%|█████████▎| 5198/5616 [1:06:13<05:35,  1.24it/s, gpt_loss=0.00613, lr=5.66e-7]
+
Train step of epoch 0:  93%|█████████▎| 5199/5616 [1:06:13<05:25,  1.28it/s, gpt_loss=0.00613, lr=5.66e-7][2025-06-30 07:44:22,825] [INFO] [logging.py:128:log_dist] [Rank 0] step=1300, skipped=0, lr=[5.645232131983643e-07, 5.645232131983643e-07], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 07:44:22,826] [INFO] [timer.py:264:stop] epoch=0/micro_step=5200/global_step=1300, RunningAvgSamplesPerSec=42.411274443232784, CurrSamplesPerSec=43.79982691967231, MemAllocated=15.92GB, MaxMemAllocated=54.86GB
+
+
Train step of epoch 0:  93%|█████████▎| 5199/5616 [1:06:14<05:25,  1.28it/s, gpt_loss=0.0205, lr=5.65e-7] 
+
Train step of epoch 0:  93%|████████��▎| 5200/5616 [1:06:14<05:18,  1.31it/s, gpt_loss=0.0205, lr=5.65e-7]
+
Train step of epoch 0:  93%|█████████▎| 5200/5616 [1:06:15<05:18,  1.31it/s, gpt_loss=0.0332, lr=5.65e-7]
+
Train step of epoch 0:  93%|█████████▎| 5201/5616 [1:06:15<05:30,  1.25it/s, gpt_loss=0.0332, lr=5.65e-7]
+
Train step of epoch 0:  93%|█████████▎| 5201/5616 [1:06:15<05:30,  1.25it/s, gpt_loss=0.0296, lr=5.65e-7]
+
Train step of epoch 0:  93%|█████████▎| 5202/5616 [1:06:15<05:15,  1.31it/s, gpt_loss=0.0296, lr=5.65e-7]
+
Train step of epoch 0:  93%|█████████▎| 5202/5616 [1:06:16<05:15,  1.31it/s, gpt_loss=0.0579, lr=5.65e-7]
+
Train step of epoch 0:  93%|█████████▎| 5203/5616 [1:06:16<05:03,  1.36it/s, gpt_loss=0.0579, lr=5.65e-7]
+
Train step of epoch 0:  93%|█████████▎| 5203/5616 [1:06:17<05:03,  1.36it/s, gpt_loss=0.00486, lr=5.63e-7]
+
Train step of epoch 0:  93%|█████████▎| 5204/5616 [1:06:17<05:01,  1.37it/s, gpt_loss=0.00486, lr=5.63e-7]
+
Train step of epoch 0:  93%|█████████▎| 5204/5616 [1:06:17<05:01,  1.37it/s, gpt_loss=0.0125, lr=5.63e-7] 
+
Train step of epoch 0:  93%|█████████▎| 5205/5616 [1:06:17<04:54,  1.40it/s, gpt_loss=0.0125, lr=5.63e-7]
+
Train step of epoch 0:  93%|█████████▎| 5205/5616 [1:06:18<04:54,  1.40it/s, gpt_loss=0.022, lr=5.63e-7] 
+
Train step of epoch 0:  93%|█████████▎| 5206/5616 [1:06:18<04:48,  1.42it/s, gpt_loss=0.022, lr=5.63e-7]
+
Train step of epoch 0:  93%|█████████▎| 5206/5616 [1:06:19<04:48,  1.42it/s, gpt_loss=0.0172, lr=5.63e-7]
+
Train step of epoch 0:  93%|█████████▎| 5207/5616 [1:06:19<05:00,  1.36it/s, gpt_loss=0.0172, lr=5.63e-7]
+
Train step of epoch 0:  93%|█████████▎| 5207/5616 [1:06:20<05:00,  1.36it/s, gpt_loss=0.0901, lr=5.62e-7]
+
Train step of epoch 0:  93%|█████████▎| 5208/5616 [1:06:20<05:05,  1.34it/s, gpt_loss=0.0901, lr=5.62e-7]
+
Train step of epoch 0:  93%|█████████▎| 5208/5616 [1:06:20<05:05,  1.34it/s, gpt_loss=0.064, lr=5.62e-7] 
+
Train step of epoch 0:  93%|█████████▎| 5209/5616 [1:06:20<04:58,  1.36it/s, gpt_loss=0.064, lr=5.62e-7]
+
Train step of epoch 0:  93%|█████████▎| 5209/5616 [1:06:21<04:58,  1.36it/s, gpt_loss=0.00845, lr=5.62e-7]
+
Train step of epoch 0:  93%|█████████▎| 5210/5616 [1:06:21<05:00,  1.35it/s, gpt_loss=0.00845, lr=5.62e-7]
+
Train step of epoch 0:  93%|█████████▎| 5210/5616 [1:06:22<05:00,  1.35it/s, gpt_loss=0.012, lr=5.62e-7]  
+
Train step of epoch 0:  93%|█████████▎| 5211/5616 [1:06:22<04:57,  1.36it/s, gpt_loss=0.012, lr=5.62e-7]
+
Train step of epoch 0:  93%|█████████▎| 5211/5616 [1:06:23<04:57,  1.36it/s, gpt_loss=0.0183, lr=5.61e-7]
+
Train step of epoch 0:  93%|█████████▎| 5212/5616 [1:06:23<05:05,  1.32it/s, gpt_loss=0.0183, lr=5.61e-7]
+
Train step of epoch 0:  93%|█████████▎| 5212/5616 [1:06:23<05:05,  1.32it/s, gpt_loss=0.00961, lr=5.61e-7]
+
Train step of epoch 0:  93%|█████████▎| 5213/5616 [1:06:23<05:02,  1.33it/s, gpt_loss=0.00961, lr=5.61e-7]
+
Train step of epoch 0:  93%|█████████▎| 5213/5616 [1:06:24<05:02,  1.33it/s, gpt_loss=0.0268, lr=5.61e-7] 
+
Train step of epoch 0:  93%|█████████▎| 5214/5616 [1:06:24<05:19,  1.26it/s, gpt_loss=0.0268, lr=5.61e-7]
+
Train step of epoch 0:  93%|█████████▎| 5214/5616 [1:06:25<05:19,  1.26it/s, gpt_loss=0.0211, lr=5.61e-7]
+
Train step of epoch 0:  93%|█████████▎| 5215/5616 [1:06:25<05:03,  1.32it/s, gpt_loss=0.0211, lr=5.61e-7]
+
Train step of epoch 0:  93%|█████████▎| 5215/5616 [1:06:26<05:03,  1.32it/s, gpt_loss=0.0111, lr=5.6e-7] 
+
Train step of epoch 0:  93%|█████████▎| 5216/5616 [1:06:26<05:02,  1.32it/s, gpt_loss=0.0111, lr=5.6e-7]
+
Train step of epoch 0:  93%|█████████▎| 5216/5616 [1:06:26<05:02,  1.32it/s, gpt_loss=0.0202, lr=5.6e-7]
+
Train step of epoch 0:  93%|█████████▎| 5217/5616 [1:06:26<04:56,  1.35it/s, gpt_loss=0.0202, lr=5.6e-7]
+
Train step of epoch 0:  93%|█████████▎| 5217/5616 [1:06:27<04:56,  1.35it/s, gpt_loss=0.0127, lr=5.6e-7]
+
Train step of epoch 0:  93%|█████████▎| 5218/5616 [1:06:27<04:56,  1.34it/s, gpt_loss=0.0127, lr=5.6e-7]
+
Train step of epoch 0:  93%|█████████▎| 5218/5616 [1:06:28<04:56,  1.34it/s, gpt_loss=0.0538, lr=5.6e-7]
+
Train step of epoch 0:  93%|█████████▎| 5219/5616 [1:06:28<04:51,  1.36it/s, gpt_loss=0.0538, lr=5.6e-7]
+
Train step of epoch 0:  93%|█████████▎| 5219/5616 [1:06:29<04:51,  1.36it/s, gpt_loss=0.00581, lr=5.58e-7]
+
Train step of epoch 0:  93%|█████████▎| 5220/5616 [1:06:29<04:55,  1.34it/s, gpt_loss=0.00581, lr=5.58e-7]
+
Train step of epoch 0:  93%|█████████▎| 5220/5616 [1:06:29<04:55,  1.34it/s, gpt_loss=0.0339, lr=5.58e-7] 
+
Train step of epoch 0:  93%|█████████▎| 5221/5616 [1:06:29<04:48,  1.37it/s, gpt_loss=0.0339, lr=5.58e-7]
+
Train step of epoch 0:  93%|█████████▎| 5221/5616 [1:06:30<04:48,  1.37it/s, gpt_loss=0.0168, lr=5.58e-7]
+
Train step of epoch 0:  93%|█████████▎| 5222/5616 [1:06:30<04:42,  1.40it/s, gpt_loss=0.0168, lr=5.58e-7]
+
Train step of epoch 0:  93%|█████████▎| 5222/5616 [1:06:31<04:42,  1.40it/s, gpt_loss=0.0142, lr=5.58e-7]
+
Train step of epoch 0:  93%|█████████▎| 5223/5616 [1:06:31<04:45,  1.38it/s, gpt_loss=0.0142, lr=5.58e-7]
+
Train step of epoch 0:  93%|█████████▎| 5223/5616 [1:06:32<04:45,  1.38it/s, gpt_loss=0.0177, lr=5.57e-7]
+
Train step of epoch 0:  93%|█████████▎| 5224/5616 [1:06:32<05:25,  1.20it/s, gpt_loss=0.0177, lr=5.57e-7]
+
Train step of epoch 0:  93%|█████████▎| 5224/5616 [1:06:33<05:25,  1.20it/s, gpt_loss=0.0315, lr=5.57e-7]
+
Train step of epoch 0:  93%|█████████▎| 5225/5616 [1:06:33<05:09,  1.26it/s, gpt_loss=0.0315, lr=5.57e-7]
+
Train step of epoch 0:  93%|█████████▎| 5225/5616 [1:06:33<05:09,  1.26it/s, gpt_loss=0.00259, lr=5.57e-7]
+
Train step of epoch 0:  93%|█████████▎| 5226/5616 [1:06:33<04:56,  1.32it/s, gpt_loss=0.00259, lr=5.57e-7]
+
Train step of epoch 0:  93%|█████████▎| 5226/5616 [1:06:34<04:56,  1.32it/s, gpt_loss=0.0244, lr=5.57e-7] 
+
Train step of epoch 0:  93%|█████████▎| 5227/5616 [1:06:34<05:02,  1.29it/s, gpt_loss=0.0244, lr=5.57e-7]
+
Train step of epoch 0:  93%|█████████▎| 5227/5616 [1:06:35<05:02,  1.29it/s, gpt_loss=0.035, lr=5.56e-7] 
+
Train step of epoch 0:  93%|█████████▎| 5228/5616 [1:06:35<05:31,  1.17it/s, gpt_loss=0.035, lr=5.56e-7]
+
Train step of epoch 0:  93%|█████████▎| 5228/5616 [1:06:36<05:31,  1.17it/s, gpt_loss=0.00661, lr=5.56e-7]
+
Train step of epoch 0:  93%|█████████▎| 5229/5616 [1:06:36<05:27,  1.18it/s, gpt_loss=0.00661, lr=5.56e-7]
+
Train step of epoch 0:  93%|█████████▎| 5229/5616 [1:06:37<05:27,  1.18it/s, gpt_loss=0.0279, lr=5.56e-7] 
+
Train step of epoch 0:  93%|█████████▎| 5230/5616 [1:06:37<05:07,  1.26it/s, gpt_loss=0.0279, lr=5.56e-7]
+
Train step of epoch 0:  93%|█████████▎| 5230/5616 [1:06:37<05:07,  1.26it/s, gpt_loss=0.00406, lr=5.56e-7]
+
Train step of epoch 0:  93%|█████████▎| 5231/5616 [1:06:37<05:07,  1.25it/s, gpt_loss=0.00406, lr=5.56e-7]
+
Train step of epoch 0:  93%|█████████▎| 5231/5616 [1:06:38<05:07,  1.25it/s, gpt_loss=0.0101, lr=5.55e-7] 
+
Train step of epoch 0:  93%|█████████▎| 5232/5616 [1:06:38<05:15,  1.22it/s, gpt_loss=0.0101, lr=5.55e-7]
+
Train step of epoch 0:  93%|█████████▎| 5232/5616 [1:06:39<05:15,  1.22it/s, gpt_loss=0.00629, lr=5.55e-7]
+
Train step of epoch 0:  93%|█████████▎| 5233/5616 [1:06:39<05:11,  1.23it/s, gpt_loss=0.00629, lr=5.55e-7]
+
Train step of epoch 0:  93%|█████████▎| 5233/5616 [1:06:40<05:11,  1.23it/s, gpt_loss=0.0117, lr=5.55e-7] 
+
Train step of epoch 0:  93%|█████████▎| 5234/5616 [1:06:40<05:05,  1.25it/s, gpt_loss=0.0117, lr=5.55e-7]
+
Train step of epoch 0:  93%|█████████▎| 5234/5616 [1:06:41<05:05,  1.25it/s, gpt_loss=0.0461, lr=5.55e-7]
+
Train step of epoch 0:  93%|█████████▎| 5235/5616 [1:06:41<04:53,  1.30it/s, gpt_loss=0.0461, lr=5.55e-7]
+
Train step of epoch 0:  93%|█████████▎| 5235/5616 [1:06:41<04:53,  1.30it/s, gpt_loss=0.0148, lr=5.54e-7]
+
Train step of epoch 0:  93%|█████████▎| 5236/5616 [1:06:41<04:54,  1.29it/s, gpt_loss=0.0148, lr=5.54e-7]
+
Train step of epoch 0:  93%|█████████▎| 5236/5616 [1:06:42<04:54,  1.29it/s, gpt_loss=0.0199, lr=5.54e-7]
+
Train step of epoch 0:  93%|█████████▎| 5237/5616 [1:06:42<04:50,  1.31it/s, gpt_loss=0.0199, lr=5.54e-7]
+
Train step of epoch 0:  93%|█████████▎| 5237/5616 [1:06:43<04:50,  1.31it/s, gpt_loss=0.0118, lr=5.54e-7]
+
Train step of epoch 0:  93%|█████████▎| 5238/5616 [1:06:43<04:39,  1.35it/s, gpt_loss=0.0118, lr=5.54e-7]
+
Train step of epoch 0:  93%|█████████▎| 5238/5616 [1:06:43<04:39,  1.35it/s, gpt_loss=0.0163, lr=5.54e-7]
+
Train step of epoch 0:  93%|█████████▎| 5239/5616 [1:06:43<04:32,  1.38it/s, gpt_loss=0.0163, lr=5.54e-7]
+
Train step of epoch 0:  93%|█████████▎| 5239/5616 [1:06:44<04:32,  1.38it/s, gpt_loss=0.0378, lr=5.53e-7]
+
Train step of epoch 0:  93%|█████████▎| 5240/5616 [1:06:44<04:33,  1.37it/s, gpt_loss=0.0378, lr=5.53e-7]
+
Train step of epoch 0:  93%|█████████▎| 5240/5616 [1:06:45<04:33,  1.37it/s, gpt_loss=0.0032, lr=5.53e-7]
+
Train step of epoch 0:  93%|█████████▎| 5241/5616 [1:06:45<04:27,  1.40it/s, gpt_loss=0.0032, lr=5.53e-7]
+
Train step of epoch 0:  93%|█████████▎| 5241/5616 [1:06:46<04:27,  1.40it/s, gpt_loss=0.00617, lr=5.53e-7]
+
Train step of epoch 0:  93%|█████████▎| 5242/5616 [1:06:46<04:41,  1.33it/s, gpt_loss=0.00617, lr=5.53e-7]
+
Train step of epoch 0:  93%|█████████▎| 5242/5616 [1:06:46<04:41,  1.33it/s, gpt_loss=0.0293, lr=5.53e-7] 
+
Train step of epoch 0:  93%|█████████▎| 5243/5616 [1:06:46<04:40,  1.33it/s, gpt_loss=0.0293, lr=5.53e-7]
+
Train step of epoch 0:  93%|█████████▎| 5243/5616 [1:06:47<04:40,  1.33it/s, gpt_loss=0.0187, lr=5.52e-7]
+
Train step of epoch 0:  93%|█████████▎| 5244/5616 [1:06:47<05:09,  1.20it/s, gpt_loss=0.0187, lr=5.52e-7]
+
Train step of epoch 0:  93%|█████████▎| 5244/5616 [1:06:48<05:09,  1.20it/s, gpt_loss=0.0101, lr=5.52e-7]
+
Train step of epoch 0:  93%|█████████▎| 5245/5616 [1:06:48<04:50,  1.28it/s, gpt_loss=0.0101, lr=5.52e-7]
+
Train step of epoch 0:  93%|█████████▎| 5245/5616 [1:06:49<04:50,  1.28it/s, gpt_loss=0.0183, lr=5.52e-7]
+
Train step of epoch 0:  93%|█████████▎| 5246/5616 [1:06:49<04:51,  1.27it/s, gpt_loss=0.0183, lr=5.52e-7]
+
Train step of epoch 0:  93%|█████████▎| 5246/5616 [1:06:50<04:51,  1.27it/s, gpt_loss=0.0136, lr=5.52e-7]
+
Train step of epoch 0:  93%|█████████▎| 5247/5616 [1:06:50<04:39,  1.32it/s, gpt_loss=0.0136, lr=5.52e-7]
+
Train step of epoch 0:  93%|█████████▎| 5247/5616 [1:06:50<04:39,  1.32it/s, gpt_loss=0.0818, lr=5.51e-7]
+
Train step of epoch 0:  93%|█████████▎| 5248/5616 [1:06:50<04:42,  1.30it/s, gpt_loss=0.0818, lr=5.51e-7]
+
Train step of epoch 0:  93%|█████████▎| 5248/5616 [1:06:51<04:42,  1.30it/s, gpt_loss=0.0535, lr=5.51e-7]
+
Train step of epoch 0:  93%|█████████▎| 5249/5616 [1:06:51<04:34,  1.34it/s, gpt_loss=0.0535, lr=5.51e-7]
+
Train step of epoch 0:  93%|█████████▎| 5249/5616 [1:06:52<04:34,  1.34it/s, gpt_loss=0.0051, lr=5.51e-7]
+
Train step of epoch 0:  93%|█████████▎| 5250/5616 [1:06:52<04:22,  1.39it/s, gpt_loss=0.0051, lr=5.51e-7]
+
Train step of epoch 0:  93%|█████████▎| 5250/5616 [1:06:52<04:22,  1.39it/s, gpt_loss=0.0327, lr=5.51e-7]
+
Train step of epoch 0:  94%|█████████▎| 5251/5616 [1:06:52<04:22,  1.39it/s, gpt_loss=0.0327, lr=5.51e-7]
+
Train step of epoch 0:  94%|█████████▎| 5251/5616 [1:06:53<04:22,  1.39it/s, gpt_loss=0.0839, lr=5.49e-7]
+
Train step of epoch 0:  94%|█████████▎| 5252/5616 [1:06:53<04:29,  1.35it/s, gpt_loss=0.0839, lr=5.49e-7]
+
Train step of epoch 0:  94%|█████████▎| 5252/5616 [1:06:54<04:29,  1.35it/s, gpt_loss=0.00641, lr=5.49e-7]
+
Train step of epoch 0:  94%|█████████▎| 5253/5616 [1:06:54<04:22,  1.38it/s, gpt_loss=0.00641, lr=5.49e-7]
+
Train step of epoch 0:  94%|█████████▎| 5253/5616 [1:06:55<04:22,  1.38it/s, gpt_loss=0.0191, lr=5.49e-7] 
+
Train step of epoch 0:  94%|█████████▎| 5254/5616 [1:06:55<04:24,  1.37it/s, gpt_loss=0.0191, lr=5.49e-7]
+
Train step of epoch 0:  94%|█████████▎| 5254/5616 [1:06:56<04:24,  1.37it/s, gpt_loss=0.00468, lr=5.49e-7]
+
Train step of epoch 0:  94%|█████████▎| 5255/5616 [1:06:56<04:32,  1.33it/s, gpt_loss=0.00468, lr=5.49e-7]
+
Train step of epoch 0:  94%|█████████▎| 5255/5616 [1:06:56<04:32,  1.33it/s, gpt_loss=0.0307, lr=5.48e-7] 
+
Train step of epoch 0:  94%|█████████▎| 5256/5616 [1:06:56<04:44,  1.27it/s, gpt_loss=0.0307, lr=5.48e-7]
+
Train step of epoch 0:  94%|█████████▎| 5256/5616 [1:06:57<04:44,  1.27it/s, gpt_loss=0.0013, lr=5.48e-7]
+
Train step of epoch 0:  94%|█████████▎| 5257/5616 [1:06:57<04:31,  1.32it/s, gpt_loss=0.0013, lr=5.48e-7]
+
Train step of epoch 0:  94%|█████████▎| 5257/5616 [1:06:58<04:31,  1.32it/s, gpt_loss=0.00875, lr=5.48e-7]
+
Train step of epoch 0:  94%|█████████▎| 5258/5616 [1:06:58<04:28,  1.33it/s, gpt_loss=0.00875, lr=5.48e-7]
+
Train step of epoch 0:  94%|█████████▎| 5258/5616 [1:06:59<04:28,  1.33it/s, gpt_loss=0.00206, lr=5.48e-7]
+
Train step of epoch 0:  94%|█████████▎| 5259/5616 [1:06:59<04:39,  1.28it/s, gpt_loss=0.00206, lr=5.48e-7]
+
Train step of epoch 0:  94%|█████████▎| 5259/5616 [1:06:59<04:39,  1.28it/s, gpt_loss=0.0632, lr=5.47e-7] 
+
Train step of epoch 0:  94%|█████████▎| 5260/5616 [1:06:59<04:44,  1.25it/s, gpt_loss=0.0632, lr=5.47e-7]
+
Train step of epoch 0:  94%|█████████▎| 5260/5616 [1:07:00<04:44,  1.25it/s, gpt_loss=0.148, lr=5.47e-7] 
+
Train step of epoch 0:  94%|█████████▎| 5261/5616 [1:07:00<04:33,  1.30it/s, gpt_loss=0.148, lr=5.47e-7]
+
Train step of epoch 0:  94%|█████████▎| 5261/5616 [1:07:01<04:33,  1.30it/s, gpt_loss=0.065, lr=5.47e-7]
+
Train step of epoch 0:  94%|█████████▎| 5262/5616 [1:07:01<04:35,  1.28it/s, gpt_loss=0.065, lr=5.47e-7]
+
Train step of epoch 0:  94%|█████████▎| 5262/5616 [1:07:02<04:35,  1.28it/s, gpt_loss=0.00109, lr=5.47e-7]
+
Train step of epoch 0:  94%|█████████▎| 5263/5616 [1:07:02<04:23,  1.34it/s, gpt_loss=0.00109, lr=5.47e-7]
+
Train step of epoch 0:  94%|█████████▎| 5263/5616 [1:07:03<04:23,  1.34it/s, gpt_loss=0.00737, lr=5.46e-7]
+
Train step of epoch 0:  94%|█████████▎| 5264/5616 [1:07:03<04:33,  1.29it/s, gpt_loss=0.00737, lr=5.46e-7]
+
Train step of epoch 0:  94%|█████████▎| 5264/5616 [1:07:03<04:33,  1.29it/s, gpt_loss=0.0277, lr=5.46e-7] 
+
Train step of epoch 0:  94%|█████████▍| 5265/5616 [1:07:03<04:24,  1.33it/s, gpt_loss=0.0277, lr=5.46e-7]
+
Train step of epoch 0:  94%|█████████▍| 5265/5616 [1:07:04<04:24,  1.33it/s, gpt_loss=0.0193, lr=5.46e-7]
+
Train step of epoch 0:  94%|█████████▍| 5266/5616 [1:07:04<04:20,  1.34it/s, gpt_loss=0.0193, lr=5.46e-7]
+
Train step of epoch 0:  94%|█████████▍| 5266/5616 [1:07:05<04:20,  1.34it/s, gpt_loss=0.0203, lr=5.46e-7]
+
Train step of epoch 0:  94%|█████████▍| 5267/5616 [1:07:05<04:25,  1.31it/s, gpt_loss=0.0203, lr=5.46e-7]
+
Train step of epoch 0:  94%|█████████▍| 5267/5616 [1:07:06<04:25,  1.31it/s, gpt_loss=0.0368, lr=5.45e-7]
+
Train step of epoch 0:  94%|█████████▍| 5268/5616 [1:07:06<04:32,  1.28it/s, gpt_loss=0.0368, lr=5.45e-7]
+
Train step of epoch 0:  94%|█████████▍| 5268/5616 [1:07:06<04:32,  1.28it/s, gpt_loss=0.0197, lr=5.45e-7]
+
Train step of epoch 0:  94%|█████████▍| 5269/5616 [1:07:06<04:17,  1.35it/s, gpt_loss=0.0197, lr=5.45e-7]
+
Train step of epoch 0:  94%|█████████▍| 5269/5616 [1:07:07<04:17,  1.35it/s, gpt_loss=0.00932, lr=5.45e-7]
+
Train step of epoch 0:  94%|█████████▍| 5270/5616 [1:07:07<04:20,  1.33it/s, gpt_loss=0.00932, lr=5.45e-7]
+
Train step of epoch 0:  94%|█████████▍| 5270/5616 [1:07:08<04:20,  1.33it/s, gpt_loss=0.00449, lr=5.45e-7]
+
Train step of epoch 0:  94%|█████████▍| 5271/5616 [1:07:08<04:11,  1.37it/s, gpt_loss=0.00449, lr=5.45e-7]
+
Train step of epoch 0:  94%|█████████▍| 5271/5616 [1:07:08<04:11,  1.37it/s, gpt_loss=0.0453, lr=5.44e-7] 
+
Train step of epoch 0:  94%|█████████▍| 5272/5616 [1:07:08<04:19,  1.33it/s, gpt_loss=0.0453, lr=5.44e-7]
+
Train step of epoch 0:  94%|█████████▍| 5272/5616 [1:07:09<04:19,  1.33it/s, gpt_loss=0.0152, lr=5.44e-7]
+
Train step of epoch 0:  94%|█████████▍| 5273/5616 [1:07:09<04:13,  1.36it/s, gpt_loss=0.0152, lr=5.44e-7]
+
Train step of epoch 0:  94%|█████████▍| 5273/5616 [1:07:10<04:13,  1.36it/s, gpt_loss=0.0275, lr=5.44e-7]
+
Train step of epoch 0:  94%|█████████▍| 5274/5616 [1:07:10<04:11,  1.36it/s, gpt_loss=0.0275, lr=5.44e-7]
+
Train step of epoch 0:  94%|█████████▍| 5274/5616 [1:07:11<04:11,  1.36it/s, gpt_loss=0.0012, lr=5.44e-7]
+
Train step of epoch 0:  94%|█████████▍| 5275/5616 [1:07:11<04:18,  1.32it/s, gpt_loss=0.0012, lr=5.44e-7]
+
Train step of epoch 0:  94%|█████████▍| 5275/5616 [1:07:12<04:18,  1.32it/s, gpt_loss=0.038, lr=5.43e-7] 
+
Train step of epoch 0:  94%|█████████▍| 5276/5616 [1:07:12<04:19,  1.31it/s, gpt_loss=0.038, lr=5.43e-7]
+
Train step of epoch 0:  94%|█████████▍| 5276/5616 [1:07:12<04:19,  1.31it/s, gpt_loss=0.00965, lr=5.43e-7]
+
Train step of epoch 0:  94%|█████████▍| 5277/5616 [1:07:12<04:24,  1.28it/s, gpt_loss=0.00965, lr=5.43e-7]
+
Train step of epoch 0:  94%|█████████▍| 5277/5616 [1:07:13<04:24,  1.28it/s, gpt_loss=0.0822, lr=5.43e-7] 
+
Train step of epoch 0:  94%|█████████▍| 5278/5616 [1:07:13<04:21,  1.29it/s, gpt_loss=0.0822, lr=5.43e-7]
+
Train step of epoch 0:  94%|█████████▍| 5278/5616 [1:07:14<04:21,  1.29it/s, gpt_loss=0.00303, lr=5.43e-7]
+
Train step of epoch 0:  94%|█████████▍| 5279/5616 [1:07:14<04:11,  1.34it/s, gpt_loss=0.00303, lr=5.43e-7]
+
Train step of epoch 0:  94%|█████████▍| 5279/5616 [1:07:15<04:11,  1.34it/s, gpt_loss=0.0148, lr=5.42e-7] 
+
Train step of epoch 0:  94%|█████████▍| 5280/5616 [1:07:15<04:18,  1.30it/s, gpt_loss=0.0148, lr=5.42e-7]
+
Train step of epoch 0:  94%|█████████▍| 5280/5616 [1:07:15<04:18,  1.30it/s, gpt_loss=0.0639, lr=5.42e-7]
+
Train step of epoch 0:  94%|█████████▍| 5281/5616 [1:07:15<04:25,  1.26it/s, gpt_loss=0.0639, lr=5.42e-7]
+
Train step of epoch 0:  94%|█████████▍| 5281/5616 [1:07:16<04:25,  1.26it/s, gpt_loss=0.00625, lr=5.42e-7]
+
Train step of epoch 0:  94%|█████████▍| 5282/5616 [1:07:16<04:20,  1.28it/s, gpt_loss=0.00625, lr=5.42e-7]
+
Train step of epoch 0:  94%|█████████▍| 5282/5616 [1:07:17<04:20,  1.28it/s, gpt_loss=0.0195, lr=5.42e-7] 
+
Train step of epoch 0:  94%|█████████▍| 5283/5616 [1:07:17<04:17,  1.29it/s, gpt_loss=0.0195, lr=5.42e-7]
+
Train step of epoch 0:  94%|█████████▍| 5283/5616 [1:07:18<04:17,  1.29it/s, gpt_loss=0.0073, lr=5.41e-7]
+
Train step of epoch 0:  94%|█████████▍| 5284/5616 [1:07:18<04:15,  1.30it/s, gpt_loss=0.0073, lr=5.41e-7]
+
Train step of epoch 0:  94%|█████████▍| 5284/5616 [1:07:18<04:15,  1.30it/s, gpt_loss=0.00164, lr=5.41e-7]
+
Train step of epoch 0:  94%|█████████▍| 5285/5616 [1:07:18<04:05,  1.35it/s, gpt_loss=0.00164, lr=5.41e-7]
+
Train step of epoch 0:  94%|█████████▍| 5285/5616 [1:07:19<04:05,  1.35it/s, gpt_loss=0.00606, lr=5.41e-7]
+
Train step of epoch 0:  94%|█████████▍| 5286/5616 [1:07:19<04:00,  1.37it/s, gpt_loss=0.00606, lr=5.41e-7]
+
Train step of epoch 0:  94%|█████████▍| 5286/5616 [1:07:20<04:00,  1.37it/s, gpt_loss=0.0166, lr=5.41e-7] 
+
Train step of epoch 0:  94%|█████████▍| 5287/5616 [1:07:20<04:44,  1.15it/s, gpt_loss=0.0166, lr=5.41e-7]
+
Train step of epoch 0:  94%|█████████▍| 5287/5616 [1:07:21<04:44,  1.15it/s, gpt_loss=0.0522, lr=5.4e-7] 
+
Train step of epoch 0:  94%|█████████▍| 5288/5616 [1:07:21<04:37,  1.18it/s, gpt_loss=0.0522, lr=5.4e-7]
+
Train step of epoch 0:  94%|█████████▍| 5288/5616 [1:07:22<04:37,  1.18it/s, gpt_loss=0.0752, lr=5.4e-7]
+
Train step of epoch 0:  94%|█████████▍| 5289/5616 [1:07:22<04:32,  1.20it/s, gpt_loss=0.0752, lr=5.4e-7]
+
Train step of epoch 0:  94%|█████████▍| 5289/5616 [1:07:23<04:32,  1.20it/s, gpt_loss=0.0127, lr=5.4e-7]
+
Train step of epoch 0:  94%|█████████▍| 5290/5616 [1:07:23<04:31,  1.20it/s, gpt_loss=0.0127, lr=5.4e-7]
+
Train step of epoch 0:  94%|█████████▍| 5290/5616 [1:07:23<04:31,  1.20it/s, gpt_loss=0.0143, lr=5.4e-7]
+
Train step of epoch 0:  94%|█████████▍| 5291/5616 [1:07:23<04:20,  1.25it/s, gpt_loss=0.0143, lr=5.4e-7]
+
Train step of epoch 0:  94%|█████████▍| 5291/5616 [1:07:25<04:20,  1.25it/s, gpt_loss=0.0489, lr=5.39e-7]
+
Train step of epoch 0:  94%|█████████▍| 5292/5616 [1:07:25<04:50,  1.12it/s, gpt_loss=0.0489, lr=5.39e-7]
+
Train step of epoch 0:  94%|█████████▍| 5292/5616 [1:07:25<04:50,  1.12it/s, gpt_loss=0.0443, lr=5.39e-7]
+
Train step of epoch 0:  94%|█████████▍| 5293/5616 [1:07:25<04:30,  1.19it/s, gpt_loss=0.0443, lr=5.39e-7]
+
Train step of epoch 0:  94%|█████████▍| 5293/5616 [1:07:26<04:30,  1.19it/s, gpt_loss=0.0521, lr=5.39e-7]
+
Train step of epoch 0:  94%|█████████▍| 5294/5616 [1:07:26<04:28,  1.20it/s, gpt_loss=0.0521, lr=5.39e-7]
+
Train step of epoch 0:  94%|█████████▍| 5294/5616 [1:07:27<04:28,  1.20it/s, gpt_loss=0.0966, lr=5.39e-7]
+
Train step of epoch 0:  94%|█████████▍| 5295/5616 [1:07:27<04:13,  1.27it/s, gpt_loss=0.0966, lr=5.39e-7]
+
Train step of epoch 0:  94%|█████████▍| 5295/5616 [1:07:28<04:13,  1.27it/s, gpt_loss=0.0165, lr=5.38e-7]
+
Train step of epoch 0:  94%|█████████▍| 5296/5616 [1:07:28<04:20,  1.23it/s, gpt_loss=0.0165, lr=5.38e-7]
+
Train step of epoch 0:  94%|█████████▍| 5296/5616 [1:07:28<04:20,  1.23it/s, gpt_loss=0.00225, lr=5.38e-7]
+
Train step of epoch 0:  94%|█████████▍| 5297/5616 [1:07:28<04:07,  1.29it/s, gpt_loss=0.00225, lr=5.38e-7]
+
Train step of epoch 0:  94%|█████████▍| 5297/5616 [1:07:29<04:07,  1.29it/s, gpt_loss=0.045, lr=5.38e-7]  
+
Train step of epoch 0:  94%|█████████▍| 5298/5616 [1:07:29<04:01,  1.32it/s, gpt_loss=0.045, lr=5.38e-7]
+
Train step of epoch 0:  94%|█████████▍| 5298/5616 [1:07:30<04:01,  1.32it/s, gpt_loss=0.142, lr=5.38e-7]
+
Train step of epoch 0:  94%|█████████▍| 5299/5616 [1:07:30<03:59,  1.33it/s, gpt_loss=0.142, lr=5.38e-7]
+
Train step of epoch 0:  94%|█████████▍| 5299/5616 [1:07:31<03:59,  1.33it/s, gpt_loss=0.0273, lr=5.37e-7]
+
Train step of epoch 0:  94%|█████████▍| 5300/5616 [1:07:31<03:57,  1.33it/s, gpt_loss=0.0273, lr=5.37e-7]
+
Train step of epoch 0:  94%|█████████▍| 5300/5616 [1:07:31<03:57,  1.33it/s, gpt_loss=0.0186, lr=5.37e-7]
+
Train step of epoch 0:  94%|█████████▍| 5301/5616 [1:07:31<03:56,  1.33it/s, gpt_loss=0.0186, lr=5.37e-7]
+
Train step of epoch 0:  94%|█████████▍| 5301/5616 [1:07:32<03:56,  1.33it/s, gpt_loss=0.0405, lr=5.37e-7]
+
Train step of epoch 0:  94%|█████████▍| 5302/5616 [1:07:32<04:02,  1.29it/s, gpt_loss=0.0405, lr=5.37e-7]
+
Train step of epoch 0:  94%|█████████▍| 5302/5616 [1:07:33<04:02,  1.29it/s, gpt_loss=0.0618, lr=5.37e-7]
+
Train step of epoch 0:  94%|█████████▍| 5303/5616 [1:07:33<04:06,  1.27it/s, gpt_loss=0.0618, lr=5.37e-7]
+
Train step of epoch 0:  94%|█████████▍| 5303/5616 [1:07:34<04:06,  1.27it/s, gpt_loss=0.00526, lr=5.36e-7]
+
Train step of epoch 0:  94%|█████████▍| 5304/5616 [1:07:34<04:08,  1.26it/s, gpt_loss=0.00526, lr=5.36e-7]
+
Train step of epoch 0:  94%|█████████▍| 5304/5616 [1:07:34<04:08,  1.26it/s, gpt_loss=0.0497, lr=5.36e-7] 
+
Train step of epoch 0:  94%|█████████▍| 5305/5616 [1:07:34<03:54,  1.33it/s, gpt_loss=0.0497, lr=5.36e-7]
+
Train step of epoch 0:  94%|█████████▍| 5305/5616 [1:07:35<03:54,  1.33it/s, gpt_loss=0.00753, lr=5.36e-7]
+
Train step of epoch 0:  94%|█████████▍| 5306/5616 [1:07:35<03:43,  1.39it/s, gpt_loss=0.00753, lr=5.36e-7]
+
Train step of epoch 0:  94%|█████████▍| 5306/5616 [1:07:36<03:43,  1.39it/s, gpt_loss=0.0319, lr=5.36e-7] 
+
Train step of epoch 0:  94%|█████████▍| 5307/5616 [1:07:36<03:42,  1.39it/s, gpt_loss=0.0319, lr=5.36e-7]
+
Train step of epoch 0:  94%|█████████▍| 5307/5616 [1:07:37<03:42,  1.39it/s, gpt_loss=0.0194, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5308/5616 [1:07:37<04:05,  1.26it/s, gpt_loss=0.0194, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5308/5616 [1:07:37<04:05,  1.26it/s, gpt_loss=0.0183, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5309/5616 [1:07:37<03:55,  1.30it/s, gpt_loss=0.0183, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5309/5616 [1:07:38<03:55,  1.30it/s, gpt_loss=0.0332, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5310/5616 [1:07:38<03:48,  1.34it/s, gpt_loss=0.0332, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5310/5616 [1:07:39<03:48,  1.34it/s, gpt_loss=0.00687, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5311/5616 [1:07:39<03:55,  1.30it/s, gpt_loss=0.00687, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5311/5616 [1:07:40<03:55,  1.30it/s, gpt_loss=0.0273, lr=5.35e-7] 
+
Train step of epoch 0:  95%|█████████▍| 5312/5616 [1:07:40<04:04,  1.24it/s, gpt_loss=0.0273, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5312/5616 [1:07:40<04:04,  1.24it/s, gpt_loss=0.0042, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5313/5616 [1:07:40<03:50,  1.32it/s, gpt_loss=0.0042, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5313/5616 [1:07:41<03:50,  1.32it/s, gpt_loss=0.0128, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5314/5616 [1:07:41<03:41,  1.36it/s, gpt_loss=0.0128, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5314/5616 [1:07:42<03:41,  1.36it/s, gpt_loss=0.0348, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5315/5616 [1:07:42<03:34,  1.40it/s, gpt_loss=0.0348, lr=5.35e-7]
+
Train step of epoch 0:  95%|█████████▍| 5315/5616 [1:07:43<03:34,  1.40it/s, gpt_loss=0.0197, lr=5.34e-7]
+
Train step of epoch 0:  95%|█████████▍| 5316/5616 [1:07:43<03:35,  1.39it/s, gpt_loss=0.0197, lr=5.34e-7]
+
Train step of epoch 0:  95%|█████████▍| 5316/5616 [1:07:43<03:35,  1.39it/s, gpt_loss=0.00533, lr=5.34e-7]
+
Train step of epoch 0:  95%|█████████▍| 5317/5616 [1:07:43<03:38,  1.37it/s, gpt_loss=0.00533, lr=5.34e-7]
+
Train step of epoch 0:  95%|█████████▍| 5317/5616 [1:07:44<03:38,  1.37it/s, gpt_loss=0.0045, lr=5.34e-7] 
+
Train step of epoch 0:  95%|█████████▍| 5318/5616 [1:07:44<03:34,  1.39it/s, gpt_loss=0.0045, lr=5.34e-7]
+
Train step of epoch 0:  95%|█████████▍| 5318/5616 [1:07:45<03:34,  1.39it/s, gpt_loss=0.0307, lr=5.34e-7]
+
Train step of epoch 0:  95%|█████████▍| 5319/5616 [1:07:45<04:11,  1.18it/s, gpt_loss=0.0307, lr=5.34e-7]
+
Train step of epoch 0:  95%|█████████▍| 5319/5616 [1:07:46<04:11,  1.18it/s, gpt_loss=0.0516, lr=5.33e-7]
+
Train step of epoch 0:  95%|█████████▍| 5320/5616 [1:07:46<03:58,  1.24it/s, gpt_loss=0.0516, lr=5.33e-7]
+
Train step of epoch 0:  95%|█████████▍| 5320/5616 [1:07:47<03:58,  1.24it/s, gpt_loss=0.0475, lr=5.33e-7]
+
Train step of epoch 0:  95%|█████████▍| 5321/5616 [1:07:47<04:08,  1.19it/s, gpt_loss=0.0475, lr=5.33e-7]
+
Train step of epoch 0:  95%|█████████▍| 5321/5616 [1:07:48<04:08,  1.19it/s, gpt_loss=0.0334, lr=5.33e-7]
+
Train step of epoch 0:  95%|█████████▍| 5322/5616 [1:07:48<03:59,  1.23it/s, gpt_loss=0.0334, lr=5.33e-7]
+
Train step of epoch 0:  95%|█████████▍| 5322/5616 [1:07:48<03:59,  1.23it/s, gpt_loss=0.166, lr=5.33e-7] 
+
Train step of epoch 0:  95%|█████████▍| 5323/5616 [1:07:48<03:50,  1.27it/s, gpt_loss=0.166, lr=5.33e-7]
+
Train step of epoch 0:  95%|█████████▍| 5323/5616 [1:07:49<03:50,  1.27it/s, gpt_loss=0.0435, lr=5.32e-7]
+
Train step of epoch 0:  95%|█████████▍| 5324/5616 [1:07:49<03:47,  1.28it/s, gpt_loss=0.0435, lr=5.32e-7]
+
Train step of epoch 0:  95%|█████████▍| 5324/5616 [1:07:50<03:47,  1.28it/s, gpt_loss=0.00314, lr=5.32e-7]
+
Train step of epoch 0:  95%|█████████▍| 5325/5616 [1:07:50<03:34,  1.35it/s, gpt_loss=0.00314, lr=5.32e-7]
+
Train step of epoch 0:  95%|█████████▍| 5325/5616 [1:07:50<03:34,  1.35it/s, gpt_loss=0.0177, lr=5.32e-7] 
+
Train step of epoch 0:  95%|█████████▍| 5326/5616 [1:07:50<03:26,  1.41it/s, gpt_loss=0.0177, lr=5.32e-7]
+
Train step of epoch 0:  95%|█████████▍| 5326/5616 [1:07:51<03:26,  1.41it/s, gpt_loss=0.0573, lr=5.32e-7]
+
Train step of epoch 0:  95%|█████████▍| 5327/5616 [1:07:51<03:21,  1.43it/s, gpt_loss=0.0573, lr=5.32e-7]
+
Train step of epoch 0:  95%|█████████▍| 5327/5616 [1:07:52<03:21,  1.43it/s, gpt_loss=0.0852, lr=5.31e-7]
+
Train step of epoch 0:  95%|█████████▍| 5328/5616 [1:07:52<03:30,  1.37it/s, gpt_loss=0.0852, lr=5.31e-7]
+
Train step of epoch 0:  95%|█████████▍| 5328/5616 [1:07:53<03:30,  1.37it/s, gpt_loss=0.0652, lr=5.31e-7]
+
Train step of epoch 0:  95%|█████████▍| 5329/5616 [1:07:53<03:40,  1.30it/s, gpt_loss=0.0652, lr=5.31e-7]
+
Train step of epoch 0:  95%|█████████▍| 5329/5616 [1:07:53<03:40,  1.30it/s, gpt_loss=0.0843, lr=5.31e-7]
+
Train step of epoch 0:  95%|█████████▍| 5330/5616 [1:07:53<03:42,  1.29it/s, gpt_loss=0.0843, lr=5.31e-7]
+
Train step of epoch 0:  95%|█████████▍| 5330/5616 [1:07:54<03:42,  1.29it/s, gpt_loss=0.0497, lr=5.31e-7]
+
Train step of epoch 0:  95%|█████████▍| 5331/5616 [1:07:54<03:54,  1.21it/s, gpt_loss=0.0497, lr=5.31e-7]
+
Train step of epoch 0:  95%|█████████▍| 5331/5616 [1:07:55<03:54,  1.21it/s, gpt_loss=0.0269, lr=5.3e-7] 
+
Train step of epoch 0:  95%|█████████▍| 5332/5616 [1:07:55<03:51,  1.23it/s, gpt_loss=0.0269, lr=5.3e-7]
+
Train step of epoch 0:  95%|█████████▍| 5332/5616 [1:07:56<03:51,  1.23it/s, gpt_loss=0.11, lr=5.3e-7]  
+
Train step of epoch 0:  95%|█████████▍| 5333/5616 [1:07:56<03:55,  1.20it/s, gpt_loss=0.11, lr=5.3e-7]
+
Train step of epoch 0:  95%|█████████▍| 5333/5616 [1:07:57<03:55,  1.20it/s, gpt_loss=0.00402, lr=5.3e-7]
+
Train step of epoch 0:  95%|█████████▍| 5334/5616 [1:07:57<03:42,  1.27it/s, gpt_loss=0.00402, lr=5.3e-7]
+
Train step of epoch 0:  95%|█████████▍| 5334/5616 [1:07:57<03:42,  1.27it/s, gpt_loss=0.000919, lr=5.3e-7]
+
Train step of epoch 0:  95%|█████████▍| 5335/5616 [1:07:57<03:33,  1.32it/s, gpt_loss=0.000919, lr=5.3e-7]
+
Train step of epoch 0:  95%|█████████▍| 5335/5616 [1:07:58<03:33,  1.32it/s, gpt_loss=0.12, lr=5.29e-7]   
+
Train step of epoch 0:  95%|█████████▌| 5336/5616 [1:07:58<03:33,  1.31it/s, gpt_loss=0.12, lr=5.29e-7]
+
Train step of epoch 0:  95%|█████████▌| 5336/5616 [1:07:59<03:33,  1.31it/s, gpt_loss=0.00678, lr=5.29e-7]
+
Train step of epoch 0:  95%|█████████▌| 5337/5616 [1:07:59<03:26,  1.35it/s, gpt_loss=0.00678, lr=5.29e-7]
+
Train step of epoch 0:  95%|█████████▌| 5337/5616 [1:08:00<03:26,  1.35it/s, gpt_loss=0.00115, lr=5.29e-7]
+
Train step of epoch 0:  95%|█████████▌| 5338/5616 [1:08:00<03:28,  1.33it/s, gpt_loss=0.00115, lr=5.29e-7]
+
Train step of epoch 0:  95%|█████████▌| 5338/5616 [1:08:00<03:28,  1.33it/s, gpt_loss=0.00811, lr=5.29e-7]
+
Train step of epoch 0:  95%|█████████▌| 5339/5616 [1:08:00<03:24,  1.36it/s, gpt_loss=0.00811, lr=5.29e-7]
+
Train step of epoch 0:  95%|█████████▌| 5339/5616 [1:08:01<03:24,  1.36it/s, gpt_loss=0.000691, lr=5.28e-7]
+
Train step of epoch 0:  95%|█████████▌| 5340/5616 [1:08:01<03:31,  1.31it/s, gpt_loss=0.000691, lr=5.28e-7]
+
Train step of epoch 0:  95%|█████████▌| 5340/5616 [1:08:02<03:31,  1.31it/s, gpt_loss=0.058, lr=5.28e-7]   
+
Train step of epoch 0:  95%|█████████▌| 5341/5616 [1:08:02<03:43,  1.23it/s, gpt_loss=0.058, lr=5.28e-7]
+
Train step of epoch 0:  95%|█████████▌| 5341/5616 [1:08:03<03:43,  1.23it/s, gpt_loss=0.00188, lr=5.28e-7]
+
Train step of epoch 0:  95%|█████████▌| 5342/5616 [1:08:03<03:33,  1.28it/s, gpt_loss=0.00188, lr=5.28e-7]
+
Train step of epoch 0:  95%|█████████▌| 5342/5616 [1:08:04<03:33,  1.28it/s, gpt_loss=0.00804, lr=5.28e-7]
+
Train step of epoch 0:  95%|█████████▌| 5343/5616 [1:08:04<03:24,  1.34it/s, gpt_loss=0.00804, lr=5.28e-7]
+
Train step of epoch 0:  95%|█████████▌| 5343/5616 [1:08:04<03:24,  1.34it/s, gpt_loss=0.152, lr=5.28e-7]  
+
Train step of epoch 0:  95%|█████████▌| 5344/5616 [1:08:04<03:26,  1.32it/s, gpt_loss=0.152, lr=5.28e-7]
+
Train step of epoch 0:  95%|█████████▌| 5344/5616 [1:08:05<03:26,  1.32it/s, gpt_loss=0.00881, lr=5.28e-7]
+
Train step of epoch 0:  95%|█████████▌| 5345/5616 [1:08:05<03:25,  1.32it/s, gpt_loss=0.00881, lr=5.28e-7]
+
Train step of epoch 0:  95%|█████████▌| 5345/5616 [1:08:06<03:25,  1.32it/s, gpt_loss=0.00616, lr=5.28e-7]
+
Train step of epoch 0:  95%|█████████▌| 5346/5616 [1:08:06<03:19,  1.35it/s, gpt_loss=0.00616, lr=5.28e-7]
+
Train step of epoch 0:  95%|█████████▌| 5346/5616 [1:08:07<03:19,  1.35it/s, gpt_loss=0.0166, lr=5.28e-7] 
+
Train step of epoch 0:  95%|█████████▌| 5347/5616 [1:08:07<03:27,  1.30it/s, gpt_loss=0.0166, lr=5.28e-7]
+
Train step of epoch 0:  95%|█████████▌| 5347/5616 [1:08:07<03:27,  1.30it/s, gpt_loss=0.00222, lr=5.27e-7]
+
Train step of epoch 0:  95%|█████████▌| 5348/5616 [1:08:07<03:27,  1.29it/s, gpt_loss=0.00222, lr=5.27e-7]
+
Train step of epoch 0:  95%|█████████▌| 5348/5616 [1:08:08<03:27,  1.29it/s, gpt_loss=0.142, lr=5.27e-7]  
+
Train step of epoch 0:  95%|█████████▌| 5349/5616 [1:08:08<03:18,  1.34it/s, gpt_loss=0.142, lr=5.27e-7]
+
Train step of epoch 0:  95%|█████████▌| 5349/5616 [1:08:09<03:18,  1.34it/s, gpt_loss=0.00746, lr=5.27e-7]
+
Train step of epoch 0:  95%|█████████▌| 5350/5616 [1:08:09<03:08,  1.41it/s, gpt_loss=0.00746, lr=5.27e-7]
+
Train step of epoch 0:  95%|█████████▌| 5350/5616 [1:08:09<03:08,  1.41it/s, gpt_loss=0.0197, lr=5.27e-7] 
+
Train step of epoch 0:  95%|█████████▌| 5351/5616 [1:08:09<03:06,  1.42it/s, gpt_loss=0.0197, lr=5.27e-7]
+
Train step of epoch 0:  95%|█████████▌| 5351/5616 [1:08:10<03:06,  1.42it/s, gpt_loss=0.0364, lr=5.26e-7]
+
Train step of epoch 0:  95%|█████████▌| 5352/5616 [1:08:10<03:07,  1.41it/s, gpt_loss=0.0364, lr=5.26e-7]
+
Train step of epoch 0:  95%|█████████▌| 5352/5616 [1:08:11<03:07,  1.41it/s, gpt_loss=0.00442, lr=5.26e-7]
+
Train step of epoch 0:  95%|█████████▌| 5353/5616 [1:08:11<03:05,  1.42it/s, gpt_loss=0.00442, lr=5.26e-7]
+
Train step of epoch 0:  95%|█████████▌| 5353/5616 [1:08:11<03:05,  1.42it/s, gpt_loss=0.0614, lr=5.26e-7] 
+
Train step of epoch 0:  95%|█████████▌| 5354/5616 [1:08:11<03:04,  1.42it/s, gpt_loss=0.0614, lr=5.26e-7]
+
Train step of epoch 0:  95%|█████████▌| 5354/5616 [1:08:12<03:04,  1.42it/s, gpt_loss=0.0272, lr=5.26e-7]
+
Train step of epoch 0:  95%|█████████▌| 5355/5616 [1:08:12<03:13,  1.35it/s, gpt_loss=0.0272, lr=5.26e-7]
+
Train step of epoch 0:  95%|█████████▌| 5355/5616 [1:08:13<03:13,  1.35it/s, gpt_loss=0.0835, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5356/5616 [1:08:13<03:26,  1.26it/s, gpt_loss=0.0835, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5356/5616 [1:08:14<03:26,  1.26it/s, gpt_loss=0.0383, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5357/5616 [1:08:14<03:28,  1.24it/s, gpt_loss=0.0383, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5357/5616 [1:08:15<03:28,  1.24it/s, gpt_loss=0.0049, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5358/5616 [1:08:15<03:20,  1.29it/s, gpt_loss=0.0049, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5358/5616 [1:08:15<03:20,  1.29it/s, gpt_loss=0.0479, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5359/5616 [1:08:15<03:14,  1.32it/s, gpt_loss=0.0479, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5359/5616 [1:08:16<03:14,  1.32it/s, gpt_loss=0.0204, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5360/5616 [1:08:16<03:15,  1.31it/s, gpt_loss=0.0204, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5360/5616 [1:08:17<03:15,  1.31it/s, gpt_loss=0.0498, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5361/5616 [1:08:17<03:06,  1.37it/s, gpt_loss=0.0498, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5361/5616 [1:08:18<03:06,  1.37it/s, gpt_loss=0.0181, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5362/5616 [1:08:18<03:02,  1.39it/s, gpt_loss=0.0181, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5362/5616 [1:08:18<03:02,  1.39it/s, gpt_loss=0.00213, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5363/5616 [1:08:18<03:00,  1.40it/s, gpt_loss=0.00213, lr=5.25e-7]
+
Train step of epoch 0:  95%|█████████▌| 5363/5616 [1:08:19<03:00,  1.40it/s, gpt_loss=0.0242, lr=5.24e-7] 
+
Train step of epoch 0:  96%|█████████▌| 5364/5616 [1:08:19<03:03,  1.38it/s, gpt_loss=0.0242, lr=5.24e-7]
+
Train step of epoch 0:  96%|█████████▌| 5364/5616 [1:08:20<03:03,  1.38it/s, gpt_loss=0.0217, lr=5.24e-7]
+
Train step of epoch 0:  96%|█████████▌| 5365/5616 [1:08:20<03:03,  1.37it/s, gpt_loss=0.0217, lr=5.24e-7]
+
Train step of epoch 0:  96%|█████████▌| 5365/5616 [1:08:21<03:03,  1.37it/s, gpt_loss=0.031, lr=5.24e-7] 
+
Train step of epoch 0:  96%|█████████▌| 5366/5616 [1:08:21<03:12,  1.30it/s, gpt_loss=0.031, lr=5.24e-7]
+
Train step of epoch 0:  96%|█████████▌| 5366/5616 [1:08:21<03:12,  1.30it/s, gpt_loss=0.00715, lr=5.24e-7]
+
Train step of epoch 0:  96%|█████████▌| 5367/5616 [1:08:21<03:04,  1.35it/s, gpt_loss=0.00715, lr=5.24e-7]
+
Train step of epoch 0:  96%|█████████▌| 5367/5616 [1:08:22<03:04,  1.35it/s, gpt_loss=0.00861, lr=5.23e-7]
+
Train step of epoch 0:  96%|█████████▌| 5368/5616 [1:08:22<03:04,  1.35it/s, gpt_loss=0.00861, lr=5.23e-7]
+
Train step of epoch 0:  96%|█████████▌| 5368/5616 [1:08:23<03:04,  1.35it/s, gpt_loss=0.0359, lr=5.23e-7] 
+
Train step of epoch 0:  96%|█████████▌| 5369/5616 [1:08:23<03:02,  1.35it/s, gpt_loss=0.0359, lr=5.23e-7]
+
Train step of epoch 0:  96%|█████████▌| 5369/5616 [1:08:24<03:02,  1.35it/s, gpt_loss=0.13, lr=5.23e-7]  
+
Train step of epoch 0:  96%|█████████▌| 5370/5616 [1:08:24<03:01,  1.35it/s, gpt_loss=0.13, lr=5.23e-7]
+
Train step of epoch 0:  96%|█████████▌| 5370/5616 [1:08:24<03:01,  1.35it/s, gpt_loss=0.0875, lr=5.23e-7]
+
Train step of epoch 0:  96%|█████████▌| 5371/5616 [1:08:24<02:57,  1.38it/s, gpt_loss=0.0875, lr=5.23e-7]
+
Train step of epoch 0:  96%|█████████▌| 5371/5616 [1:08:25<02:57,  1.38it/s, gpt_loss=0.0126, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5372/5616 [1:08:25<03:04,  1.32it/s, gpt_loss=0.0126, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5372/5616 [1:08:26<03:04,  1.32it/s, gpt_loss=0.025, lr=5.22e-7] 
+
Train step of epoch 0:  96%|█████████▌| 5373/5616 [1:08:26<02:57,  1.37it/s, gpt_loss=0.025, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5373/5616 [1:08:26<02:57,  1.37it/s, gpt_loss=0.0484, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5374/5616 [1:08:26<02:55,  1.38it/s, gpt_loss=0.0484, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5374/5616 [1:08:27<02:55,  1.38it/s, gpt_loss=0.0194, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5375/5616 [1:08:27<02:56,  1.37it/s, gpt_loss=0.0194, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5375/5616 [1:08:28<02:56,  1.37it/s, gpt_loss=0.00185, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5376/5616 [1:08:28<02:56,  1.36it/s, gpt_loss=0.00185, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5376/5616 [1:08:29<02:56,  1.36it/s, gpt_loss=0.00374, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5377/5616 [1:08:29<02:56,  1.36it/s, gpt_loss=0.00374, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5377/5616 [1:08:29<02:56,  1.36it/s, gpt_loss=0.111, lr=5.22e-7]  
+
Train step of epoch 0:  96%|█████████▌| 5378/5616 [1:08:29<02:58,  1.33it/s, gpt_loss=0.111, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5378/5616 [1:08:30<02:58,  1.33it/s, gpt_loss=0.0248, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5379/5616 [1:08:30<02:49,  1.40it/s, gpt_loss=0.0248, lr=5.22e-7]
+
Train step of epoch 0:  96%|█████████▌| 5379/5616 [1:08:31<02:49,  1.40it/s, gpt_loss=0.0375, lr=5.21e-7]
+
Train step of epoch 0:  96%|█████████▌| 5380/5616 [1:08:31<02:54,  1.35it/s, gpt_loss=0.0375, lr=5.21e-7]
+
Train step of epoch 0:  96%|█████████▌| 5380/5616 [1:08:32<02:54,  1.35it/s, gpt_loss=0.0032, lr=5.21e-7]
+
Train step of epoch 0:  96%|█████████▌| 5381/5616 [1:08:32<02:57,  1.32it/s, gpt_loss=0.0032, lr=5.21e-7]
+
Train step of epoch 0:  96%|█████████▌| 5381/5616 [1:08:32<02:57,  1.32it/s, gpt_loss=0.00479, lr=5.21e-7]
+
Train step of epoch 0:  96%|█████████▌| 5382/5616 [1:08:32<02:52,  1.36it/s, gpt_loss=0.00479, lr=5.21e-7]
+
Train step of epoch 0:  96%|█████████▌| 5382/5616 [1:08:33<02:52,  1.36it/s, gpt_loss=0.0935, lr=5.21e-7] 
+
Train step of epoch 0:  96%|█████████▌| 5383/5616 [1:08:33<02:50,  1.37it/s, gpt_loss=0.0935, lr=5.21e-7]
+
Train step of epoch 0:  96%|█████████▌| 5383/5616 [1:08:34<02:50,  1.37it/s, gpt_loss=0.0232, lr=5.2e-7] 
+
Train step of epoch 0:  96%|█████████▌| 5384/5616 [1:08:34<02:52,  1.35it/s, gpt_loss=0.0232, lr=5.2e-7]
+
Train step of epoch 0:  96%|█████████▌| 5384/5616 [1:08:35<02:52,  1.35it/s, gpt_loss=0.0023, lr=5.2e-7]
+
Train step of epoch 0:  96%|█████████▌| 5385/5616 [1:08:35<02:47,  1.38it/s, gpt_loss=0.0023, lr=5.2e-7]
+
Train step of epoch 0:  96%|█████████▌| 5385/5616 [1:08:35<02:47,  1.38it/s, gpt_loss=0.0146, lr=5.2e-7]
+
Train step of epoch 0:  96%|█████████▌| 5386/5616 [1:08:35<02:43,  1.41it/s, gpt_loss=0.0146, lr=5.2e-7]
+
Train step of epoch 0:  96%|█████████▌| 5386/5616 [1:08:36<02:43,  1.41it/s, gpt_loss=0.0841, lr=5.2e-7]
+
Train step of epoch 0:  96%|█████████▌| 5387/5616 [1:08:36<02:41,  1.42it/s, gpt_loss=0.0841, lr=5.2e-7]
+
Train step of epoch 0:  96%|█████████▌| 5387/5616 [1:08:37<02:41,  1.42it/s, gpt_loss=0.0108, lr=5.19e-7]
+
Train step of epoch 0:  96%|█████████▌| 5388/5616 [1:08:37<02:45,  1.38it/s, gpt_loss=0.0108, lr=5.19e-7]
+
Train step of epoch 0:  96%|█████████▌| 5388/5616 [1:08:37<02:45,  1.38it/s, gpt_loss=0.00951, lr=5.19e-7]
+
Train step of epoch 0:  96%|█████████▌| 5389/5616 [1:08:37<02:43,  1.39it/s, gpt_loss=0.00951, lr=5.19e-7]
+
Train step of epoch 0:  96%|█████████▌| 5389/5616 [1:08:38<02:43,  1.39it/s, gpt_loss=0.00575, lr=5.19e-7]
+
Train step of epoch 0:  96%|█████████▌| 5390/5616 [1:08:38<02:41,  1.40it/s, gpt_loss=0.00575, lr=5.19e-7]
+
Train step of epoch 0:  96%|█████████▌| 5390/5616 [1:08:39<02:41,  1.40it/s, gpt_loss=0.0936, lr=5.19e-7] 
+
Train step of epoch 0:  96%|█████████▌| 5391/5616 [1:08:39<02:44,  1.37it/s, gpt_loss=0.0936, lr=5.19e-7]
+
Train step of epoch 0:  96%|█████████▌| 5391/5616 [1:08:40<02:44,  1.37it/s, gpt_loss=0.043, lr=5.19e-7] 
+
Train step of epoch 0:  96%|█████████▌| 5392/5616 [1:08:40<02:56,  1.27it/s, gpt_loss=0.043, lr=5.19e-7]
+
Train step of epoch 0:  96%|█████████▌| 5392/5616 [1:08:40<02:56,  1.27it/s, gpt_loss=0.0242, lr=5.19e-7]
+
Train step of epoch 0:  96%|█████████▌| 5393/5616 [1:08:40<02:45,  1.35it/s, gpt_loss=0.0242, lr=5.19e-7]
+
Train step of epoch 0:  96%|█████████▌| 5393/5616 [1:08:41<02:45,  1.35it/s, gpt_loss=0.0121, lr=5.19e-7]
+
Train step of epoch 0:  96%|█████████▌| 5394/5616 [1:08:41<02:41,  1.37it/s, gpt_loss=0.0121, lr=5.19e-7]
+
Train step of epoch 0:  96%|█████████▌| 5394/5616 [1:08:42<02:41,  1.37it/s, gpt_loss=0.12, lr=5.19e-7]  
+
Train step of epoch 0:  96%|█████████▌| 5395/5616 [1:08:42<02:43,  1.35it/s, gpt_loss=0.12, lr=5.19e-7]
+
Train step of epoch 0:  96%|█████████▌| 5395/5616 [1:08:43<02:43,  1.35it/s, gpt_loss=0.0556, lr=5.18e-7]
+
Train step of epoch 0:  96%|█████████▌| 5396/5616 [1:08:43<02:45,  1.33it/s, gpt_loss=0.0556, lr=5.18e-7]
+
Train step of epoch 0:  96%|█████████▌| 5396/5616 [1:08:43<02:45,  1.33it/s, gpt_loss=0.033, lr=5.18e-7] 
+
Train step of epoch 0:  96%|█████████▌| 5397/5616 [1:08:43<02:40,  1.37it/s, gpt_loss=0.033, lr=5.18e-7]
+
Train step of epoch 0:  96%|█████████▌| 5397/5616 [1:08:44<02:40,  1.37it/s, gpt_loss=0.0469, lr=5.18e-7]
+
Train step of epoch 0:  96%|█████████▌| 5398/5616 [1:08:44<02:37,  1.38it/s, gpt_loss=0.0469, lr=5.18e-7]
+
Train step of epoch 0:  96%|█████████▌| 5398/5616 [1:08:45<02:37,  1.38it/s, gpt_loss=0.0321, lr=5.18e-7]
+
Train step of epoch 0:  96%|█████████▌| 5399/5616 [1:08:45<02:36,  1.39it/s, gpt_loss=0.0321, lr=5.18e-7]
+
Train step of epoch 0:  96%|█████████▌| 5399/5616 [1:08:46<02:36,  1.39it/s, gpt_loss=0.0272, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▌| 5400/5616 [1:08:46<03:00,  1.20it/s, gpt_loss=0.0272, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▌| 5400/5616 [1:08:47<03:00,  1.20it/s, gpt_loss=0.0782, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▌| 5401/5616 [1:08:47<02:58,  1.20it/s, gpt_loss=0.0782, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▌| 5401/5616 [1:08:48<02:58,  1.20it/s, gpt_loss=0.00355, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▌| 5402/5616 [1:08:48<03:05,  1.16it/s, gpt_loss=0.00355, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▌| 5402/5616 [1:08:48<03:05,  1.16it/s, gpt_loss=0.0346, lr=5.17e-7] 
+
Train step of epoch 0:  96%|█████████▌| 5403/5616 [1:08:48<02:56,  1.21it/s, gpt_loss=0.0346, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▌| 5403/5616 [1:08:49<02:56,  1.21it/s, gpt_loss=0.0878, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▌| 5404/5616 [1:08:49<02:51,  1.23it/s, gpt_loss=0.0878, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▌| 5404/5616 [1:08:50<02:51,  1.23it/s, gpt_loss=0.00983, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▌| 5405/5616 [1:08:50<03:02,  1.16it/s, gpt_loss=0.00983, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▌| 5405/5616 [1:08:51<03:02,  1.16it/s, gpt_loss=0.00489, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▋| 5406/5616 [1:08:51<02:53,  1.21it/s, gpt_loss=0.00489, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▋| 5406/5616 [1:08:52<02:53,  1.21it/s, gpt_loss=0.024, lr=5.17e-7]  
+
Train step of epoch 0:  96%|█████████▋| 5407/5616 [1:08:52<02:48,  1.24it/s, gpt_loss=0.024, lr=5.17e-7]
+
Train step of epoch 0:  96%|█████████▋| 5407/5616 [1:08:52<02:48,  1.24it/s, gpt_loss=0.00644, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5408/5616 [1:08:52<02:44,  1.26it/s, gpt_loss=0.00644, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5408/5616 [1:08:53<02:44,  1.26it/s, gpt_loss=0.00725, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5409/5616 [1:08:53<02:35,  1.33it/s, gpt_loss=0.00725, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5409/5616 [1:08:54<02:35,  1.33it/s, gpt_loss=0.00215, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5410/5616 [1:08:54<02:33,  1.35it/s, gpt_loss=0.00215, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5410/5616 [1:08:54<02:33,  1.35it/s, gpt_loss=0.0583, lr=5.16e-7] 
+
Train step of epoch 0:  96%|█████████▋| 5411/5616 [1:08:54<02:27,  1.39it/s, gpt_loss=0.0583, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5411/5616 [1:08:55<02:27,  1.39it/s, gpt_loss=0.00158, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5412/5616 [1:08:55<02:38,  1.29it/s, gpt_loss=0.00158, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5412/5616 [1:08:56<02:38,  1.29it/s, gpt_loss=0.0102, lr=5.16e-7] 
+
Train step of epoch 0:  96%|█████████▋| 5413/5616 [1:08:56<02:40,  1.26it/s, gpt_loss=0.0102, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5413/5616 [1:08:57<02:40,  1.26it/s, gpt_loss=0.0455, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5414/5616 [1:08:57<02:35,  1.30it/s, gpt_loss=0.0455, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5414/5616 [1:08:58<02:35,  1.30it/s, gpt_loss=0.0132, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5415/5616 [1:08:58<02:31,  1.32it/s, gpt_loss=0.0132, lr=5.16e-7]
+
Train step of epoch 0:  96%|█████████▋| 5415/5616 [1:08:58<02:31,  1.32it/s, gpt_loss=0.000782, lr=5.15e-7]
+
Train step of epoch 0:  96%|█████████▋| 5416/5616 [1:08:58<02:31,  1.32it/s, gpt_loss=0.000782, lr=5.15e-7]
+
Train step of epoch 0:  96%|█████████▋| 5416/5616 [1:08:59<02:31,  1.32it/s, gpt_loss=0.0372, lr=5.15e-7]  
+
Train step of epoch 0:  96%|█████████▋| 5417/5616 [1:08:59<02:29,  1.33it/s, gpt_loss=0.0372, lr=5.15e-7]
+
Train step of epoch 0:  96%|█████████▋| 5417/5616 [1:09:00<02:29,  1.33it/s, gpt_loss=0.0246, lr=5.15e-7]
+
Train step of epoch 0:  96%|█████████▋| 5418/5616 [1:09:00<02:25,  1.36it/s, gpt_loss=0.0246, lr=5.15e-7]
+
Train step of epoch 0:  96%|█████████▋| 5418/5616 [1:09:01<02:25,  1.36it/s, gpt_loss=0.0368, lr=5.15e-7]
+
Train step of epoch 0:  96%|█████████▋| 5419/5616 [1:09:01<02:26,  1.34it/s, gpt_loss=0.0368, lr=5.15e-7]
+
Train step of epoch 0:  96%|█████████▋| 5419/5616 [1:09:01<02:26,  1.34it/s, gpt_loss=0.0141, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5420/5616 [1:09:01<02:29,  1.31it/s, gpt_loss=0.0141, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5420/5616 [1:09:02<02:29,  1.31it/s, gpt_loss=0.0999, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5421/5616 [1:09:02<02:25,  1.34it/s, gpt_loss=0.0999, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5421/5616 [1:09:03<02:25,  1.34it/s, gpt_loss=0.0328, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5422/5616 [1:09:03<02:22,  1.37it/s, gpt_loss=0.0328, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5422/5616 [1:09:04<02:22,  1.37it/s, gpt_loss=0.073, lr=5.14e-7] 
+
Train step of epoch 0:  97%|█████████▋| 5423/5616 [1:09:04<02:24,  1.33it/s, gpt_loss=0.073, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5423/5616 [1:09:04<02:24,  1.33it/s, gpt_loss=0.0354, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5424/5616 [1:09:05<02:32,  1.25it/s, gpt_loss=0.0354, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5424/5616 [1:09:05<02:32,  1.25it/s, gpt_loss=0.0787, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5425/5616 [1:09:05<02:24,  1.32it/s, gpt_loss=0.0787, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5425/5616 [1:09:06<02:24,  1.32it/s, gpt_loss=0.0248, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5426/5616 [1:09:06<02:24,  1.32it/s, gpt_loss=0.0248, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5426/5616 [1:09:07<02:24,  1.32it/s, gpt_loss=0.0421, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5427/5616 [1:09:07<02:25,  1.30it/s, gpt_loss=0.0421, lr=5.14e-7]
+
Train step of epoch 0:  97%|█████████▋| 5427/5616 [1:09:07<02:25,  1.30it/s, gpt_loss=0.0359, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5428/5616 [1:09:07<02:24,  1.30it/s, gpt_loss=0.0359, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5428/5616 [1:09:08<02:24,  1.30it/s, gpt_loss=0.00757, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5429/5616 [1:09:08<02:17,  1.36it/s, gpt_loss=0.00757, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5429/5616 [1:09:09<02:17,  1.36it/s, gpt_loss=0.0026, lr=5.13e-7] 
+
Train step of epoch 0:  97%|█████████▋| 5430/5616 [1:09:09<02:12,  1.41it/s, gpt_loss=0.0026, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5430/5616 [1:09:09<02:12,  1.41it/s, gpt_loss=0.0475, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5431/5616 [1:09:09<02:07,  1.45it/s, gpt_loss=0.0475, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5431/5616 [1:09:10<02:07,  1.45it/s, gpt_loss=0.00123, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5432/5616 [1:09:10<02:10,  1.41it/s, gpt_loss=0.00123, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5432/5616 [1:09:11<02:10,  1.41it/s, gpt_loss=0.0108, lr=5.13e-7] 
+
Train step of epoch 0:  97%|█████████▋| 5433/5616 [1:09:11<02:07,  1.43it/s, gpt_loss=0.0108, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5433/5616 [1:09:12<02:07,  1.43it/s, gpt_loss=0.0102, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5434/5616 [1:09:12<02:08,  1.42it/s, gpt_loss=0.0102, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5434/5616 [1:09:12<02:08,  1.42it/s, gpt_loss=0.0159, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5435/5616 [1:09:12<02:13,  1.36it/s, gpt_loss=0.0159, lr=5.13e-7]
+
Train step of epoch 0:  97%|█████████▋| 5435/5616 [1:09:13<02:13,  1.36it/s, gpt_loss=0.00628, lr=5.12e-7]
+
Train step of epoch 0:  97%|█████████▋| 5436/5616 [1:09:13<02:14,  1.34it/s, gpt_loss=0.00628, lr=5.12e-7]
+
Train step of epoch 0:  97%|█████████▋| 5436/5616 [1:09:14<02:14,  1.34it/s, gpt_loss=0.0272, lr=5.12e-7] 
+
Train step of epoch 0:  97%|█████████▋| 5437/5616 [1:09:14<02:06,  1.41it/s, gpt_loss=0.0272, lr=5.12e-7]
+
Train step of epoch 0:  97%|█████████▋| 5437/5616 [1:09:15<02:06,  1.41it/s, gpt_loss=0.0155, lr=5.12e-7]
+
Train step of epoch 0:  97%|█████████▋| 5438/5616 [1:09:15<02:09,  1.37it/s, gpt_loss=0.0155, lr=5.12e-7]
+
Train step of epoch 0:  97%|█████████▋| 5438/5616 [1:09:15<02:09,  1.37it/s, gpt_loss=0.012, lr=5.12e-7] 
+
Train step of epoch 0:  97%|█████████▋| 5439/5616 [1:09:15<02:06,  1.39it/s, gpt_loss=0.012, lr=5.12e-7]
+
Train step of epoch 0:  97%|█████████▋| 5439/5616 [1:09:16<02:06,  1.39it/s, gpt_loss=0.0152, lr=5.12e-7]
+
Train step of epoch 0:  97%|█████████▋| 5440/5616 [1:09:16<02:11,  1.34it/s, gpt_loss=0.0152, lr=5.12e-7]
+
Train step of epoch 0:  97%|█████████▋| 5440/5616 [1:09:17<02:11,  1.34it/s, gpt_loss=0.0358, lr=5.12e-7]
+
Train step of epoch 0:  97%|█████████▋| 5441/5616 [1:09:17<02:26,  1.19it/s, gpt_loss=0.0358, lr=5.12e-7]
+
Train step of epoch 0:  97%|█████████▋| 5441/5616 [1:09:18<02:26,  1.19it/s, gpt_loss=0.053, lr=5.12e-7] 
+
Train step of epoch 0:  97%|█████████▋| 5442/5616 [1:09:18<02:20,  1.24it/s, gpt_loss=0.053, lr=5.12e-7]
+
Train step of epoch 0:  97%|█████████▋| 5442/5616 [1:09:19<02:20,  1.24it/s, gpt_loss=0.0017, lr=5.12e-7]
+
Train step of epoch 0:  97%|█████████▋| 5443/5616 [1:09:19<02:12,  1.30it/s, gpt_loss=0.0017, lr=5.12e-7]
+
Train step of epoch 0:  97%|█████████▋| 5443/5616 [1:09:19<02:12,  1.30it/s, gpt_loss=0.079, lr=5.11e-7] 
+
Train step of epoch 0:  97%|█████████▋| 5444/5616 [1:09:19<02:16,  1.26it/s, gpt_loss=0.079, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5444/5616 [1:09:20<02:16,  1.26it/s, gpt_loss=0.0263, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5445/5616 [1:09:20<02:10,  1.31it/s, gpt_loss=0.0263, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5445/5616 [1:09:21<02:10,  1.31it/s, gpt_loss=0.00579, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5446/5616 [1:09:21<02:06,  1.35it/s, gpt_loss=0.00579, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5446/5616 [1:09:21<02:06,  1.35it/s, gpt_loss=0.043, lr=5.11e-7]  
+
Train step of epoch 0:  97%|█████████▋| 5447/5616 [1:09:21<02:03,  1.37it/s, gpt_loss=0.043, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5447/5616 [1:09:22<02:03,  1.37it/s, gpt_loss=0.0708, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5448/5616 [1:09:22<02:11,  1.28it/s, gpt_loss=0.0708, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5448/5616 [1:09:23<02:11,  1.28it/s, gpt_loss=0.0849, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5449/5616 [1:09:23<02:07,  1.31it/s, gpt_loss=0.0849, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5449/5616 [1:09:24<02:07,  1.31it/s, gpt_loss=0.0179, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5450/5616 [1:09:24<02:06,  1.31it/s, gpt_loss=0.0179, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5450/5616 [1:09:25<02:06,  1.31it/s, gpt_loss=0.0313, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5451/5616 [1:09:25<02:05,  1.31it/s, gpt_loss=0.0313, lr=5.11e-7]
+
Train step of epoch 0:  97%|█████████▋| 5451/5616 [1:09:25<02:05,  1.31it/s, gpt_loss=0.0117, lr=5.1e-7] 
+
Train step of epoch 0:  97%|█████████▋| 5452/5616 [1:09:25<02:07,  1.29it/s, gpt_loss=0.0117, lr=5.1e-7]
+
Train step of epoch 0:  97%|█████████▋| 5452/5616 [1:09:26<02:07,  1.29it/s, gpt_loss=0.0809, lr=5.1e-7]
+
Train step of epoch 0:  97%|█████████▋| 5453/5616 [1:09:26<02:05,  1.30it/s, gpt_loss=0.0809, lr=5.1e-7]
+
Train step of epoch 0:  97%|█████████▋| 5453/5616 [1:09:27<02:05,  1.30it/s, gpt_loss=0.00266, lr=5.1e-7]
+
Train step of epoch 0:  97%|█████████▋| 5454/5616 [1:09:27<02:06,  1.28it/s, gpt_loss=0.00266, lr=5.1e-7]
+
Train step of epoch 0:  97%|█████████▋| 5454/5616 [1:09:28<02:06,  1.28it/s, gpt_loss=0.0274, lr=5.1e-7] 
+
Train step of epoch 0:  97%|█████████▋| 5455/5616 [1:09:28<02:05,  1.28it/s, gpt_loss=0.0274, lr=5.1e-7]
+
Train step of epoch 0:  97%|█████████▋| 5455/5616 [1:09:29<02:05,  1.28it/s, gpt_loss=0.00443, lr=5.1e-7]
+
Train step of epoch 0:  97%|█████████▋| 5456/5616 [1:09:29<02:04,  1.28it/s, gpt_loss=0.00443, lr=5.1e-7]
+
Train step of epoch 0:  97%|█████████▋| 5456/5616 [1:09:30<02:04,  1.28it/s, gpt_loss=0.0357, lr=5.1e-7] 
+
Train step of epoch 0:  97%|█████████▋| 5457/5616 [1:09:30<02:14,  1.18it/s, gpt_loss=0.0357, lr=5.1e-7]
+
Train step of epoch 0:  97%|█████████▋| 5457/5616 [1:09:30<02:14,  1.18it/s, gpt_loss=0.00852, lr=5.1e-7]
+
Train step of epoch 0:  97%|█████████▋| 5458/5616 [1:09:30<02:03,  1.28it/s, gpt_loss=0.00852, lr=5.1e-7]
+
Train step of epoch 0:  97%|█████████▋| 5458/5616 [1:09:31<02:03,  1.28it/s, gpt_loss=0.0538, lr=5.1e-7] 
+
Train step of epoch 0:  97%|█████████▋| 5459/5616 [1:09:31<02:05,  1.25it/s, gpt_loss=0.0538, lr=5.1e-7]
+
Train step of epoch 0:  97%|█████████▋| 5459/5616 [1:09:32<02:05,  1.25it/s, gpt_loss=0.0404, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5460/5616 [1:09:32<02:05,  1.24it/s, gpt_loss=0.0404, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5460/5616 [1:09:33<02:05,  1.24it/s, gpt_loss=0.0618, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5461/5616 [1:09:33<02:02,  1.27it/s, gpt_loss=0.0618, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5461/5616 [1:09:33<02:02,  1.27it/s, gpt_loss=0.0144, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5462/5616 [1:09:33<02:02,  1.26it/s, gpt_loss=0.0144, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5462/5616 [1:09:34<02:02,  1.26it/s, gpt_loss=0.0108, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5463/5616 [1:09:34<02:04,  1.23it/s, gpt_loss=0.0108, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5463/5616 [1:09:35<02:04,  1.23it/s, gpt_loss=0.0213, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5464/5616 [1:09:35<02:07,  1.19it/s, gpt_loss=0.0213, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5464/5616 [1:09:36<02:07,  1.19it/s, gpt_loss=0.0876, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5465/5616 [1:09:36<01:58,  1.27it/s, gpt_loss=0.0876, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5465/5616 [1:09:37<01:58,  1.27it/s, gpt_loss=0.0131, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5466/5616 [1:09:37<01:53,  1.33it/s, gpt_loss=0.0131, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5466/5616 [1:09:37<01:53,  1.33it/s, gpt_loss=0.0716, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5467/5616 [1:09:37<01:51,  1.34it/s, gpt_loss=0.0716, lr=5.09e-7]
+
Train step of epoch 0:  97%|█████████▋| 5467/5616 [1:09:38<01:51,  1.34it/s, gpt_loss=0.0134, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5468/5616 [1:09:38<01:54,  1.29it/s, gpt_loss=0.0134, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5468/5616 [1:09:39<01:54,  1.29it/s, gpt_loss=0.0252, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5469/5616 [1:09:39<01:53,  1.30it/s, gpt_loss=0.0252, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5469/5616 [1:09:40<01:53,  1.30it/s, gpt_loss=0.0052, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5470/5616 [1:09:40<01:50,  1.32it/s, gpt_loss=0.0052, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5470/5616 [1:09:40<01:50,  1.32it/s, gpt_loss=0.0172, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5471/5616 [1:09:40<01:46,  1.36it/s, gpt_loss=0.0172, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5471/5616 [1:09:41<01:46,  1.36it/s, gpt_loss=0.0264, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5472/5616 [1:09:41<01:45,  1.37it/s, gpt_loss=0.0264, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5472/5616 [1:09:42<01:45,  1.37it/s, gpt_loss=0.0786, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5473/5616 [1:09:42<01:43,  1.38it/s, gpt_loss=0.0786, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5473/5616 [1:09:42<01:43,  1.38it/s, gpt_loss=0.0146, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5474/5616 [1:09:42<01:42,  1.38it/s, gpt_loss=0.0146, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5474/5616 [1:09:43<01:42,  1.38it/s, gpt_loss=0.037, lr=5.08e-7] 
+
Train step of epoch 0:  97%|█████████▋| 5475/5616 [1:09:43<01:44,  1.36it/s, gpt_loss=0.037, lr=5.08e-7]
+
Train step of epoch 0:  97%|█████████▋| 5475/5616 [1:09:44<01:44,  1.36it/s, gpt_loss=0.0424, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5476/5616 [1:09:44<01:44,  1.34it/s, gpt_loss=0.0424, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5476/5616 [1:09:45<01:44,  1.34it/s, gpt_loss=0.102, lr=5.07e-7] 
+
Train step of epoch 0:  98%|█████████▊| 5477/5616 [1:09:45<01:40,  1.38it/s, gpt_loss=0.102, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5477/5616 [1:09:45<01:40,  1.38it/s, gpt_loss=0.0761, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5478/5616 [1:09:45<01:42,  1.34it/s, gpt_loss=0.0761, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5478/5616 [1:09:46<01:42,  1.34it/s, gpt_loss=0.0642, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5479/5616 [1:09:46<01:42,  1.34it/s, gpt_loss=0.0642, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5479/5616 [1:09:47<01:42,  1.34it/s, gpt_loss=0.0396, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5480/5616 [1:09:47<01:41,  1.34it/s, gpt_loss=0.0396, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5480/5616 [1:09:48<01:41,  1.34it/s, gpt_loss=0.0531, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5481/5616 [1:09:48<01:43,  1.30it/s, gpt_loss=0.0531, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5481/5616 [1:09:48<01:43,  1.30it/s, gpt_loss=0.0085, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5482/5616 [1:09:48<01:43,  1.30it/s, gpt_loss=0.0085, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5482/5616 [1:09:49<01:43,  1.30it/s, gpt_loss=0.0146, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5483/5616 [1:09:49<01:45,  1.26it/s, gpt_loss=0.0146, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5483/5616 [1:09:50<01:45,  1.26it/s, gpt_loss=0.00925, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5484/5616 [1:09:50<01:41,  1.30it/s, gpt_loss=0.00925, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5484/5616 [1:09:51<01:41,  1.30it/s, gpt_loss=0.0332, lr=5.07e-7] 
+
Train step of epoch 0:  98%|█████████▊| 5485/5616 [1:09:51<01:39,  1.32it/s, gpt_loss=0.0332, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5485/5616 [1:09:52<01:39,  1.32it/s, gpt_loss=0.0357, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5486/5616 [1:09:52<01:39,  1.31it/s, gpt_loss=0.0357, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5486/5616 [1:09:52<01:39,  1.31it/s, gpt_loss=0.0279, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5487/5616 [1:09:52<01:34,  1.37it/s, gpt_loss=0.0279, lr=5.07e-7]
+
Train step of epoch 0:  98%|█████████▊| 5487/5616 [1:09:53<01:34,  1.37it/s, gpt_loss=0.038, lr=5.06e-7] 
+
Train step of epoch 0:  98%|█████████▊| 5488/5616 [1:09:53<01:32,  1.38it/s, gpt_loss=0.038, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5488/5616 [1:09:54<01:32,  1.38it/s, gpt_loss=0.00841, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5489/5616 [1:09:54<01:35,  1.33it/s, gpt_loss=0.00841, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5489/5616 [1:09:54<01:35,  1.33it/s, gpt_loss=0.00228, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5490/5616 [1:09:54<01:32,  1.36it/s, gpt_loss=0.00228, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5490/5616 [1:09:55<01:32,  1.36it/s, gpt_loss=0.00524, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5491/5616 [1:09:55<01:30,  1.39it/s, gpt_loss=0.00524, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5491/5616 [1:09:56<01:30,  1.39it/s, gpt_loss=0.00326, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5492/5616 [1:09:56<01:40,  1.24it/s, gpt_loss=0.00326, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5492/5616 [1:09:57<01:40,  1.24it/s, gpt_loss=0.00419, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5493/5616 [1:09:57<01:34,  1.30it/s, gpt_loss=0.00419, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5493/5616 [1:09:58<01:34,  1.30it/s, gpt_loss=0.0883, lr=5.06e-7] 
+
Train step of epoch 0:  98%|█████████▊| 5494/5616 [1:09:58<01:31,  1.34it/s, gpt_loss=0.0883, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5494/5616 [1:09:58<01:31,  1.34it/s, gpt_loss=0.0037, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5495/5616 [1:09:58<01:31,  1.33it/s, gpt_loss=0.0037, lr=5.06e-7]
+
Train step of epoch 0:  98%|█████████▊| 5495/5616 [1:09:59<01:31,  1.33it/s, gpt_loss=0.0596, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5496/5616 [1:09:59<01:30,  1.32it/s, gpt_loss=0.0596, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5496/5616 [1:10:00<01:30,  1.32it/s, gpt_loss=0.0951, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5497/5616 [1:10:00<01:31,  1.30it/s, gpt_loss=0.0951, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5497/5616 [1:10:01<01:31,  1.30it/s, gpt_loss=0.0541, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5498/5616 [1:10:01<01:35,  1.23it/s, gpt_loss=0.0541, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5498/5616 [1:10:01<01:35,  1.23it/s, gpt_loss=0.019, lr=5.05e-7] 
+
Train step of epoch 0:  98%|█████████▊| 5499/5616 [1:10:01<01:31,  1.28it/s, gpt_loss=0.019, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5499/5616 [1:10:02<01:31,  1.28it/s, gpt_loss=0.0127, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5500/5616 [1:10:02<01:31,  1.27it/s, gpt_loss=0.0127, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5500/5616 [1:10:03<01:31,  1.27it/s, gpt_loss=0.0302, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5501/5616 [1:10:03<01:27,  1.31it/s, gpt_loss=0.0302, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5501/5616 [1:10:04<01:27,  1.31it/s, gpt_loss=0.00406, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5502/5616 [1:10:04<01:26,  1.32it/s, gpt_loss=0.00406, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5502/5616 [1:10:05<01:26,  1.32it/s, gpt_loss=0.06, lr=5.05e-7]   
+
Train step of epoch 0:  98%|█████████▊| 5503/5616 [1:10:05<01:29,  1.26it/s, gpt_loss=0.06, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5503/5616 [1:10:05<01:29,  1.26it/s, gpt_loss=0.0578, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5504/5616 [1:10:05<01:28,  1.26it/s, gpt_loss=0.0578, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5504/5616 [1:10:06<01:28,  1.26it/s, gpt_loss=0.0398, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5505/5616 [1:10:06<01:28,  1.26it/s, gpt_loss=0.0398, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5505/5616 [1:10:07<01:28,  1.26it/s, gpt_loss=0.0279, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5506/5616 [1:10:07<01:32,  1.19it/s, gpt_loss=0.0279, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5506/5616 [1:10:08<01:32,  1.19it/s, gpt_loss=0.00423, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5507/5616 [1:10:08<01:27,  1.24it/s, gpt_loss=0.00423, lr=5.05e-7]
+
Train step of epoch 0:  98%|█████████▊| 5507/5616 [1:10:09<01:27,  1.24it/s, gpt_loss=0.0118, lr=5.04e-7] 
+
Train step of epoch 0:  98%|█████████▊| 5508/5616 [1:10:09<01:26,  1.25it/s, gpt_loss=0.0118, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5508/5616 [1:10:09<01:26,  1.25it/s, gpt_loss=0.00739, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5509/5616 [1:10:09<01:23,  1.29it/s, gpt_loss=0.00739, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5509/5616 [1:10:10<01:23,  1.29it/s, gpt_loss=0.0165, lr=5.04e-7] 
+
Train step of epoch 0:  98%|█████████▊| 5510/5616 [1:10:10<01:19,  1.34it/s, gpt_loss=0.0165, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5510/5616 [1:10:11<01:19,  1.34it/s, gpt_loss=0.0188, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5511/5616 [1:10:11<01:18,  1.34it/s, gpt_loss=0.0188, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5511/5616 [1:10:12<01:18,  1.34it/s, gpt_loss=0.0752, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5512/5616 [1:10:12<01:17,  1.34it/s, gpt_loss=0.0752, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5512/5616 [1:10:13<01:17,  1.34it/s, gpt_loss=0.0554, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5513/5616 [1:10:13<01:25,  1.20it/s, gpt_loss=0.0554, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5513/5616 [1:10:13<01:25,  1.20it/s, gpt_loss=0.0057, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5514/5616 [1:10:13<01:21,  1.26it/s, gpt_loss=0.0057, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5514/5616 [1:10:14<01:21,  1.26it/s, gpt_loss=0.0294, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5515/5616 [1:10:14<01:19,  1.27it/s, gpt_loss=0.0294, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5515/5616 [1:10:15<01:19,  1.27it/s, gpt_loss=0.0379, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5516/5616 [1:10:15<01:19,  1.26it/s, gpt_loss=0.0379, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5516/5616 [1:10:16<01:19,  1.26it/s, gpt_loss=0.00338, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5517/5616 [1:10:16<01:17,  1.28it/s, gpt_loss=0.00338, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5517/5616 [1:10:17<01:17,  1.28it/s, gpt_loss=0.0634, lr=5.04e-7] 
+
Train step of epoch 0:  98%|█████████▊| 5518/5616 [1:10:17<01:23,  1.18it/s, gpt_loss=0.0634, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5518/5616 [1:10:18<01:23,  1.18it/s, gpt_loss=0.00293, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5519/5616 [1:10:18<01:24,  1.15it/s, gpt_loss=0.00293, lr=5.04e-7]
+
Train step of epoch 0:  98%|█████████▊| 5519/5616 [1:10:18<01:24,  1.15it/s, gpt_loss=0.00712, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5520/5616 [1:10:18<01:20,  1.19it/s, gpt_loss=0.00712, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5520/5616 [1:10:19<01:20,  1.19it/s, gpt_loss=0.0857, lr=5.03e-7] 
+
Train step of epoch 0:  98%|█████████▊| 5521/5616 [1:10:19<01:21,  1.16it/s, gpt_loss=0.0857, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5521/5616 [1:10:20<01:21,  1.16it/s, gpt_loss=0.0119, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5522/5616 [1:10:20<01:22,  1.15it/s, gpt_loss=0.0119, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5522/5616 [1:10:21<01:22,  1.15it/s, gpt_loss=0.103, lr=5.03e-7] 
+
Train step of epoch 0:  98%|█████████▊| 5523/5616 [1:10:21<01:18,  1.19it/s, gpt_loss=0.103, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5523/5616 [1:10:22<01:18,  1.19it/s, gpt_loss=0.0115, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5524/5616 [1:10:22<01:15,  1.21it/s, gpt_loss=0.0115, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5524/5616 [1:10:22<01:15,  1.21it/s, gpt_loss=0.0184, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5525/5616 [1:10:22<01:10,  1.28it/s, gpt_loss=0.0184, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5525/5616 [1:10:23<01:10,  1.28it/s, gpt_loss=0.0229, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5526/5616 [1:10:23<01:10,  1.29it/s, gpt_loss=0.0229, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5526/5616 [1:10:24<01:10,  1.29it/s, gpt_loss=0.0178, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5527/5616 [1:10:24<01:06,  1.33it/s, gpt_loss=0.0178, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5527/5616 [1:10:25<01:06,  1.33it/s, gpt_loss=0.0467, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5528/5616 [1:10:25<01:05,  1.35it/s, gpt_loss=0.0467, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5528/5616 [1:10:25<01:05,  1.35it/s, gpt_loss=0.0138, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5529/5616 [1:10:25<01:03,  1.37it/s, gpt_loss=0.0138, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5529/5616 [1:10:26<01:03,  1.37it/s, gpt_loss=0.00858, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5530/5616 [1:10:26<01:01,  1.41it/s, gpt_loss=0.00858, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5530/5616 [1:10:27<01:01,  1.41it/s, gpt_loss=0.0799, lr=5.03e-7] 
+
Train step of epoch 0:  98%|█████████▊| 5531/5616 [1:10:27<01:00,  1.41it/s, gpt_loss=0.0799, lr=5.03e-7]
+
Train step of epoch 0:  98%|█████████▊| 5531/5616 [1:10:27<01:00,  1.41it/s, gpt_loss=0.00208, lr=5.03e-7]
+
Train step of epoch 0:  99%|█████████▊| 5532/5616 [1:10:27<01:02,  1.35it/s, gpt_loss=0.00208, lr=5.03e-7]
+
Train step of epoch 0:  99%|█████████▊| 5532/5616 [1:10:28<01:02,  1.35it/s, gpt_loss=0.0439, lr=5.03e-7] 
+
Train step of epoch 0:  99%|█████████▊| 5533/5616 [1:10:28<01:01,  1.35it/s, gpt_loss=0.0439, lr=5.03e-7]
+
Train step of epoch 0:  99%|█████████▊| 5533/5616 [1:10:29<01:01,  1.35it/s, gpt_loss=0.0337, lr=5.03e-7]
+
Train step of epoch 0:  99%|█████████▊| 5534/5616 [1:10:29<00:59,  1.39it/s, gpt_loss=0.0337, lr=5.03e-7]
+
Train step of epoch 0:  99%|█████████▊| 5534/5616 [1:10:30<00:59,  1.39it/s, gpt_loss=0.0111, lr=5.03e-7]
+
Train step of epoch 0:  99%|█████████▊| 5535/5616 [1:10:30<00:57,  1.40it/s, gpt_loss=0.0111, lr=5.03e-7]
+
Train step of epoch 0:  99%|█████████▊| 5535/5616 [1:10:31<00:57,  1.40it/s, gpt_loss=0.132, lr=5.02e-7] 
+
Train step of epoch 0:  99%|█████████▊| 5536/5616 [1:10:31<01:06,  1.20it/s, gpt_loss=0.132, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5536/5616 [1:10:31<01:06,  1.20it/s, gpt_loss=0.0896, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5537/5616 [1:10:31<01:02,  1.26it/s, gpt_loss=0.0896, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5537/5616 [1:10:32<01:02,  1.26it/s, gpt_loss=0.0513, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5538/5616 [1:10:32<00:58,  1.33it/s, gpt_loss=0.0513, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5538/5616 [1:10:33<00:58,  1.33it/s, gpt_loss=0.0266, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5539/5616 [1:10:33<00:57,  1.34it/s, gpt_loss=0.0266, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5539/5616 [1:10:33<00:57,  1.34it/s, gpt_loss=0.0164, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5540/5616 [1:10:33<00:57,  1.32it/s, gpt_loss=0.0164, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5540/5616 [1:10:34<00:57,  1.32it/s, gpt_loss=0.0359, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5541/5616 [1:10:34<00:56,  1.32it/s, gpt_loss=0.0359, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5541/5616 [1:10:35<00:56,  1.32it/s, gpt_loss=0.119, lr=5.02e-7] 
+
Train step of epoch 0:  99%|█████████▊| 5542/5616 [1:10:35<00:54,  1.36it/s, gpt_loss=0.119, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5542/5616 [1:10:36<00:54,  1.36it/s, gpt_loss=0.0121, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5543/5616 [1:10:36<00:54,  1.33it/s, gpt_loss=0.0121, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5543/5616 [1:10:36<00:54,  1.33it/s, gpt_loss=0.0139, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5544/5616 [1:10:36<00:53,  1.34it/s, gpt_loss=0.0139, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5544/5616 [1:10:37<00:53,  1.34it/s, gpt_loss=0.00685, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5545/5616 [1:10:37<00:55,  1.28it/s, gpt_loss=0.00685, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▊| 5545/5616 [1:10:38<00:55,  1.28it/s, gpt_loss=0.105, lr=5.02e-7]  
+
Train step of epoch 0:  99%|█████████▉| 5546/5616 [1:10:38<00:52,  1.32it/s, gpt_loss=0.105, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5546/5616 [1:10:39<00:52,  1.32it/s, gpt_loss=0.0142, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5547/5616 [1:10:39<00:50,  1.37it/s, gpt_loss=0.0142, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5547/5616 [1:10:39<00:50,  1.37it/s, gpt_loss=0.0155, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5548/5616 [1:10:39<00:50,  1.36it/s, gpt_loss=0.0155, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5548/5616 [1:10:40<00:50,  1.36it/s, gpt_loss=0.0104, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5549/5616 [1:10:40<00:47,  1.40it/s, gpt_loss=0.0104, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5549/5616 [1:10:41<00:47,  1.40it/s, gpt_loss=0.0158, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5550/5616 [1:10:41<00:46,  1.41it/s, gpt_loss=0.0158, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5550/5616 [1:10:42<00:46,  1.41it/s, gpt_loss=0.0319, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5551/5616 [1:10:42<00:47,  1.38it/s, gpt_loss=0.0319, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5551/5616 [1:10:42<00:47,  1.38it/s, gpt_loss=0.0439, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5552/5616 [1:10:42<00:46,  1.37it/s, gpt_loss=0.0439, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5552/5616 [1:10:43<00:46,  1.37it/s, gpt_loss=0.0458, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5553/5616 [1:10:43<00:45,  1.38it/s, gpt_loss=0.0458, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5553/5616 [1:10:44<00:45,  1.38it/s, gpt_loss=0.00169, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5554/5616 [1:10:44<00:43,  1.41it/s, gpt_loss=0.00169, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5554/5616 [1:10:44<00:43,  1.41it/s, gpt_loss=0.0149, lr=5.02e-7] 
+
Train step of epoch 0:  99%|█████████▉| 5555/5616 [1:10:44<00:41,  1.46it/s, gpt_loss=0.0149, lr=5.02e-7]
+
Train step of epoch 0:  99%|█████████▉| 5555/5616 [1:10:45<00:41,  1.46it/s, gpt_loss=0.0166, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5556/5616 [1:10:45<00:44,  1.36it/s, gpt_loss=0.0166, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5556/5616 [1:10:46<00:44,  1.36it/s, gpt_loss=0.0264, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5557/5616 [1:10:46<00:43,  1.36it/s, gpt_loss=0.0264, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5557/5616 [1:10:47<00:43,  1.36it/s, gpt_loss=0.0342, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5558/5616 [1:10:47<00:43,  1.34it/s, gpt_loss=0.0342, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5558/5616 [1:10:47<00:43,  1.34it/s, gpt_loss=0.0753, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5559/5616 [1:10:47<00:41,  1.38it/s, gpt_loss=0.0753, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5559/5616 [1:10:48<00:41,  1.38it/s, gpt_loss=0.0134, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5560/5616 [1:10:48<00:44,  1.25it/s, gpt_loss=0.0134, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5560/5616 [1:10:49<00:44,  1.25it/s, gpt_loss=0.0294, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5561/5616 [1:10:49<00:42,  1.30it/s, gpt_loss=0.0294, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5561/5616 [1:10:50<00:42,  1.30it/s, gpt_loss=0.0402, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5562/5616 [1:10:50<00:40,  1.33it/s, gpt_loss=0.0402, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5562/5616 [1:10:51<00:40,  1.33it/s, gpt_loss=0.0616, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5563/5616 [1:10:51<00:43,  1.23it/s, gpt_loss=0.0616, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5563/5616 [1:10:52<00:43,  1.23it/s, gpt_loss=0.00718, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5564/5616 [1:10:52<00:48,  1.08it/s, gpt_loss=0.00718, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5564/5616 [1:10:53<00:48,  1.08it/s, gpt_loss=0.0123, lr=5.01e-7] 
+
Train step of epoch 0:  99%|█████████▉| 5565/5616 [1:10:53<00:43,  1.18it/s, gpt_loss=0.0123, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5565/5616 [1:10:53<00:43,  1.18it/s, gpt_loss=0.0389, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5566/5616 [1:10:53<00:41,  1.22it/s, gpt_loss=0.0389, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5566/5616 [1:10:54<00:41,  1.22it/s, gpt_loss=0.0285, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5567/5616 [1:10:54<00:38,  1.29it/s, gpt_loss=0.0285, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5567/5616 [1:10:55<00:38,  1.29it/s, gpt_loss=0.00987, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5568/5616 [1:10:55<00:37,  1.27it/s, gpt_loss=0.00987, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5568/5616 [1:10:56<00:37,  1.27it/s, gpt_loss=0.0496, lr=5.01e-7] 
+
Train step of epoch 0:  99%|█████████▉| 5569/5616 [1:10:56<00:35,  1.31it/s, gpt_loss=0.0496, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5569/5616 [1:10:56<00:35,  1.31it/s, gpt_loss=0.0199, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5570/5616 [1:10:56<00:34,  1.33it/s, gpt_loss=0.0199, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5570/5616 [1:10:57<00:34,  1.33it/s, gpt_loss=0.00808, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5571/5616 [1:10:57<00:33,  1.34it/s, gpt_loss=0.00808, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5571/5616 [1:10:58<00:33,  1.34it/s, gpt_loss=0.00684, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5572/5616 [1:10:58<00:33,  1.32it/s, gpt_loss=0.00684, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5572/5616 [1:10:59<00:33,  1.32it/s, gpt_loss=0.0217, lr=5.01e-7] 
+
Train step of epoch 0:  99%|█████████▉| 5573/5616 [1:10:59<00:32,  1.31it/s, gpt_loss=0.0217, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5573/5616 [1:10:59<00:32,  1.31it/s, gpt_loss=0.114, lr=5.01e-7] 
+
Train step of epoch 0:  99%|█████████▉| 5574/5616 [1:10:59<00:31,  1.34it/s, gpt_loss=0.114, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5574/5616 [1:11:00<00:31,  1.34it/s, gpt_loss=0.0264, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5575/5616 [1:11:00<00:30,  1.36it/s, gpt_loss=0.0264, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5575/5616 [1:11:01<00:30,  1.36it/s, gpt_loss=0.0803, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5576/5616 [1:11:01<00:30,  1.30it/s, gpt_loss=0.0803, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5576/5616 [1:11:02<00:30,  1.30it/s, gpt_loss=0.0128, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5577/5616 [1:11:02<00:31,  1.23it/s, gpt_loss=0.0128, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5577/5616 [1:11:02<00:31,  1.23it/s, gpt_loss=0.00291, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5578/5616 [1:11:02<00:29,  1.29it/s, gpt_loss=0.00291, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5578/5616 [1:11:03<00:29,  1.29it/s, gpt_loss=0.0558, lr=5.01e-7] 
+
Train step of epoch 0:  99%|█████████▉| 5579/5616 [1:11:03<00:27,  1.32it/s, gpt_loss=0.0558, lr=5.01e-7]
+
Train step of epoch 0:  99%|█████████▉| 5579/5616 [1:11:04<00:27,  1.32it/s, gpt_loss=0.016, lr=5e-7]    
+
Train step of epoch 0:  99%|█████████▉| 5580/5616 [1:11:04<00:27,  1.31it/s, gpt_loss=0.016, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5580/5616 [1:11:05<00:27,  1.31it/s, gpt_loss=0.117, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5581/5616 [1:11:05<00:25,  1.35it/s, gpt_loss=0.117, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5581/5616 [1:11:05<00:25,  1.35it/s, gpt_loss=0.0138, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5582/5616 [1:11:05<00:24,  1.40it/s, gpt_loss=0.0138, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5582/5616 [1:11:06<00:24,  1.40it/s, gpt_loss=0.0127, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5583/5616 [1:11:06<00:23,  1.42it/s, gpt_loss=0.0127, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5583/5616 [1:11:07<00:23,  1.42it/s, gpt_loss=0.0138, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5584/5616 [1:11:07<00:22,  1.41it/s, gpt_loss=0.0138, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5584/5616 [1:11:07<00:22,  1.41it/s, gpt_loss=0.00743, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5585/5616 [1:11:07<00:21,  1.44it/s, gpt_loss=0.00743, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5585/5616 [1:11:08<00:21,  1.44it/s, gpt_loss=0.00596, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5586/5616 [1:11:08<00:21,  1.41it/s, gpt_loss=0.00596, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5586/5616 [1:11:09<00:21,  1.41it/s, gpt_loss=0.0682, lr=5e-7] 
+
Train step of epoch 0:  99%|█████████▉| 5587/5616 [1:11:09<00:20,  1.41it/s, gpt_loss=0.0682, lr=5e-7]
+
Train step of epoch 0:  99%|█████████▉| 5587/5616 [1:11:10<00:20,  1.41it/s, gpt_loss=0.00363, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5588/5616 [1:11:10<00:20,  1.34it/s, gpt_loss=0.00363, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5588/5616 [1:11:10<00:20,  1.34it/s, gpt_loss=0.0429, lr=5e-7] 
+
Train step of epoch 0: 100%|█████████▉| 5589/5616 [1:11:10<00:19,  1.37it/s, gpt_loss=0.0429, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5589/5616 [1:11:11<00:19,  1.37it/s, gpt_loss=0.00448, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5590/5616 [1:11:11<00:18,  1.39it/s, gpt_loss=0.00448, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5590/5616 [1:11:12<00:18,  1.39it/s, gpt_loss=0.00857, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5591/5616 [1:11:12<00:18,  1.38it/s, gpt_loss=0.00857, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5591/5616 [1:11:12<00:18,  1.38it/s, gpt_loss=0.067, lr=5e-7]  
+
Train step of epoch 0: 100%|█████████▉| 5592/5616 [1:11:12<00:17,  1.38it/s, gpt_loss=0.067, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5592/5616 [1:11:13<00:17,  1.38it/s, gpt_loss=0.0611, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5593/5616 [1:11:13<00:16,  1.36it/s, gpt_loss=0.0611, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5593/5616 [1:11:14<00:16,  1.36it/s, gpt_loss=0.0156, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5594/5616 [1:11:14<00:15,  1.38it/s, gpt_loss=0.0156, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5594/5616 [1:11:15<00:15,  1.38it/s, gpt_loss=0.0527, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5595/5616 [1:11:15<00:15,  1.36it/s, gpt_loss=0.0527, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5595/5616 [1:11:15<00:15,  1.36it/s, gpt_loss=0.059, lr=5e-7] 
+
Train step of epoch 0: 100%|█████████▉| 5596/5616 [1:11:15<00:14,  1.35it/s, gpt_loss=0.059, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5596/5616 [1:11:16<00:14,  1.35it/s, gpt_loss=0.0786, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5597/5616 [1:11:16<00:13,  1.36it/s, gpt_loss=0.0786, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5597/5616 [1:11:17<00:13,  1.36it/s, gpt_loss=0.0402, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5598/5616 [1:11:17<00:13,  1.32it/s, gpt_loss=0.0402, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5598/5616 [1:11:18<00:13,  1.32it/s, gpt_loss=0.0122, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5599/5616 [1:11:18<00:12,  1.35it/s, gpt_loss=0.0122, lr=5e-7][2025-06-30 07:49:27,400] [INFO] [logging.py:128:log_dist] [Rank 0] step=1400, skipped=0, lr=[5.000959075760203e-07, 5.000959075760203e-07], mom=[(0.9, 0.95), (0.9, 0.95)]
+[2025-06-30 07:49:27,401] [INFO] [timer.py:264:stop] epoch=0/micro_step=5600/global_step=1400, RunningAvgSamplesPerSec=42.42336034697613, CurrSamplesPerSec=43.673520707207736, MemAllocated=16.08GB, MaxMemAllocated=54.86GB
+
+
Train step of epoch 0: 100%|█████████▉| 5599/5616 [1:11:18<00:12,  1.35it/s, gpt_loss=0.00978, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5600/5616 [1:11:18<00:11,  1.35it/s, gpt_loss=0.00978, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5600/5616 [1:11:19<00:11,  1.35it/s, gpt_loss=0.0411, lr=5e-7] 
+
Train step of epoch 0: 100%|█████████▉| 5601/5616 [1:11:19<00:11,  1.36it/s, gpt_loss=0.0411, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5601/5616 [1:11:20<00:11,  1.36it/s, gpt_loss=0.0459, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5602/5616 [1:11:20<00:10,  1.35it/s, gpt_loss=0.0459, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5602/5616 [1:11:21<00:10,  1.35it/s, gpt_loss=0.00646, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5603/5616 [1:11:21<00:09,  1.39it/s, gpt_loss=0.00646, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5603/5616 [1:11:21<00:09,  1.39it/s, gpt_loss=0.0606, lr=5e-7] 
+
Train step of epoch 0: 100%|█████████▉| 5604/5616 [1:11:21<00:09,  1.31it/s, gpt_loss=0.0606, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5604/5616 [1:11:22<00:09,  1.31it/s, gpt_loss=0.0521, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5605/5616 [1:11:22<00:08,  1.34it/s, gpt_loss=0.0521, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5605/5616 [1:11:23<00:08,  1.34it/s, gpt_loss=0.023, lr=5e-7] 
+
Train step of epoch 0: 100%|█████████▉| 5606/5616 [1:11:23<00:07,  1.37it/s, gpt_loss=0.023, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5606/5616 [1:11:23<00:07,  1.37it/s, gpt_loss=0.0175, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5607/5616 [1:11:23<00:06,  1.37it/s, gpt_loss=0.0175, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5607/5616 [1:11:24<00:06,  1.37it/s, gpt_loss=0.0215, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5608/5616 [1:11:24<00:06,  1.28it/s, gpt_loss=0.0215, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5608/5616 [1:11:25<00:06,  1.28it/s, gpt_loss=0.0426, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5609/5616 [1:11:25<00:05,  1.30it/s, gpt_loss=0.0426, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5609/5616 [1:11:26<00:05,  1.30it/s, gpt_loss=0.0135, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5610/5616 [1:11:26<00:04,  1.25it/s, gpt_loss=0.0135, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5610/5616 [1:11:27<00:04,  1.25it/s, gpt_loss=0.0418, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5611/5616 [1:11:27<00:03,  1.28it/s, gpt_loss=0.0418, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5611/5616 [1:11:27<00:03,  1.28it/s, gpt_loss=0.0382, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5612/5616 [1:11:27<00:03,  1.32it/s, gpt_loss=0.0382, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5612/5616 [1:11:28<00:03,  1.32it/s, gpt_loss=0.0152, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5613/5616 [1:11:28<00:02,  1.31it/s, gpt_loss=0.0152, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5613/5616 [1:11:29<00:02,  1.31it/s, gpt_loss=0.00546, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5614/5616 [1:11:29<00:01,  1.31it/s, gpt_loss=0.00546, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5614/5616 [1:11:30<00:01,  1.31it/s, gpt_loss=0.0465, lr=5e-7] 
+
Train step of epoch 0: 100%|█████████▉| 5615/5616 [1:11:30<00:00,  1.35it/s, gpt_loss=0.0465, lr=5e-7]
+
Train step of epoch 0: 100%|█████████▉| 5615/5616 [1:11:30<00:00,  1.35it/s, gpt_loss=0.0173, lr=5e-7]
+
Train step of epoch 0: 100%|██████████| 5616/5616 [1:11:30<00:00,  1.35it/s, gpt_loss=0.0173, lr=5e-7]
+
+
Eval stage of steps 1404:   0%|          | 0/168 [00:00<?, ?it/s]
+
+
Eval stage of steps 1404:   1%|          | 1/168 [00:00<00:40,  4.12it/s]
+
+
Eval stage of steps 1404:   1%|          | 1/168 [00:00<00:40,  4.12it/s, eval gpt_loss=0.00777]
+
+
Eval stage of steps 1404:   1%|          | 2/168 [00:00<00:37,  4.42it/s, eval gpt_loss=0.00777]
+
+
Eval stage of steps 1404:   1%|          | 2/168 [00:00<00:37,  4.42it/s, eval gpt_loss=0.00978]
+
+
Eval stage of steps 1404:   2%|▏         | 3/168 [00:00<00:39,  4.22it/s, eval gpt_loss=0.00978]
+
+
Eval stage of steps 1404:   2%|▏         | 3/168 [00:00<00:39,  4.22it/s, eval gpt_loss=0.0189] 
+
+
Eval stage of steps 1404:   2%|▏         | 4/168 [00:00<00:38,  4.22it/s, eval gpt_loss=0.0189]
+
+
Eval stage of steps 1404:   2%|▏         | 4/168 [00:00<00:38,  4.22it/s, eval gpt_loss=0.0145]
+
+
Eval stage of steps 1404:   3%|▎         | 5/168 [00:01<00:40,  4.05it/s, eval gpt_loss=0.0145]
+
+
Eval stage of steps 1404:   3%|▎         | 5/168 [00:01<00:40,  4.05it/s, eval gpt_loss=0.0143]
+
+
Eval stage of steps 1404:   4%|▎         | 6/168 [00:01<00:42,  3.83it/s, eval gpt_loss=0.0143]
+
+
Eval stage of steps 1404:   4%|▎         | 6/168 [00:01<00:42,  3.83it/s, eval gpt_loss=0.0153]
+
+
Eval stage of steps 1404:   4%|▍         | 7/168 [00:01<00:42,  3.77it/s, eval gpt_loss=0.0153]
+
+
Eval stage of steps 1404:   4%|▍         | 7/168 [00:01<00:42,  3.77it/s, eval gpt_loss=0.0254]
+
+
Eval stage of steps 1404:   5%|▍         | 8/168 [00:02<00:42,  3.81it/s, eval gpt_loss=0.0254]
+
+
Eval stage of steps 1404:   5%|▍         | 8/168 [00:02<00:42,  3.81it/s, eval gpt_loss=0.0226]
+
+
Eval stage of steps 1404:   5%|▌         | 9/168 [00:02<00:39,  4.02it/s, eval gpt_loss=0.0226]
+
+
Eval stage of steps 1404:   5%|▌         | 9/168 [00:02<00:39,  4.02it/s, eval gpt_loss=0.0205]
+
+
Eval stage of steps 1404:   6%|▌         | 10/168 [00:02<00:41,  3.81it/s, eval gpt_loss=0.0205]
+
+
Eval stage of steps 1404:   6%|▌         | 10/168 [00:02<00:41,  3.81it/s, eval gpt_loss=0.0201]
+
+
Eval stage of steps 1404:   7%|▋         | 11/168 [00:02<00:39,  3.93it/s, eval gpt_loss=0.0201]
+
+
Eval stage of steps 1404:   7%|▋         | 11/168 [00:02<00:39,  3.93it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:   7%|▋         | 12/168 [00:03<00:39,  3.94it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:   7%|▋         | 12/168 [00:03<00:39,  3.94it/s, eval gpt_loss=0.0187]
+
+
Eval stage of steps 1404:   8%|▊         | 13/168 [00:03<00:40,  3.85it/s, eval gpt_loss=0.0187]
+
+
Eval stage of steps 1404:   8%|▊         | 13/168 [00:03<00:40,  3.85it/s, eval gpt_loss=0.0226]
+
+
Eval stage of steps 1404:   8%|▊         | 14/168 [00:03<00:40,  3.82it/s, eval gpt_loss=0.0226]
+
+
Eval stage of steps 1404:   8%|▊         | 14/168 [00:03<00:40,  3.82it/s, eval gpt_loss=0.021] 
+
+
Eval stage of steps 1404:   9%|▉         | 15/168 [00:03<00:38,  3.98it/s, eval gpt_loss=0.021]
+
+
Eval stage of steps 1404:   9%|▉         | 15/168 [00:03<00:38,  3.98it/s, eval gpt_loss=0.0204]
+
+
Eval stage of steps 1404:  10%|▉         | 16/168 [00:04<00:39,  3.88it/s, eval gpt_loss=0.0204]
+
+
Eval stage of steps 1404:  10%|▉         | 16/168 [00:04<00:39,  3.88it/s, eval gpt_loss=0.0217]
+
+
Eval stage of steps 1404:  10%|█         | 17/168 [00:04<00:37,  3.99it/s, eval gpt_loss=0.0217]
+
+
Eval stage of steps 1404:  10%|█         | 17/168 [00:04<00:37,  3.99it/s, eval gpt_loss=0.0213]
+
+
Eval stage of steps 1404:  11%|█         | 18/168 [00:04<00:36,  4.08it/s, eval gpt_loss=0.0213]
+
+
Eval stage of steps 1404:  11%|█         | 18/168 [00:04<00:36,  4.08it/s, eval gpt_loss=0.0208]
+
+
Eval stage of steps 1404:  11%|█▏        | 19/168 [00:04<00:35,  4.24it/s, eval gpt_loss=0.0208]
+
+
Eval stage of steps 1404:  11%|█▏        | 19/168 [00:04<00:35,  4.24it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  12%|█▏        | 20/168 [00:04<00:34,  4.31it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  12%|█▏        | 20/168 [00:04<00:34,  4.31it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  12%|█▎        | 21/168 [00:05<00:34,  4.24it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  12%|█▎        | 21/168 [00:05<00:34,  4.24it/s, eval gpt_loss=0.021] 
+
+
Eval stage of steps 1404:  13%|█▎        | 22/168 [00:05<00:34,  4.19it/s, eval gpt_loss=0.021]
+
+
Eval stage of steps 1404:  13%|█▎        | 22/168 [00:05<00:34,  4.19it/s, eval gpt_loss=0.0223]
+
+
Eval stage of steps 1404:  14%|█▎        | 23/168 [00:05<00:38,  3.73it/s, eval gpt_loss=0.0223]
+
+
Eval stage of steps 1404:  14%|█▎        | 23/168 [00:05<00:38,  3.73it/s, eval gpt_loss=0.022] 
+
+
Eval stage of steps 1404:  14%|█▍        | 24/168 [00:06<00:38,  3.77it/s, eval gpt_loss=0.022]
+
+
Eval stage of steps 1404:  14%|█▍        | 24/168 [00:06<00:38,  3.77it/s, eval gpt_loss=0.0215]
+
+
Eval stage of steps 1404:  15%|█▍        | 25/168 [00:06<00:38,  3.72it/s, eval gpt_loss=0.0215]
+
+
Eval stage of steps 1404:  15%|█▍        | 25/168 [00:06<00:38,  3.72it/s, eval gpt_loss=0.0207]
+
+
Eval stage of steps 1404:  15%|█▌        | 26/168 [00:06<00:37,  3.83it/s, eval gpt_loss=0.0207]
+
+
Eval stage of steps 1404:  15%|█▌        | 26/168 [00:06<00:37,  3.83it/s, eval gpt_loss=0.0215]
+
+
Eval stage of steps 1404:  16%|█▌        | 27/168 [00:06<00:35,  4.00it/s, eval gpt_loss=0.0215]
+
+
Eval stage of steps 1404:  16%|█▌        | 27/168 [00:06<00:35,  4.00it/s, eval gpt_loss=0.0209]
+
+
Eval stage of steps 1404:  17%|█▋        | 28/168 [00:07<00:35,  3.91it/s, eval gpt_loss=0.0209]
+
+
Eval stage of steps 1404:  17%|█▋        | 28/168 [00:07<00:35,  3.91it/s, eval gpt_loss=0.0212]
+
+
Eval stage of steps 1404:  17%|█▋        | 29/168 [00:07<00:34,  3.98it/s, eval gpt_loss=0.0212]
+
+
Eval stage of steps 1404:  17%|█▋        | 29/168 [00:07<00:34,  3.98it/s, eval gpt_loss=0.0242]
+
+
Eval stage of steps 1404:  18%|█▊        | 30/168 [00:07<00:33,  4.09it/s, eval gpt_loss=0.0242]
+
+
Eval stage of steps 1404:  18%|█▊        | 30/168 [00:07<00:33,  4.09it/s, eval gpt_loss=0.0238]
+
+
Eval stage of steps 1404:  18%|█▊        | 31/168 [00:07<00:32,  4.16it/s, eval gpt_loss=0.0238]
+
+
Eval stage of steps 1404:  18%|█▊        | 31/168 [00:07<00:32,  4.16it/s, eval gpt_loss=0.0231]
+
+
Eval stage of steps 1404:  19%|█▉        | 32/168 [00:08<00:32,  4.13it/s, eval gpt_loss=0.0231]
+
+
Eval stage of steps 1404:  19%|█▉        | 32/168 [00:08<00:32,  4.13it/s, eval gpt_loss=0.0229]
+
+
Eval stage of steps 1404:  20%|█▉        | 33/168 [00:08<00:32,  4.13it/s, eval gpt_loss=0.0229]
+
+
Eval stage of steps 1404:  20%|█▉        | 33/168 [00:08<00:32,  4.13it/s, eval gpt_loss=0.0225]
+
+
Eval stage of steps 1404:  20%|██        | 34/168 [00:08<00:31,  4.29it/s, eval gpt_loss=0.0225]
+
+
Eval stage of steps 1404:  20%|██        | 34/168 [00:08<00:31,  4.29it/s, eval gpt_loss=0.0219]
+
+
Eval stage of steps 1404:  21%|██        | 35/168 [00:08<00:34,  3.85it/s, eval gpt_loss=0.0219]
+
+
Eval stage of steps 1404:  21%|██        | 35/168 [00:08<00:34,  3.85it/s, eval gpt_loss=0.0223]
+
+
Eval stage of steps 1404:  21%|██▏       | 36/168 [00:09<00:33,  3.89it/s, eval gpt_loss=0.0223]
+
+
Eval stage of steps 1404:  21%|██▏       | 36/168 [00:09<00:33,  3.89it/s, eval gpt_loss=0.0225]
+
+
Eval stage of steps 1404:  22%|██▏       | 37/168 [00:09<00:33,  3.92it/s, eval gpt_loss=0.0225]
+
+
Eval stage of steps 1404:  22%|██▏       | 37/168 [00:09<00:33,  3.92it/s, eval gpt_loss=0.0222]
+
+
Eval stage of steps 1404:  23%|██▎       | 38/168 [00:09<00:34,  3.73it/s, eval gpt_loss=0.0222]
+
+
Eval stage of steps 1404:  23%|██▎       | 38/168 [00:09<00:34,  3.73it/s, eval gpt_loss=0.0228]
+
+
Eval stage of steps 1404:  23%|██▎       | 39/168 [00:09<00:36,  3.58it/s, eval gpt_loss=0.0228]
+
+
Eval stage of steps 1404:  23%|██▎       | 39/168 [00:09<00:36,  3.58it/s, eval gpt_loss=0.0232]
+
+
Eval stage of steps 1404:  24%|██▍       | 40/168 [00:10<00:34,  3.75it/s, eval gpt_loss=0.0232]
+
+
Eval stage of steps 1404:  24%|██▍       | 40/168 [00:10<00:34,  3.75it/s, eval gpt_loss=0.0237]
+
+
Eval stage of steps 1404:  24%|██▍       | 41/168 [00:10<00:31,  3.98it/s, eval gpt_loss=0.0237]
+
+
Eval stage of steps 1404:  24%|██▍       | 41/168 [00:10<00:31,  3.98it/s, eval gpt_loss=0.0233]
+
+
Eval stage of steps 1404:  25%|██▌       | 42/168 [00:10<00:31,  3.96it/s, eval gpt_loss=0.0233]
+
+
Eval stage of steps 1404:  25%|██▌       | 42/168 [00:10<00:31,  3.96it/s, eval gpt_loss=0.0229]
+
+
Eval stage of steps 1404:  26%|██▌       | 43/168 [00:10<00:30,  4.09it/s, eval gpt_loss=0.0229]
+
+
Eval stage of steps 1404:  26%|██▌       | 43/168 [00:10<00:30,  4.09it/s, eval gpt_loss=0.0227]
+
+
Eval stage of steps 1404:  26%|██▌       | 44/168 [00:11<00:29,  4.20it/s, eval gpt_loss=0.0227]
+
+
Eval stage of steps 1404:  26%|██▌       | 44/168 [00:11<00:29,  4.20it/s, eval gpt_loss=0.0223]
+
+
Eval stage of steps 1404:  27%|██▋       | 45/168 [00:11<00:29,  4.22it/s, eval gpt_loss=0.0223]
+
+
Eval stage of steps 1404:  27%|██▋       | 45/168 [00:11<00:29,  4.22it/s, eval gpt_loss=0.0222]
+
+
Eval stage of steps 1404:  27%|██▋       | 46/168 [00:11<00:28,  4.26it/s, eval gpt_loss=0.0222]
+
+
Eval stage of steps 1404:  27%|██▋       | 46/168 [00:11<00:28,  4.26it/s, eval gpt_loss=0.0218]
+
+
Eval stage of steps 1404:  28%|██▊       | 47/168 [00:11<00:28,  4.20it/s, eval gpt_loss=0.0218]
+
+
Eval stage of steps 1404:  28%|██▊       | 47/168 [00:11<00:28,  4.20it/s, eval gpt_loss=0.0215]
+
+
Eval stage of steps 1404:  29%|██▊       | 48/168 [00:12<00:30,  3.92it/s, eval gpt_loss=0.0215]
+
+
Eval stage of steps 1404:  29%|██▊       | 48/168 [00:12<00:30,  3.92it/s, eval gpt_loss=0.0211]
+
+
Eval stage of steps 1404:  29%|██▉       | 49/168 [00:12<00:29,  3.97it/s, eval gpt_loss=0.0211]
+
+
Eval stage of steps 1404:  29%|██▉       | 49/168 [00:12<00:29,  3.97it/s, eval gpt_loss=0.0212]
+
+
Eval stage of steps 1404:  30%|██▉       | 50/168 [00:12<00:30,  3.86it/s, eval gpt_loss=0.0212]
+
+
Eval stage of steps 1404:  30%|██▉       | 50/168 [00:12<00:30,  3.86it/s, eval gpt_loss=0.0211]
+
+
Eval stage of steps 1404:  30%|███       | 51/168 [00:12<00:30,  3.84it/s, eval gpt_loss=0.0211]
+
+
Eval stage of steps 1404:  30%|███       | 51/168 [00:12<00:30,  3.84it/s, eval gpt_loss=0.021] 
+
+
Eval stage of steps 1404:  31%|███       | 52/168 [00:13<00:29,  3.94it/s, eval gpt_loss=0.021]
+
+
Eval stage of steps 1404:  31%|███       | 52/168 [00:13<00:29,  3.94it/s, eval gpt_loss=0.0206]
+
+
Eval stage of steps 1404:  32%|███▏      | 53/168 [00:13<00:28,  3.97it/s, eval gpt_loss=0.0206]
+
+
Eval stage of steps 1404:  32%|███▏      | 53/168 [00:13<00:28,  3.97it/s, eval gpt_loss=0.0205]
+
+
Eval stage of steps 1404:  32%|███▏      | 54/168 [00:13<00:28,  3.95it/s, eval gpt_loss=0.0205]
+
+
Eval stage of steps 1404:  32%|███▏      | 54/168 [00:13<00:28,  3.95it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  33%|███▎      | 55/168 [00:13<00:27,  4.07it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  33%|███▎      | 55/168 [00:13<00:27,  4.07it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  33%|███▎      | 56/168 [00:14<00:27,  4.02it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  33%|███▎      | 56/168 [00:14<00:27,  4.02it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  34%|███▍      | 57/168 [00:14<00:27,  4.04it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  34%|███▍      | 57/168 [00:14<00:27,  4.04it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  35%|███▍      | 58/168 [00:14<00:27,  3.95it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  35%|███▍      | 58/168 [00:14<00:27,  3.95it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  35%|███▌      | 59/168 [00:14<00:28,  3.85it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  35%|███▌      | 59/168 [00:14<00:28,  3.85it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  36%|███▌      | 60/168 [00:15<00:27,  3.96it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  36%|███▌      | 60/168 [00:15<00:27,  3.96it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  36%|███▋      | 61/168 [00:15<00:27,  3.88it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  36%|███▋      | 61/168 [00:15<00:27,  3.88it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  37%|███▋      | 62/168 [00:15<00:27,  3.84it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  37%|███▋      | 62/168 [00:15<00:27,  3.84it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  38%|███▊      | 63/168 [00:15<00:26,  3.95it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  38%|███▊      | 63/168 [00:15<00:26,  3.95it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  38%|███▊      | 64/168 [00:16<00:25,  4.05it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  38%|███▊      | 64/168 [00:16<00:25,  4.05it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  39%|███▊      | 65/168 [00:16<00:26,  3.85it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  39%|███▊      | 65/168 [00:16<00:26,  3.85it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  39%|███▉      | 66/168 [00:16<00:26,  3.92it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  39%|███▉      | 66/168 [00:16<00:26,  3.92it/s, eval gpt_loss=0.0192]
+
+
Eval stage of steps 1404:  40%|███▉      | 67/168 [00:16<00:24,  4.11it/s, eval gpt_loss=0.0192]
+
+
Eval stage of steps 1404:  40%|███▉      | 67/168 [00:16<00:24,  4.11it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  40%|████      | 68/168 [00:17<00:23,  4.21it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  40%|████      | 68/168 [00:17<00:23,  4.21it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  41%|████      | 69/168 [00:17<00:23,  4.30it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  41%|████      | 69/168 [00:17<00:23,  4.30it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  42%|████▏     | 70/168 [00:17<00:23,  4.26it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  42%|████▏     | 70/168 [00:17<00:23,  4.26it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  42%|████▏     | 71/168 [00:17<00:23,  4.21it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  42%|████▏     | 71/168 [00:17<00:23,  4.21it/s, eval gpt_loss=0.0191]
+
+
Eval stage of steps 1404:  43%|████▎     | 72/168 [00:17<00:22,  4.32it/s, eval gpt_loss=0.0191]
+
+
Eval stage of steps 1404:  43%|████▎     | 72/168 [00:17<00:22,  4.32it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  43%|████▎     | 73/168 [00:18<00:21,  4.35it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  43%|████▎     | 73/168 [00:18<00:21,  4.35it/s, eval gpt_loss=0.02]  
+
+
Eval stage of steps 1404:  44%|████▍     | 74/168 [00:18<00:21,  4.34it/s, eval gpt_loss=0.02]
+
+
Eval stage of steps 1404:  44%|████▍     | 74/168 [00:18<00:21,  4.34it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  45%|████▍     | 75/168 [00:18<00:21,  4.36it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  45%|████▍     | 75/168 [00:18<00:21,  4.36it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  45%|████▌     | 76/168 [00:18<00:22,  4.14it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  45%|████▌     | 76/168 [00:18<00:22,  4.14it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  46%|████▌     | 77/168 [00:19<00:22,  3.98it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  46%|████▌     | 77/168 [00:19<00:22,  3.98it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  46%|████▋     | 78/168 [00:19<00:21,  4.18it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  46%|████▋     | 78/168 [00:19<00:21,  4.18it/s, eval gpt_loss=0.0191]
+
+
Eval stage of steps 1404:  47%|████▋     | 79/168 [00:19<00:22,  3.95it/s, eval gpt_loss=0.0191]
+
+
Eval stage of steps 1404:  47%|████▋     | 79/168 [00:19<00:22,  3.95it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  48%|████▊     | 80/168 [00:19<00:21,  4.15it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  48%|████▊     | 80/168 [00:19<00:21,  4.15it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  48%|████▊     | 81/168 [00:20<00:21,  4.13it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  48%|████▊     | 81/168 [00:20<00:21,  4.13it/s, eval gpt_loss=0.0192]
+
+
Eval stage of steps 1404:  49%|████▉     | 82/168 [00:20<00:20,  4.17it/s, eval gpt_loss=0.0192]
+
+
Eval stage of steps 1404:  49%|████▉     | 82/168 [00:20<00:20,  4.17it/s, eval gpt_loss=0.0191]
+
+
Eval stage of steps 1404:  49%|████▉     | 83/168 [00:20<00:19,  4.29it/s, eval gpt_loss=0.0191]
+
+
Eval stage of steps 1404:  49%|████▉     | 83/168 [00:20<00:19,  4.29it/s, eval gpt_loss=0.019] 
+
+
Eval stage of steps 1404:  50%|█████     | 84/168 [00:20<00:20,  4.15it/s, eval gpt_loss=0.019]
+
+
Eval stage of steps 1404:  50%|█████     | 84/168 [00:20<00:20,  4.15it/s, eval gpt_loss=0.0192]
+
+
Eval stage of steps 1404:  51%|█████     | 85/168 [00:21<00:19,  4.24it/s, eval gpt_loss=0.0192]
+
+
Eval stage of steps 1404:  51%|█████     | 85/168 [00:21<00:19,  4.24it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  51%|█████     | 86/168 [00:21<00:18,  4.34it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  51%|█████     | 86/168 [00:21<00:18,  4.34it/s, eval gpt_loss=0.0191]
+
+
Eval stage of steps 1404:  52%|█████▏    | 87/168 [00:21<00:18,  4.30it/s, eval gpt_loss=0.0191]
+
+
Eval stage of steps 1404:  52%|█████▏    | 87/168 [00:21<00:18,  4.30it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  52%|█████▏    | 88/168 [00:21<00:20,  3.85it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  52%|█████▏    | 88/168 [00:21<00:20,  3.85it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  53%|█████▎    | 89/168 [00:22<00:20,  3.91it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  53%|█████▎    | 89/168 [00:22<00:20,  3.91it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  54%|█████▎    | 90/168 [00:22<00:19,  4.00it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  54%|█████▎    | 90/168 [00:22<00:19,  4.00it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  54%|█████▍    | 91/168 [00:22<00:18,  4.07it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  54%|█████▍    | 91/168 [00:22<00:18,  4.07it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  55%|█████▍    | 92/168 [00:22<00:19,  3.93it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  55%|█████▍    | 92/168 [00:22<00:19,  3.93it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  55%|█████▌    | 93/168 [00:23<00:20,  3.73it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  55%|█████▌    | 93/168 [00:23<00:20,  3.73it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  56%|█████▌    | 94/168 [00:23<00:19,  3.83it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  56%|█████▌    | 94/168 [00:23<00:19,  3.83it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  57%|█████▋    | 95/168 [00:23<00:18,  3.96it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  57%|█████▋    | 95/168 [00:23<00:18,  3.96it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  57%|█████▋    | 96/168 [00:23<00:18,  3.89it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  57%|█████▋    | 96/168 [00:23<00:18,  3.89it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  58%|█████▊    | 97/168 [00:24<00:17,  4.05it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  58%|█████▊    | 97/168 [00:24<00:17,  4.05it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  58%|█████▊    | 98/168 [00:24<00:16,  4.13it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  58%|█████▊    | 98/168 [00:24<00:16,  4.13it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  59%|█████▉    | 99/168 [00:24<00:16,  4.16it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  59%|█████▉    | 99/168 [00:24<00:16,  4.16it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  60%|█████▉    | 100/168 [00:24<00:16,  4.17it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  60%|█████▉    | 100/168 [00:24<00:16,  4.17it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  60%|██████    | 101/168 [00:25<00:15,  4.25it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  60%|██████    | 101/168 [00:25<00:15,  4.25it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  61%|██████    | 102/168 [00:25<00:15,  4.15it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  61%|██████    | 102/168 [00:25<00:15,  4.15it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  61%|██████▏   | 103/168 [00:25<00:15,  4.11it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  61%|██████▏   | 103/168 [00:25<00:15,  4.11it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  62%|██████▏   | 104/168 [00:25<00:15,  4.05it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  62%|██████▏   | 104/168 [00:25<00:15,  4.05it/s, eval gpt_loss=0.0192]
+
+
Eval stage of steps 1404:  62%|██████▎   | 105/168 [00:26<00:16,  3.76it/s, eval gpt_loss=0.0192]
+
+
Eval stage of steps 1404:  62%|██████▎   | 105/168 [00:26<00:16,  3.76it/s, eval gpt_loss=0.0191]
+
+
Eval stage of steps 1404:  63%|██████▎   | 106/168 [00:26<00:16,  3.85it/s, eval gpt_loss=0.0191]
+
+
Eval stage of steps 1404:  63%|██████▎   | 106/168 [00:26<00:16,  3.85it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  64%|██████▎   | 107/168 [00:26<00:16,  3.78it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  64%|██████▎   | 107/168 [00:26<00:16,  3.78it/s, eval gpt_loss=0.02]  
+
+
Eval stage of steps 1404:  64%|██████▍   | 108/168 [00:26<00:15,  3.91it/s, eval gpt_loss=0.02]
+
+
Eval stage of steps 1404:  64%|██████▍   | 108/168 [00:26<00:15,  3.91it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  65%|██████▍   | 109/168 [00:27<00:15,  3.92it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  65%|██████▍   | 109/168 [00:27<00:15,  3.92it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  65%|██████▌   | 110/168 [00:27<00:15,  3.81it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  65%|██████▌   | 110/168 [00:27<00:15,  3.81it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  66%|██████▌   | 111/168 [00:27<00:14,  3.87it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  66%|██████▌   | 111/168 [00:27<00:14,  3.87it/s, eval gpt_loss=0.0201]
+
+
Eval stage of steps 1404:  67%|██████▋   | 112/168 [00:27<00:14,  3.95it/s, eval gpt_loss=0.0201]
+
+
Eval stage of steps 1404:  67%|██████▋   | 112/168 [00:27<00:14,  3.95it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  67%|██████▋   | 113/168 [00:28<00:13,  4.01it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  67%|██████▋   | 113/168 [00:28<00:13,  4.01it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  68%|██████▊   | 114/168 [00:28<00:13,  4.02it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  68%|██████▊   | 114/168 [00:28<00:13,  4.02it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  68%|██████▊   | 115/168 [00:28<00:12,  4.13it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  68%|██████▊   | 115/168 [00:28<00:12,  4.13it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  69%|██████▉   | 116/168 [00:28<00:12,  4.19it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  69%|██████▉   | 116/168 [00:28<00:12,  4.19it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  70%|██████▉   | 117/168 [00:29<00:12,  4.09it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  70%|██████▉   | 117/168 [00:29<00:12,  4.09it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  70%|███████   | 118/168 [00:29<00:12,  3.99it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  70%|███████   | 118/168 [00:29<00:12,  3.99it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  71%|███████   | 119/168 [00:29<00:12,  3.90it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  71%|███████   | 119/168 [00:29<00:12,  3.90it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  71%|███████▏  | 120/168 [00:29<00:12,  3.87it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  71%|███████▏  | 120/168 [00:29<00:12,  3.87it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  72%|███████▏  | 121/168 [00:30<00:12,  3.83it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  72%|███████▏  | 121/168 [00:30<00:12,  3.83it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  73%|███████▎  | 122/168 [00:30<00:11,  3.91it/s, eval gpt_loss=0.0193]
+
+
Eval stage of steps 1404:  73%|███████▎  | 122/168 [00:30<00:11,  3.91it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  73%|███████▎  | 123/168 [00:30<00:11,  3.84it/s, eval gpt_loss=0.0194]
+
+
Eval stage of steps 1404:  73%|███████▎  | 123/168 [00:30<00:11,  3.84it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  74%|███████▍  | 124/168 [00:30<00:11,  3.89it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  74%|███████▍  | 124/168 [00:30<00:11,  3.89it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  74%|███████▍  | 125/168 [00:31<00:11,  3.88it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  74%|████��██▍  | 125/168 [00:31<00:11,  3.88it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  75%|███████▌  | 126/168 [00:31<00:10,  4.00it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  75%|███████▌  | 126/168 [00:31<00:10,  4.00it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  76%|███████▌  | 127/168 [00:31<00:10,  3.98it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  76%|███████▌  | 127/168 [00:31<00:10,  3.98it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  76%|███████▌  | 128/168 [00:31<00:10,  3.85it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  76%|███████▌  | 128/168 [00:32<00:10,  3.85it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  77%|███████▋  | 129/168 [00:32<00:09,  3.96it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  77%|███████▋  | 129/168 [00:32<00:09,  3.96it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  77%|███████▋  | 130/168 [00:32<00:09,  3.99it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  77%|███████▋  | 130/168 [00:32<00:09,  3.99it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  78%|███████▊  | 131/168 [00:32<00:09,  4.11it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  78%|███████▊  | 131/168 [00:32<00:09,  4.11it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  79%|███████▊  | 132/168 [00:33<00:09,  3.79it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  79%|███████▊  | 132/168 [00:33<00:09,  3.79it/s, eval gpt_loss=0.0205]
+
+
Eval stage of steps 1404:  79%|███████▉  | 133/168 [00:33<00:08,  3.95it/s, eval gpt_loss=0.0205]
+
+
Eval stage of steps 1404:  79%|███████▉  | 133/168 [00:33<00:08,  3.95it/s, eval gpt_loss=0.0203]
+
+
Eval stage of steps 1404:  80%|███████▉  | 134/168 [00:33<00:08,  3.90it/s, eval gpt_loss=0.0203]
+
+
Eval stage of steps 1404:  80%|███████▉  | 134/168 [00:33<00:08,  3.90it/s, eval gpt_loss=0.0203]
+
+
Eval stage of steps 1404:  80%|████████  | 135/168 [00:33<00:08,  3.93it/s, eval gpt_loss=0.0203]
+
+
Eval stage of steps 1404:  80%|████████  | 135/168 [00:33<00:08,  3.93it/s, eval gpt_loss=0.0204]
+
+
Eval stage of steps 1404:  81%|████████  | 136/168 [00:34<00:08,  3.94it/s, eval gpt_loss=0.0204]
+
+
Eval stage of steps 1404:  81%|████████  | 136/168 [00:34<00:08,  3.94it/s, eval gpt_loss=0.0204]
+
+
Eval stage of steps 1404:  82%|████████▏ | 137/168 [00:34<00:08,  3.84it/s, eval gpt_loss=0.0204]
+
+
Eval stage of steps 1404:  82%|████████▏ | 137/168 [00:34<00:08,  3.84it/s, eval gpt_loss=0.0203]
+
+
Eval stage of steps 1404:  82%|████████▏ | 138/168 [00:34<00:07,  3.89it/s, eval gpt_loss=0.0203]
+
+
Eval stage of steps 1404:  82%|████████▏ | 138/168 [00:34<00:07,  3.89it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  83%|████████▎ | 139/168 [00:34<00:07,  4.02it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  83%|████████▎ | 139/168 [00:34<00:07,  4.02it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  83%|████████▎ | 140/168 [00:35<00:06,  4.03it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  83%|████████▎ | 140/168 [00:35<00:06,  4.03it/s, eval gpt_loss=0.0204]
+
+
Eval stage of steps 1404:  84%|████████▍ | 141/168 [00:35<00:06,  4.09it/s, eval gpt_loss=0.0204]
+
+
Eval stage of steps 1404:  84%|████████▍ | 141/168 [00:35<00:06,  4.09it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  85%|████████▍ | 142/168 [00:35<00:06,  4.20it/s, eval gpt_loss=0.0202]
+
+
Eval stage of steps 1404:  85%|████████▍ | 142/168 [00:35<00:06,  4.20it/s, eval gpt_loss=0.0201]
+
+
Eval stage of steps 1404:  85%|████████▌ | 143/168 [00:35<00:06,  4.06it/s, eval gpt_loss=0.0201]
+
+
Eval stage of steps 1404:  85%|████████▌ | 143/168 [00:35<00:06,  4.06it/s, eval gpt_loss=0.0201]
+
+
Eval stage of steps 1404:  86%|████████▌ | 144/168 [00:36<00:06,  3.74it/s, eval gpt_loss=0.0201]
+
+
Eval stage of steps 1404:  86%|████████▌ | 144/168 [00:36<00:06,  3.74it/s, eval gpt_loss=0.02]  
+
+
Eval stage of steps 1404:  86%|████████▋ | 145/168 [00:36<00:05,  3.90it/s, eval gpt_loss=0.02]
+
+
Eval stage of steps 1404:  86%|████████▋ | 145/168 [00:36<00:05,  3.90it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  87%|█████���██▋ | 146/168 [00:36<00:05,  3.93it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  87%|████████▋ | 146/168 [00:36<00:05,  3.93it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  88%|████████▊ | 147/168 [00:36<00:05,  3.85it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  88%|████████▊ | 147/168 [00:36<00:05,  3.85it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  88%|████████▊ | 148/168 [00:37<00:05,  3.95it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  88%|████████▊ | 148/168 [00:37<00:05,  3.95it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  89%|████████▊ | 149/168 [00:37<00:04,  3.91it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  89%|████████▊ | 149/168 [00:37<00:04,  3.91it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  89%|████████▉ | 150/168 [00:37<00:04,  3.99it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  89%|████████▉ | 150/168 [00:37<00:04,  3.99it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  90%|████████▉ | 151/168 [00:37<00:04,  3.99it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  90%|████████▉ | 151/168 [00:37<00:04,  3.99it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  90%|█████████ | 152/168 [00:38<00:03,  4.03it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  90%|█████████ | 152/168 [00:38<00:03,  4.03it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  91%|█████████ | 153/168 [00:38<00:03,  3.78it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  91%|█████████ | 153/168 [00:38<00:03,  3.78it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  92%|█████████▏| 154/168 [00:38<00:03,  3.98it/s, eval gpt_loss=0.0199]
+
+
Eval stage of steps 1404:  92%|█████████▏| 154/168 [00:38<00:03,  3.98it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  92%|█████████▏| 155/168 [00:38<00:03,  3.92it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  92%|█████████▏| 155/168 [00:38<00:03,  3.92it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  93%|█████████▎| 156/168 [00:39<00:03,  3.89it/s, eval gpt_loss=0.0198]
+
+
Eval stage of steps 1404:  93%|█████████▎| 156/168 [00:39<00:03,  3.89it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  93%|█████████▎| 157/168 [00:39<00:02,  3.95it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  93%|█████████▎| 157/168 [00:39<00:02,  3.95it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  94%|█████████▍| 158/168 [00:39<00:02,  3.95it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  94%|█████████▍| 158/168 [00:39<00:02,  3.95it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  95%|█████████▍| 159/168 [00:39<00:02,  3.78it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  95%|█████████▍| 159/168 [00:39<00:02,  3.78it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  95%|█████████▌| 160/168 [00:40<00:02,  3.88it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  95%|█████████▌| 160/168 [00:40<00:02,  3.88it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  96%|█████████▌| 161/168 [00:40<00:01,  3.96it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  96%|█████████▌| 161/168 [00:40<00:01,  3.96it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  96%|█████████▋| 162/168 [00:40<00:01,  4.10it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  96%|█████████▋| 162/168 [00:40<00:01,  4.10it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  97%|█████████▋| 163/168 [00:40<00:01,  3.98it/s, eval gpt_loss=0.0195]
+
+
Eval stage of steps 1404:  97%|█████████▋| 163/168 [00:40<00:01,  3.98it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  98%|█████████▊| 164/168 [00:41<00:00,  4.02it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  98%|█████████▊| 164/168 [00:41<00:00,  4.02it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  98%|█████████▊| 165/168 [00:41<00:00,  3.98it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404:  98%|█████████▊| 165/168 [00:41<00:00,  3.98it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  99%|█████████▉| 166/168 [00:41<00:00,  4.03it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  99%|█████████▉| 166/168 [00:41<00:00,  4.03it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  99%|█████████▉| 167/168 [00:41<00:00,  3.90it/s, eval gpt_loss=0.0197]
+
+
Eval stage of steps 1404:  99%|█████████▉| 167/168 [00:41<00:00,  3.90it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404: 100%|██████████| 168/168 [00:42<00:00,  3.88it/s, eval gpt_loss=0.0196]
+
+
Eval stage of steps 1404: 100%|██████████| 168/168 [00:42<00:00,  3.88it/s, eval gpt_loss=0.0196]
Eval stage of steps 1404: 100%|██████████| 168/168 [00:42<00:00,  3.99it/s, eval gpt_loss=0.0196]
+
Train epoch: 100%|██████████| 1/1 [1:12:13<00:00, 4333.05s/it]wandb:                                                                                
+wandb: 
+wandb: Run history:
+wandb: eval/eval gpt_loss ▁
+wandb:   eval/global_step ▁
+wandb:  train/global_step ▁▁▁▂▂▂▂▂▃▃▄▄▄▄▄▄▄▅▅▅▅▆▆▆▆▆▆▆▆▇▇▇▇▇▇█████
+wandb:     train/gpt_loss ▆▄▅▂▁▂█▂▃▂▁▃▅▃▂▁▂▆▂▂▂▁▄▁▁▂▁▁▂▂▃▂▃▂▁▃▁▁▁▁
+wandb:    train/loss_mean ▄▁▂▃▁█▁▂▁▃▃▁▂▄▂▁▃▃▅▂▁▃▂▁▁▂▄▁▁▁▁▂▁▁▁▃▁▁▂▂
+wandb:           train/lr ██████▇▇▇▆▆▆▅▅▅▅▅▄▄▄▄▄▃▃▃▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁
+wandb: 
+wandb: Run summary:
+wandb: eval/eval gpt_loss 0.01957
+wandb:   eval/global_step 1404
+wandb:  train/global_step 1404
+wandb:     train/gpt_loss 0.01729
+wandb:    train/loss_mean 0.00426
+wandb:           train/lr 0.0
+wandb: 
+wandb: 🚀 View run RLVR_sft_augmented_data_180k_20250630-0636 at: https://wandb.ai/sarosavo/magic_spell/runs/afb04fb4-d0ce-4a7e-9a3c-547fe30cc0d8
+wandb: ⭐️ View project at: https://wandb.ai/sarosavo/magic_spell
+wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
+wandb: Find logs at: ./wandb/run-20250630_063805-afb04fb4-d0ce-4a7e-9a3c-547fe30cc0d8/logs
+/root/miniconda3/envs/openrlhf/lib/python3.10/site-packages/wandb/sdk/wandb_run.py:2310: UserWarning: Run (afb04fb4-d0ce-4a7e-9a3c-547fe30cc0d8) is finished. The call to `_console_raw_callback` will be ignored. Please make sure that you are using an active run.
+  lambda data: self._console_raw_callback("stderr", data),
+
Train epoch: 100%|██████████| 1/1 [1:12:16<00:00, 4336.00s/it]
+
Train step of epoch 0: 100%|██████████| 5616/5616 [1:12:16<00:00,  1.30it/s, gpt_loss=0.0173, lr=5e-7]
+[2025-06-30 07:50:36,039] [INFO] [launch.py:351:main] Process 856603 exits successfully.
+[2025-06-30 07:50:36,040] [INFO] [launch.py:351:main] Process 856601 exits successfully.
+[2025-06-30 07:50:36,041] [INFO] [launch.py:351:main] Process 856598 exits successfully.
+[2025-06-30 07:50:36,041] [INFO] [launch.py:351:main] Process 856599 exits successfully.
+[2025-06-30 07:50:37,043] [INFO] [launch.py:351:main] Process 856602 exits successfully.
+[2025-06-30 07:50:37,043] [INFO] [launch.py:351:main] Process 856600 exits successfully.
+[2025-06-30 07:50:37,044] [INFO] [launch.py:351:main] Process 856604 exits successfully.
+[2025-06-30 07:50:51,055] [INFO] [launch.py:351:main] Process 856597 exits successfully.