Question about training step

[estuday]

Hi Apoorv Great work with KGT5 model.

I follow your code to training the T5 from scratch. However , it does not work after staring fresh. Have you ever encountered such a problem or could you give me some suggestions？ Starting fresh 0%| | 0/166748 [00:00<?, ?batches/s]

This is my code to start training CUDA_VISIBLE_DEVICES=4,5,6,7 torchrun \ --nproc_per_node 4 \ main_accelerate.py \ --save_prefix wd5m \ --model_size small --dataset wikidata5m \ --batch_size 64 --save_steps 5000 \ --loss_steps 500