tisgotos

Code Release

3

Hi, thanks for this interesting work, Is it okay to ask when the code was released?

pytorch version

2

Hello! I have obtained a ViT model from timm, and I want to train it using your OSDP method. However, OSDP requires torch version 1.10.2, while timm needs a higher...

前向传播返回值缺少bal_loss

2

在应用完补丁执行pretrain_gpt.py遇到的问题 Traceback (most recent call last): File "pretrain_gpt.py", line 126, in pretrain(train_valid_test_datasets_provider, model_provider, forward_step, File "/workspace/Megatron-LM/megatron/training.py", line 157, in pretrain iteration = train(forward_step_func, File "/workspace/Megatron-LM/megatron/training.py", line 630, in train train_step(forward_step_func,...