nakroy

[BUG]Get an AtrributeError when trying to convert llama3-8B model from HF format to mcore format

3

Describe the bug Get an AtrributeError when trying to convert llama3-8B model from HF format to mcore format, the error is below: `AttributeError: 'Tokenizer' object has no attribute 'vocab_size'` To...

[BUG]Get an AtrributeError when trying to finetune llama3-8B model with multi nodes

5

**Describe the bug** I try to finetune `llama3-8B` model with multi nodes but get an AtrributeError when finishing loading mcore format checkpoint and starting to build datasets, the error is...

Adding DDP implementation that can be trained with multi GPUs, some arguments and usages are modified and changed to fit in with this implementation.