elie

icon indicating a website link https://huggingface.co/eliebak

icon company @huggingface icon location Paris, France icon location ML Research Engineer @huggingface

Results 3 comments of elie

NCCL only multi-gpu multi-node training without MPI

hey @chinthysl do you have the ckpt somewhere? :)

TypeError: DatatroveFolderDataset.__init__() got an unexpected keyword argument 'eos_token_id'

We are using this branch of datatrove: https://github.com/huggingface/datatrove/tree/nouamane/avoid-s3 (cc @NouamaneTazi) and make sure the config is: https://github.com/huggingface/smollm/blob/main/text/pretraining/smollm3/stage1_8T.yaml Updated the info on the smollm repo to add datatrove branch thanks https://github.com/huggingface/smollm/pull/97

TypeError: DatatroveFolderDataset.__init__() got an unexpected keyword argument 'eos_token_id'

Nanotron readme is not up to date, you need to use this branch of datatrove with the current dataloader https://github.com/huggingface/datatrove/tree/nouamane/avoid-s3 I recommend following this readme https://github.com/huggingface/smollm/tree/main/text/pretraining if you want to...