Sergey Edunov

icon company Facebook AI Research icon location San Francisco icon location Director of Engineering in GenAI @ Meta. I work on LLMs

Results 2 comments of Sergey Edunov

How to train a simple, vanilla transformers translation model from scratch with Fairseq

It will be easier to help you if you provide output logs for both training and preprocessing, there are many things that could have gone wrong. Your setup seems correct...

How to train a simple, vanilla transformers translation model from scratch with Fairseq

"Delayed updates can also improve training speed by reducing inter-GPU communication costs and by saving idle time caused by variance in workload across GPUs." - yes, but there are different...