Mayank Mishra

From what I see, this has a lot of intricacies: ramp_up_batch_size, curriculum_learning (changine sequence length) so my suggestion is to compute as time_per_iter * (total_tokens - consumed_tokens) /86400 However, I...

This is not possible. To download DS checkpoints refer to this issue: https://github.com/bigscience-workshop/Megatron-DeepSpeed/issues/319

I don't understand the issue. Do you just need to run inference? If that is the case, DS-inference is compatible with all Huggingface models.

@AnShengqiang Its non-trivial to convert models for training. People are actively exploring this as far as I know. This repository saves something called a universal checkpoint which can be converted...

Hmm, its not working for me even within a single node with batch size = 1, 8x A100 80gb Same, CUDA illegal memory access error

@pohunghuang-nctu nothing like that in my logs This is the full trace ``` [2022-07-26 11:41:08,472] [WARNING] [runner.py:159:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only. [2022-07-26...

I am not sure why I am getting the same error ^^ for batch size = 1. @pai4451 Any pointers?

> > I am not sure why I am getting the same error ^^ for batch size = 1. @pai4451 > > Any pointers? > > What is your CUDA...

> @mayank31398 Perhaps try the `ds-inference/bloom-fix` branch of deepspeed? Ill try this today. thanks

@KMFODA @thomasw21 , https://github.com/bigscience-workshop/Megatron-DeepSpeed/pull/328 Already added the ability to benchmark system, interactive cli, and a generation server. Testing a few things. Will try to get this merged by this week