Olatunji Ruwase

@lucasleesw, thanks for the PR. Can you please use [this](https://github.com/microsoft/DeepSpeed/blob/master/CONTRIBUTING.md#prerequisites) to resolve the formatting issues

@buttercutter, you are missing a [deepspeed config file](https://www.deepspeed.ai/getting-started/#launching-deepspeed-training) on the command passed by `--deepspeed_config`. Alternatively, you can pass a dict as [config_params](https://deepspeed.readthedocs.io/en/latest/initialize.html#training-initialization) to `deepspeed.initialize()`

DeepSpeed configuration is meant to be network-agnostic, so in reality that configuration file would work except for `auto` fields which are defined for the HF frontend. The configuration file is...

Set "train_micro_batch_size_per_gpu" to 8 in the configuration file.

It should be, but please report any issues.

Sorry, it appears #1149 was never merged. Unfortunately, it has a conflict with master. Can you please try picking that up?

@buttercutter, #1149 is now merged. Please try master.

@pacman100, we made some recent changes to ZeRO 3, so I wonder if those are source of regression. Could you please try v0.6.0?

@pacman100, thanks for the update. So, you see a totally different error with old release :(. Let's ignore the old release then. I will work on setting up a repro...

@pacman100, thanks for sharing your update. I am glad that performance problem is resolved in the latest code. I have created this #1974 to suppress the warning noise. The PR...