Arnaud Fickinger

Results 5 comments of Arnaud Fickinger

hi, same problem, have you solved it?

@botbw thank you the [low-level](https://github.com/hpcaitech/ColossalAI/blob/74f4a297342f17e6d9447633cc26ce45db33b59d/tests/test_zero/test_low_level/test_zero1_2.py#L164) snippet is working! By the way which of gemini or low-level should I use for best performance with 1 to 8 A100 GPUs and 500M...

@botbw when I define 2 param_groups the id() of the parameters of the second group do not match any keys of optimizer._grad_store._grads_of_params[1]

@botbw I have written a min repro with a simple network and in this case the keys actually match! I will take a closer look at my code and get...