Arnaud Fickinger
Arnaud Fickinger
hi, same problem, have you solved it?
Did you solve it? I have the same error.
@botbw thank you the [low-level](https://github.com/hpcaitech/ColossalAI/blob/74f4a297342f17e6d9447633cc26ce45db33b59d/tests/test_zero/test_low_level/test_zero1_2.py#L164) snippet is working! By the way which of gemini or low-level should I use for best performance with 1 to 8 A100 GPUs and 500M...
@botbw when I define 2 param_groups the id() of the parameters of the second group do not match any keys of optimizer._grad_store._grads_of_params[1]
@botbw I have written a min repro with a simple network and in this case the keys actually match! I will take a closer look at my code and get...