DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Modify unit test to cover more cases

Open RezaYazdaniAminabadi opened this issue 2 years ago • 1 comments

This PR adds two set of modification to the unit tests for better test coverage on optimizer functionality:

  1. Increase the parameters groups to catch errors better at optimizer step function.
  2. Reduce the loss scale by using "initial_scale_power" to reduce the likelihood of overflowing in the backward and the optimizer step will be called into eventually.

cc: @tjruwase, @jeffra

RezaYazdaniAminabadi avatar Feb 15 '22 19:02 RezaYazdaniAminabadi

Can one of the admins verify this patch?

rocm-mici avatar Jun 09 '22 20:06 rocm-mici