nntrainer icon indicating copy to clipboard operation
nntrainer copied to clipboard

[Wait for #2591] [ TEST ] Add more unittest and fixes for mixed precision

Open jijoongmoon opened this issue 9 months ago • 7 comments

In this PR

This PR includes more unittest and fixes for mixed precision. . Model Unittest . 2 fc layer which generates NaN or Inf Gradient from Troch. . MSE Loss and Check the whole procedure of the mixed precision training. . Even if the FC model only has one weight, it is good enough to validate the mixed precision. . Torch model also works similar way to NNTrainer. . Some fixes about the execution order of applying gradient when the mixed precision is on. . Update SGD to support Mixed Precision training

Changes proposed in this PR:

  • Added TOC generator for README.md

Resolves:

Self evaluation:

  1. Build test: [X]Passed [ ]Failed [ ]Skipped
  2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon [email protected]

jijoongmoon avatar May 24 '24 01:05 jijoongmoon