cgm-ml icon indicating copy to clipboard operation
cgm-ml copied to clipboard

Fix callback in multiartifact CNN fine-tuning

Open markus-hinsche opened this issue 3 years ago • 0 comments

Problem: We see that when fine tuning starts, the loss jumps up.

Details:

  • Nadam has a LR per layer
  • when we fine-tune (we make base network's layers trainable), then these LR might be too big

Solution (besides considering a smaller LR)

Task: Improve code:

  • Add callback for unfreezing (e.g. after 1000 layers) instead of calling model.fit() again.
  • This will have the effect that the Adam statistics (momentum etc) can be continued to use

markus-hinsche avatar Apr 26 '21 13:04 markus-hinsche