cgm-ml
cgm-ml copied to clipboard
Fix callback in multiartifact CNN fine-tuning
Problem: We see that when fine tuning starts, the loss jumps up.
Details:
- Nadam has a LR per layer
- when we fine-tune (we make base network's layers trainable), then these LR might be too big
Solution (besides considering a smaller LR)
Task: Improve code:
- Add callback for unfreezing (e.g. after 1000 layers) instead of calling model.fit() again.
- This will have the effect that the Adam statistics (momentum etc) can be continued to use