mammoth
mammoth copied to clipboard
task-based LR scheduling, finer-grained val reporting
- [ ] break down accuracy and validation loss per task when reporting stats
- [ ] introduce validation-based LR scheduling (reduce LR on plateau) that's properly adjusted depending on the performances of individual modules:
- for each module, initialize module-specific LRs to the default value
- at validation:
- compute perf per task
- aggregate (mean or task-weighted mean) perf per module across task
- reduce module-specific LR based on the aggregate perf