mammoth icon indicating copy to clipboard operation
mammoth copied to clipboard

task-based LR scheduling, finer-grained val reporting

Open TimotheeMickus opened this issue 2 years ago • 0 comments

  • [ ] break down accuracy and validation loss per task when reporting stats
  • [ ] introduce validation-based LR scheduling (reduce LR on plateau) that's properly adjusted depending on the performances of individual modules:
    • for each module, initialize module-specific LRs to the default value
    • at validation:
      • compute perf per task
      • aggregate (mean or task-weighted mean) perf per module across task
      • reduce module-specific LR based on the aggregate perf

TimotheeMickus avatar Oct 16 '23 12:10 TimotheeMickus