darts icon indicating copy to clipboard operation
darts copied to clipboard

Metrics for categorical forecasting

Open jonasblanc opened this issue 8 months ago • 1 comments

Checklist before merging this PR:

  • [ ] Mentioned all issues that this PR fixes or addresses.
  • [ ] Summarized the updates of this PR under Summary.
  • [ ] Added an entry under Unreleased in the Changelog.

Fixes #.

Summary

Implement metrics for categorical forecasting.

TODO:

  • extend list of supported categorical metrics
  • categorical specific tests
  • adapt metrics decorator to support per class score and cross-class averaging

Other Information

jonasblanc avatar Apr 10 '25 09:04 jonasblanc

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.10%. Comparing base (2391d8b) to head (61d9a1c). Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2767      +/-   ##
==========================================
- Coverage   95.11%   95.10%   -0.01%     
==========================================
  Files         145      146       +1     
  Lines       15157    15342     +185     
==========================================
+ Hits        14416    14591     +175     
- Misses        741      751      +10     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Apr 11 '25 10:04 codecov[bot]

Thanks a lot for this @jonasblanc 🚀
I refactored a bit to:

  • integrate the value getters (for class labels) into the main _get_values* helper functions
  • refactored the decorators a bit
  • added more tests (for sampled and predicted probabilities)

Last remaining point is to add new metrics. Should be rather straightforward though :) 🚀

dennisbader avatar Jul 13 '25 12:07 dennisbader