scoringutils
scoringutils copied to clipboard
Should `score()`/`apply_metrics()` return warning about failed metrics in a batch?
Should we also print the warnings in a batch instead of eagerly at the time of occurrence? They could easily get lost in the other warnings as the functions are getting executed.
I imagine a setup where at the time of printing the successful scores dt, we also print "In addition, the following metrics failed with the following warning". This is nicely discussed in this blog post about deferring errors. Worth considering.
Originally posted by @jamesmbaazam in https://github.com/epiforecasts/scoringutils/pull/803#discussion_r1592175575