gluonts
gluonts copied to clipboard
Weighted average for custom evaluation function
Description
gluonts.evaluation.Evaluator
provides a parameter, custom_eval_fn
which allows defining custom evaluation functions. However, the global (across multiple TS) and local (within TS) aggregations are limited to mean & sum, and mean & median, respectively. My custom metric involves a weighted average-based aggregation, which the current implementation does not provide. It would be useful to have a way to define more complex custom evaluation functions.
Seems to be relevant to https://github.com/awslabs/gluon-ts/issues/2048
@eugeneteoh exactly, thanks for the issue: flexibility of the evaluator is something we're currently working on, I'll keep you posted when there are updates