gluonts icon indicating copy to clipboard operation
gluonts copied to clipboard

Weighted average for custom evaluation function

Open eugeneteoh opened this issue 2 years ago • 2 comments

Description

gluonts.evaluation.Evaluator provides a parameter, custom_eval_fn which allows defining custom evaluation functions. However, the global (across multiple TS) and local (within TS) aggregations are limited to mean & sum, and mean & median, respectively. My custom metric involves a weighted average-based aggregation, which the current implementation does not provide. It would be useful to have a way to define more complex custom evaluation functions.

eugeneteoh avatar Aug 15 '22 08:08 eugeneteoh

Seems to be relevant to https://github.com/awslabs/gluon-ts/issues/2048

eugeneteoh avatar Aug 16 '22 10:08 eugeneteoh

@eugeneteoh exactly, thanks for the issue: flexibility of the evaluator is something we're currently working on, I'll keep you posted when there are updates

lostella avatar Aug 16 '22 14:08 lostella