sktime icon indicating copy to clipboard operation
sktime copied to clipboard

[ENH] Add Overall Weighted Average (OWA) Score

Open jgyasu opened this issue 1 month ago • 1 comments

This PR adds overall weighted average score which was used in the M4 forecasting competition.

The metric is defined as: OWA = 0.5 * ( MASE / MASE_Naive2 + sMAPE / sMAPE_Naive2 ). Naive2 here is defined as following:

Screenshot 2025-11-25 at 1 41 50 AM

I am not aware of the method to get Naive2 in sktime, so in the doctest example, I have used pre-defined array as y_pred_benchmark.

jgyasu avatar Nov 24 '25 20:11 jgyasu

I am not aware of the method to get Naive2 in sktime, so in the doctest example, I have used pre-defined array as y_pred_benchmark.

I see, that is the right decision - simply make y_pred_benchmark arbitrary.

Btw, you can get Naive2 by using Deseasonalizer, and the TransformIf compositor together with the SeasonalityACF estimator (but that is not needed).

fkiraly avatar Nov 25 '25 00:11 fkiraly