climpred icon indicating copy to clipboard operation
climpred copied to clipboard

implement `train_test_split='fair-sliding'/'fair-all'`

Open aaronspring opened this issue 3 years ago • 1 comments

Is your feature request related to a problem? Please describe. New paper Risbey et al. 2021 demonstrates how forecaster choices for time selection in bias reduction impacts skill.

Reference: https://t.co/HE4PoROb8Z?amp=1

Describe the solution you'd like hindcast.remove_bias(how='additive_mean', alignment='same_verifs', method='fair_sliding')

implemented method from:

  • [x] unfair #655
  • [x] unfair-cv #655
  • [x] fair #655
  • [ ] fair-sliding
  • [ ] fair-all

unfair-cv is often standard, see #643, but unfair wrt real-time forecast/attainable skill, i.e. unfair means artificial skill

Demo: https://gist.github.com/aaronspring/94e24fb7e0a536be61caf1970287f3b5

aaronspring avatar Aug 01 '21 15:08 aaronspring

API: method

  • fair: train_init=slice(start, end) ~~,test_init=slice(start, end)~~ (need to change maybe to train_time for same_verifs)
  • fair-sliding: train_init=slice(start, end),~~test_init=slice(start, end)~~
  • fair-all: test_init=slice(start, end) ~~,train_init=slice(start, end)~~
  • unfair-cv, cv='loo'
  • unfair: now default

aaronspring avatar Aug 02 '21 21:08 aaronspring