Use CRPS or DSS instead of RMSE to select the model?

[stephaneghozzi]

From Sebastian Funk: RMSE seems a slightly odd choice here - since the accuracy of the method will depend on probabilistic calibration (the outlier detection doesn't care about the mean), why not choose a proper scoring rule that reflects this, e.g. CRPS or DSS?

[dirkschumacher]

Good point. The reason we used RMSE initially was to have something that is implemented in yardstick