pyaerocom icon indicating copy to clipboard operation
pyaerocom copied to clipboard

Indicate scores from a degraded dataset

Open rkouznetsov opened this issue 2 months ago • 4 comments

The time series of e.g. RMSE show substantial degradation (or disturbance) of scores for periods where the observations have too little data points. see e.g. 21.10.2025 in https://regional-evaluation.atmosphere.copernicus.eu/pages/overall/?project=cams2-83&experiment=forecast-last-week&tab=timeseries&statistic=nrms&region=ALL# The reason is that only ~20 out of usual 1400 stations decide on these scores for that day, see https://regional-evaluation.atmosphere.copernicus.eu/pages/overall/?project=cams2-83&experiment=forecast-last-week&tab=timeseries&statistic=num_valid&region=ALL#

I wonder if there should be some way to mark in all scores the data that have for e.g. with colors for >90% (green) >50% (yellow) and >10% (red) of the maximum available data..

Thank you!

rkouznetsov avatar Oct 28 '25 07:10 rkouznetsov

Hi Rostislav, thanks! Yes, we're aware. This was due to a technical issue between Meteo France and EEA which lasted for about one day. Only an extremly small number of measurements was available... It hit some models more than others, e.g. MOCAGE PM10 https://regional-evaluation.atmosphere.copernicus.eu/pages/overall/?project=cams2-83&time=2025&season=All&tab=timeseries&statistic=nmb&parameter=concpm10&experiment=forecast-last-week&model=MOCAGE&frequency=daily&region=ALL&station=# I wanted to discuss this with MOCAGE and Meteo France at a meeting this morning, but they didn't show up. Yes, it would be good to make it clear somehow when there is a severe lack of measurement data. I'll put it on my list (which is quite long right now) and discuss it with the team.

michaelgau avatar Oct 28 '25 14:10 michaelgau

Thanks, Michael! Probably, it hits those who are especially off in Estonia and Kosovo (two countries that actually represented for that period). EEA has not been super-reliable data source. Do you have any mechanism to backfiill the observations?

rkouznetsov avatar Oct 28 '25 15:10 rkouznetsov

Backfilling observations would be good (in my opinion), but it is not in the workplan and we have not implemented operationally. Would need to be agreed with ECMWF, as it would change the scores retrospectively, and lead to deviation from published EQC reports.

michaelgau avatar Oct 31 '25 10:10 michaelgau

I believe, that before the report, back-filling should not do any harm. I would assume that the most-complete dataset by the moment of the report preparation should be used for the quarterly reports.. Is that the case currently?

rkouznetsov avatar Oct 31 '25 13:10 rkouznetsov