pyaerocom icon indicating copy to clipboard operation
pyaerocom copied to clipboard

Model maps shown for EMEP should use the frequency of the Base_*.nc file rather than the colocated data and then resampling

Open lewisblake opened this issue 1 year ago • 2 comments

Is your feature request related to a problem? Please describe. An inefficiency in the creation of the model maps is that for some models, such as EMEP, we already have yearly files available, so reading in daily data and then temporally averaging is unnecessarily expensive.

Describe the solution you would like to see In the case of making maps for the EMEP model, the maps_freq argument set in a config file should be used to read the Base_{maps_freq}.nc file already available, regardless of the colocation frequency main_freq. Some information in the logger should be present to inform that this option is being used.

Additional context Blocked by #1259

lewisblake avatar Jul 12 '24 12:07 lewisblake

This is critical for the trends-evaluation, which does collocation in monthly data which takes 45min for 4 periods without model-maps. Model-maps for one period takes 6h alone, since daily data is read.

heikoklein avatar Jul 19 '24 06:07 heikoklein

I've started on the problem in #1312. If I understand the problem correctly, this should solve it, even though it is not in the most elegant way:

I'm testing out simply changing main_freq to maps_freq when the maps are calculated(or the coarsest freq if that option is chosen). It of course then colocates the obs wrong. It seems that ModelMapsEngine creates a colocator, but only uses the model data, in which case the obs are irrelevant. And as I understand it, the obs shown on the map are created elsewhere, so this does not matter anyway(?). The obs shown will also be just the July data for that year, and not a mean of the year.

The freq change is done for all models, no special case for EMEP. I can change that to check if the model is EMEP, as we are not always sure to have yearly data

I think my code is a bit misplaced, but due to the overuse of OOP, it is not easy to know where things are happening. But we can discuss that later

dulte avatar Aug 16 '24 16:08 dulte