no2 and o3 maps are not plotted

[andagit]

Describe the bug Please provide a clear and concise description of what the bug is.

• Pyaerocom version: "0.23.0"
• Computing platform: Lustre PPI
• Configuration file (if applicable): cfg_cams2-82_IFS-ESUITE-beta.py
• Error message (if applicable): Maps of no2 and o3 are not appearing in https://aeroval-test.met.no/annac/pages/maps/?project=cams2-82&experiment=IFS-ESUITE-beta&tab=timeseries&time=2003-2023&parameter=vmrno2&station=ALL

To Reproduce Steps to reproduce the behavior:

1. module load /modules/MET/rhel8/user-modules/fou-kl/aerotools/pya-v2024.06.cams82.conda
2. cd /home/annac/Aeroval/config/config_files
3. aeroval_parallelize --cachegen-only --cacheram 70 -m /modules/MET/rhel8/user-modules/fou-kl/aerotools/pya-v2024.06.cams82.conda cfg_cams2-82_IFS-ESUITE-beta.py aeroval_parallelize --nocache --anaram 64 -m /modules/MET/rhel8/user-modules/fou-kl/aerotools/pya-v2024.06.cams82.conda cfg_cams2-82_IFS-ESUITE-beta.py

Expected behavior A clear and concise description. With the PyAerocom version "0.20.dev1" and same configuration file no2 and o3 maps are plotted, but not anylonger.... annac@ppi-r8login-b1:~/Aeroval/data/cams2-82/IFS-ESUITE-beta$ ls contour/no2 ls: cannot access 'contour/no2': No such file or directory

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.

[lewisblake]

As we discussed, this could be a memory issue. Charlie had a similar issue with the CAMS2_83 run last week but it solved itself on a rerun. Please try this and report back :)

[andagit]

It is not a memory issue, but Jan is suggesting that when introducing the observations to the maps, PyAerocom map plotting became a bit more strict on model variable name and is struggling with reading mmro3 and mmrno2 (req. vmro3 and vmrno2) from the ECMWF model input model files.

[lewisblake]

Probably we should introduce a target unit into the ModelMapsEngine (if there isn't one already). We should check to see if in the model maps whether these conversions are occurring for species which are e.g., ug S m-3 vs ug m-3

[lewisblake]

@MichaelSchulzMETNO Is this a priority for CAMS2_82?

[MichaelSchulzMETNO]

I believe that should be "standard quality", to be able to show the maps for all evaluated variables. Does it help to plot only monthly maps?

[lewisblake]

@andagit when you get a chance, could you please try this fix #1422 and report back?

[andagit]

https://aeroval-test.met.no/annac/pages/maps/?project=cams2-82&experiment=IFStest is run with pya-v2024.12 and the maps for O3 and NO2 are now showing obs, but no model data: "No contour data available"

[lewisblake]

Jan and I would like to touch base about how to approach this. We think it may be better to invest our time into converting the data before reading it into pyaerocom. The conversion there is not very sophisticated and slows down the runs quite a bit. In this case. applying the same simple conversions ahead of time might be cheaper.

[lewisblake]

@jgriesfeller will help write a script to do the conversion. @michaelgau will decide on the conversion factor

[michaelgau]

I assume that IFS provides concentrations in ug/m3, correct? The conversion factor to volume mixing ratio (I assume you want ppbv) depends on p and T. Do we have p and T data available? And does any of you know if IFS report their results for ambient p and T (i.e. the 'real' p and T used in the model) or if they convert this to what it would have been at standard p and T? The conversion factor depends on that... But if you don't know, and cannot find out, then I recommend a conversion factor of 2.00 for ozone, and 1.91 for NO2. To get from ozone in ug/m3 to ppbv you divide by 2.

(Btw, 'mass mixing ratio' should be expressed in g/kg or kg/kg or something like that. It's dimensionless, mass per mass, but that's another discussion.)

AND VERY IMPORTANT: use these factors only at the surface!! If you need to convert at higher altitudes then you MUST use p and T (either the real ones, or assuming a standard atmosphere)

[michaelgau]

In CAMS regional production and regional evaluation this discussion of unit conversion has been going on for years, and it's still not solved (only last weekend I re-opened Pandora's box), but that would be a longer post.

[jgriesfeller]

I had a quick look at the data. The files we have at the moment have both kg kg**-1 as unit:

mmrno2:units = "kg kg**-1" ;
mmro3:units = "kg kg**-1" ;

@andagit that's the unit you get from ECMWF, right?)

According to aeroval we want ppb (that's in the lower left panel of the evaluation page) or nmol mol-1 as unit (that's what the lower right panel says). The factor is of course the same.

[michaelgau]

OK, that makes things easier! Mass mixing ratios, given in kg/kg have the advantage of being independent of p and T, just like volume mixing ratios (at least within the p and T ranges we're dealing with :)

Btw, strictly speaking, we should write ppbv in our plots, and not just ppb. @AugustinMortier ?

And in order to get from kg/kg to ppbv one needs to multiply by the ratio of molecular weights for air (28.97g/mol) and the species in question (e.g. ozone: 48g/mol) and air (28.97g/mol) and 1e9.

Michela wrote a nice summary here: https://forum.ecmwf.int/t/convert-mass-mixing-ratio-mmr-to-mass-concentration-or-to-volume-mixing-ratio-vmr/1253/2 see section "Convert MMR to Volume Mixing Ratio (VMR, units ppmv or ppbv)" I suggest we use exactly these factors.

[michaelgau]

and yes, ppbv and nmol/mol are identical. nmol/mol being a bit more fancy, and also SI-compliant