Adding support for ICON-EU in WPS

[grafmi]

The ICON-EU grib2 data set is now able to be ingested by the ungrib program.

1. New Vtable:
   • A new Vtable file for ICON-EU (EU-Nest) has been created with the fields required by the WRF model.
2. Information about the data:
   • The input data (grib2) of ICON-EU can be found here: https://opendata.dwd.de/weather/nwp/icon-eu/grib/
   • You will find the documentation about ICON and ICON-EU here: https://www.dwd.de/DWD/forschung/nwv/fepub/icon_database_main.pdf
3. Modifications to the ungrib source code:
   • I added support for level code 150 in rd_grid2.F because model levels of ICON-EU are on generalized vertical height coordinates. The general vertical height coordinate is not specified by vertical coordinate parameters but by providing a 3D field, that specifies the geometric height of every model grid point in meters. For more details, see: http://www.cosmo-model.org/content/model/modules/coding/grib/gribVerticalCoordinates.pdf
4. Addition of new soil levels:
   • New soil levels from ICON-EU are added to METGRID.TBL.ARW for metgrid.exe. Details about the soil levels in ICON-EU can be found in the documentation (see above)
5. Conversion of soil moisture units to those expected by the WRF model:
   • The preprocessing in rrpr.F has to be modified to convert the units of column-integrated soil moisture provided by ICON-EU from kg m-2 to m3 m-2 for WRF input.

At the moment, I'm using a layer which exists only in ICON-EU as trigger to do the conversion. Maybe, there's a better solution.

I have performed two model runs with exactly the same configuration and input from GFS and ICON-EU (init time 20200531_00 and lead time 72 h). The output of both runs are similar, but it can be recognized, e. g., that there is less shallow convection over Germany in the afternoon of the 31th May 2020. This was expected because GFS often shows too high low-level moisture values over Central Europe, especially in spring and summer.

GFS: https://www.meteoprime.ch/wrf_gfs/20200531_00/ ICON-EU: https://www.meteoprime.ch/wrf_icon/20200531_00/

[davegill]

@grafmi Michael, Would you provide a good commit message. For example, a few bullets for your modifications, and then a sentence or two for each bullet.

1. Since this is a pre-processor, it is difficult to gauge the correctness of the data you are using (assigning, etc). For your testing, do you have a sample model output with two different input data sources? Could you show those images (this input vs that input)? We need to know that the real.exe program is providing data correctly to the model with this new input.
2. Also, if this is an available data set, in the commit message and in the Vtable, can you put the location of the online data source. An example for this type of information would be in the file WPS/ungrib/Variable_Tables/Vtable.GFS.

[grafmi]

@davegill Dave, I provide a better commit message. The first tests show that there was a problem with the soil moisture input (kg m-2), so I had to modify rrpr.F. Now, everything seems to work fine.

[davegill]

@grafmi Michael, I looked through the WRF-generated output fields. I could easily see the low-level moisture bias that you mentioned from the RH2m field. This looks pretty good to me.

Were any changes required in the WRF model to support this data? That would be a separate matter, but I am curious.

[davegill]

@mgduda @jimbresch Folks, Would you review this PR. The WRF model output results seem to indicate that this was done correctly.

[davegill]

@mgduda @jimbresch I have highlighted an issue that the contributor posed about how to trigger the soil moisture conversion. Right now that conversion is based on the presence of a particular level.

[davegill]

@grafmi Michael,

1. Is the resolution of the data of the grib data about 6 km over Europe?
2. Is this data freely available in real-time?
3. Is access to the global 13-km data freely available also?
4. What files does a person download? There seem to be MANY files in the directory.

[smileMchen]

@grafmi I would like to run a test case using the ICON-EU data. Would you please specify what data I should download from the website you provided (https://opendata.dwd.de/weather/nwp/icon-eu/grib/)? I looked at this website, there are many sub-directory and sub-subdirectory. Please clarify what data I should download from which sub-directory. Thanks.

[grafmi]

@davegill Dave, I didn't change anything at the WRF model at the moment. Currently, I don't understand completely how and where the interpolation of the soil moisture and temperature layers is done - I'm using the Noah Land Surface Model (option 2) with 4 layers. I will dive into the code in the next few days and check if there is room for further optimizations in the WRF model code.

1. The resolution of the data is about 6.5 km on the native (triangular) grid or 0.06° on the regular lat-lon grid.
2. Yes, the data are freely available in real-time. I'm using the data on the regular lat-lon grid.
3. Yes, the global data are also freely available, but most necessary fields for WRF are on the native (triangular) grid. I don't know, whether WPS is able to handle that.
4. @smileMchen I attached you a bash script that I'm using to download the necessary files. Hope that it helps. download_icon.zip

[smileMchen]

@grafmi Hi Michael, Thank you!

[sekluzia]

Hi,

It would be great also to include the global ICON fields on the native (triangular) grid in the WPS.

Kind regards, Artur

[mgduda]

The final result of the changes in the seven commits in this PR looks fine to me. Recognizing that this would create more work, and that we really should provide better documentation to help code contributors, I'd like to propose two changes:

1. In general, we'd like PRs to be created from a new branch, e.g., ICON-support, rather than from the master branch on a fork of the repository. We're moving toward the convention of including the name of the branch in the merge commit message, and it looks odd if the message says, e.g., that we're merging master onto develop for any reason except to include bug-fixes from a release branch. Committing changes to a new branch, and creating a PR from that, also helps contributors to keep the history of their master branches identical to the history of the main master branch; hard-resets can fix this, but having to hard-reset one's master branch suggests that an alternate development strategy might be better.
2. Unlike in the WRF repository, we don't "squash-and-merge" in the WPS repository, which means that the complete commit history is preserved in the repository. Consequently, it's nice to keep the commit history of branches as tidy as possible. For this PR, I think two commits should suffice: one to update ungrib, and a second to update the METGRID.TBL file.

[grafmi]

@mgduda As you proposed, I created another branch with the name ICON-support and reduced the number of commits from 7 to 4 for each modified file, which seems meaningful to me. If needed I can reduce the number of commits even further. Do I have to do another pull request to change the source branch from grafmi:master to grafmi:ICON-support?

[sfalmo]

Is this pull request still considered?

The possibility of ICON-EU based WRF runs for regions in Europe is certainly interesting. I think many users could benefit from this feature and would be glad to have it incorporated in a future release.

[weiwangncar]

@smileMchen Are you able to run a test with this dataset?

[smileMchen]

@weiwangncar

I met some problems when ungribbing the data. The error message is like:

Subsoil level 0 in the GRIB2 file, was not found in the Vtable Subsoil level 243 in the GRIB2 file, was not found in the Vtable ERROR: Data not found: 2019-04-19_01:00:00.0000

However, the soil are included in the file downloaded from icon_eu. Vtable.ICON-EU also looks fine.

Ming

[weiwangncar]

@smileMchen Can you look into this a bit more, and work with @grafmi to resolve the problem? Thanks.

[smileMchen]

@weiwangncar

I am using the master branch instead of the ICON-support branch, --- I believe the two branches are the same. But I will try the ICON-support branch just in case.

[weiwangncar]

@smileMchen That would be good start. @grafmi did make changes to the code and Vtable.

[smileMchen]

@weiwangncar @grafmi Wei, ungrib still doesn't work, and I haven't figured out yet what is wrong. The datafile includes everything WPS requires. Michael, I have made slight modification to the download script, basically to ensure one singe file only contains data for a single time. All others remain the same as in your original script. It is mysterious why ungrib cannot process the data I downloaded. Would you please send me one single-time ICON-EU datafile to take a look? Thanks. Ming

[smileMchen]

@grafmi Michael, Would you please send me one single ICON-EU datafile to take a look? Or if you want to take a look at the ICON-EU datafile I downloaded, please let me know. I can send you the file so that we can compare the difference. Thanks. Ming

[sfalmo]

I've had a look at this again and the following has all been done with the @grafmi:ICON-support branch.

I downloaded the ICON data with the script download_icon.sh from above (after creating and setting the directory for the data), ran link_ungrib.csh which produced a lot of GRIBFILE.XXX as expected and then started ungrib.exe.

ERROR: Data not found: 2019-04-19_01:00:00.0000

@smileMchen This error is most likely due to a wrong start or end date in namelist.wps, isn't it? At least I can avoid it that way, and get ungrib to finish successfully (but with warnings).

Subsoil level 0 in the GRIB2 file, was not found in the Vtable Subsoil level 243 in the GRIB2 file, was not found in the Vtable

I am getting the same two warnings about superfluous soil levels as @smileMchen. But in my opinion, these can be ignored as they just point out that more soil levels are present in the grib files than ungrib is being told to process in the Vtable. Note that 6 levels for soil temperature and soil moisture are specified in Vtable.ICON-EU but 7 are downloaded in download_icon.sh respectively.

One would either have to alter the Vtable to include the additional levels as well, or change the download script to not fetch them in the first place. Btw: There are actually 8 levels for soil moisture and 9 levels for soil temperature as low as 20m on the OpenData server, see e.g. https://opendata.dwd.de/weather/nwp/icon-eu/grib/00/w_so/ and https://opendata.dwd.de/weather/nwp/icon-eu/grib/00/t_so/. @grafmi Why doesn't the Vtable and METGRID.TBL.ARW include all available levels?

Anyway, even with the currently used ones, I could continue with metgrid.exe, real.exe (after setting the values num_metgrid_soil_levels = 6 and num_metgrid_levels = 61 in namelist.input) and finally ran wrf.exe successfully. The output parameters I looked at seemed alright, so I think with a bit of cleanup this feature could get merged very soon.

[dudhia]

The real.exe program can only handle specific soil levels from GFS, RUC and ECMWF so far.

On Mon, Jun 21, 2021 at 9:37 AM sfalmo @.***> wrote:

I've had a look at this again and the following has all been done with the @grafmi https://github.com/grafmi:ICON-support branch.

I downloaded the ICON data with the script download_icon.sh from above (after creating and setting the directory for the data), ran link_ungrib.csh which produced a lot of GRIBFILE.XXX as expected and then started ungrib.exe.

ERROR: Data not found: 2019-04-19_01:00:00.0000

@smileMchen https://github.com/smileMchen This error is most likely due to a wrong start or end date in namelist.wps, isn't it? At least I can avoid it that way, and get ungrib to finish successfully (but with warnings).

Subsoil level 0 in the GRIB2 file, was not found in the Vtable Subsoil level 243 in the GRIB2 file, was not found in the Vtable

I am getting the same two warnings about superfluous soil levels as @smileMchen https://github.com/smileMchen. But in my opinion, these can be ignored as they just point out that more soil levels are present in the grib files than ungrib is being told to process in the Vtable. Note that 6 levels for soil temperature and soil moisture are specified in Vtable.ICON-EU https://github.com/grafmi/WPS/blob/ICON-support/ungrib/Variable_Tables/Vtable.ICON-EU but 7 are downloaded in download_icon.sh respectively.

One would either have to alter the Vtable to include the additional levels as well, or change the download script to not fetch them in the first place. Btw: There are actually 8 levels for soil moisture and 9 levels for soil temperature as low as 20m on the OpenData server, see e.g. https://opendata.dwd.de/weather/nwp/icon-eu/grib/00/w_so/ and https://opendata.dwd.de/weather/nwp/icon-eu/grib/00/t_so/. @grafmi https://github.com/grafmi Why doesn't the Vtable and METGRID.TBL.ARW include all available levels?

Anyway, even with the currently used ones, I could continue with metgrid.exe, real.exe (after setting the values num_metgrid_soil_levels = 6 and num_metgrid_levels = 61 in namelist.input) and finally ran wrf.exe successfully. The output parameters I looked at seemed alright, so I think with a bit of cleanup this feature could get merged very soon.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/wrf-model/WPS/pull/154#issuecomment-865133387, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEIZ77GPUIFPR4QXUK5DM4TTT5MFNANCNFSM4NMO6VAA .

[weiwangncar]

@dudhia real should be able to handle any soil levels if they are set correctly in Vtable as well as METGRID.TBL. It looks like this PR has that.

[dudhia]

It has hardcoded names for ST00* etc., but maybe that has been changed.

On Mon, Jun 21, 2021 at 11:13 AM weiwangncar @.***> wrote:

@dudhia https://github.com/dudhia real should be able to handle any soil levels if they are set correctly in Vtable as well as METGRID.TBL. It looks like this PR has that.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wrf-model/WPS/pull/154#issuecomment-865205924, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEIZ77FNDF7ZWIAY3I5ITXTTT5XNNANCNFSM4NMO6VAA .

[alexrsanchez]

Hi everyone, I don't know if this is the right place to ask this question, but seemed logic to me as is about ICON-EU in WRF. Sorry in advance if it's not the proper place. I am trying to run WRF using ICON-EU as input data instead of GFS. So far I've been able to edit the WPS properly in order to add the soil levels that ICON uses as described @grafmi in the first message of this conversation.

However, when running ./real.exe I got the following error:

Using sfcprs to compute psfc ----- ERROR: The reference pressure is not monotonically decreasing This tends to be caused by very high topography (i,j) = 1 1 , topography = 0.0000000E+00 m k = 1 , reference pressure = 100000.0 Pa k = 2 , reference pressure = 100024.0 Pa -------------- FATAL CALLED --------------- FATAL CALLED FROM FILE: LINE: 1236 In the dynamics namelist record, reduce etac from 9.9999998E-03

---

As you know better than me, this is usually an error that raises with high topography, but as you can see, topography is 0m in my case. I am almost certain that the error raises because ICON vertical levels are reversed: as you can see in the attached image (WRF at the left part and ICON at the right), ICON assigns the first level (lev=1) to the top one and the last level (lev=61) to the surface layer, so pressure increases between levels 1 and 2 as the error points out, while GFS and WRF level assignment starts at the surface and ends at the highest desired level. How can I reverse the levels in the ICON grib2 files so I can remove this error?

[smileMchen]

@sfalmo @alexrsanchez

I downloaded ICON data back in April using the script provided. I have the four datafile listed below:

icon_2021041900_000.grib2 icon_2021041900_001.grib2 icon_2021041900_002.grib2 icon_2021041900_003.grib2

All the above 4 files seem to have the same time, and ungrib is not able to find correct time contained in these files. I can only tell their time based on the file name, i.e., 2021041900_000 (initialized time), 2021041900_001(1-hour FCTS), 2021041900_002(2-hour FCST), etc.

Would you please describe how you manage to make ungrib recognize the times in the field?

I would appreciate if you can provide a sample download script, Vtable, and namelist.wps. I would like to at least ungrib these files first.

Thanks in advance.

Ming

[sfalmo]

@smileMchen Sure, I did the following:

1. Compile the ICON-support branch of @grafmi.
2. In the WPS directory: mkdir -p data/icon data/tmp and create the download_icon.sh script.
3. Run ./download_icon.sh <YYYYMMDD> <HH>, so e.g. for today's 00Z data: ./download_icon.sh 20210630 00. This takes some time, so you might want to run it in the background and observe the progress e.g. via watch ls data/icon. You should see this directory gradually populate with icon_YYMMDDHH_XXX.grib2 files, where XXX is the forecast hour.
4. Run ./link_grib.csh data/icon, which produces GRIBFILE.AAA, GRIBFILE.AAB, and so on.
5. Link the correct Vtable, i.e. ln -s ungrib/Variable_Tables/Vtable.ICON-EU Vtable. You see that I use the one provided by the ICON-support branch.
6. Make sure you have a valid namelist.wps, especially considering start_date and end_date. See below for the one I used.
7. Run ./ungrib.exe, do not worry about warnings.
8. To proceed, you must have set up a domain with geogrid.exe. In my case, I just copied the geo_em* files from another setup (after I checked that the region is indeed covered by the ICON-EU model).
9. Link METGRID.TBL from the ICON-support branch, i.e. ln -s metgrid/METGRID.TBL.ARW METGRID.TBL and run ./metgrid.exe.

And there you have it, a complete run of the WPS with ICON data.

Finally, here are the exact files I used: icon.zip

[sfalmo]

@alexrsanchez And by the way, I do not get errors when running real.exe and wrf.exe after doing the WPS procedures from the previous comment.

Here's an example for the namelist.input I use: namelist.input.zip

[smileMchen]

@sfalmo Thank you very much for the kind information ! I am able to successfully run WRF using the ICON data as input. Results are reasonable.

@alexrsanchez Would you please try WRFV4.3? I have attached a sample namelist.input just in case you want to take a look.

Ming

namelist.input.txt

[sekluzia]

@sfalmo

Will this work for the global ICON data (https://opendata.dwd.de/weather/nwp/icon/grib/)?