Challenge #31 -ECMWF forecast data in 3D with Blender

[EsperanzaCuartero]

Challenge 31- ECMWF forecast data in 3D with Blender

Stream 3 - Visualize weather, climate and atmosphere

Goal

Extend existing visualisation software to support ECMWF forecast data + examples

Mentors and skills

• Mentors: @StephanSiemen , @sylvielamythepaut
• Skills required:
  • Python
  • xarray
  • Blender
  • git + GitHub

---

Challenge description

For creating high quality visualisations of forecast data, Blender has proven a valuable tool. There has been various attempts to load GRIB data but they have not been sustainable and scalable. A new project started called BlenderNC to offer a Python add-on using xarray to load NetCDF data. This project should use the xarray extension cfgrib to extend BlenderNC to be able to read and visualise GRIB data. It is also expected that the documentation at https://blendernc.readthedocs.io will be updated with examples of using GRIB data. Interested candidates have a chance to work on an exciting Python project and learn how to handle weather model data and how to visualise it.

[whatnick]

I would be interested in this challenge. I have the relevant experience in Python, Xarray and Blender. I work regularly with NetCDF and more recently with GRIB files for forcing WRF models.

[EsperanzaCuartero]

Hi Tisham, thanks for your interest. If you have a question, you can start a conversation with the mentors of this challenge and tailor your proposal before submitting it. The deadline for applications is on 16 April. Best, Esperanza

[jwagemann]

Hi, join us for the ECMWF Summer of Weather Code Ask Me Anything session and learn all things ESoWC.

When: Wednesday, 24 March 2021 at 4 pm GMT

What: learn everything about ESoWC - how it works, the challenges this year, some tips for your proposal and listen to ESoWC experiences from previous participants

How: register here.

[whatnick]

Hi @sylvielamythepaut @StephanSiemen , @josuemtzmo and I are putting together a joint proposal for this challenge. Josue is a maintainer for the BlenderNC project and in Canberra with myself. We look forward to working on BlenderNC.

[StephanSiemen]

Hi @whatnick, Thanks for letting us know. Sounds great! Looking forward to your proposal :-)

[whatnick]

@StephanSiemen I currently have this challenge loading the ECMWF forecast data I have using cfgrib. https://github.com/ecmwf/cfgrib/issues/232 , some assistance will be appreciated.

[iainrussell]

Hi @whatnick , I'm writing a comment into the cfgrib issue, but in the meantime can you successfully use the ecCodes tool 'grib_to_netcdf' as described here if you only want to do a file conversion? https://confluence.ecmwf.int/display/ECC/grib_to_netcdf You will need to install ecCodes, perhaps through conda-forge. It might work, but mixing GRIB editions could be problematic, let's see.

[josuemtzmo]

Hello, As far as I know, BlenderNC could support cgrib files by implementing:

 xr.load_dataset('filename.grib', engine='cfgrib')

an xarray example can be found at: http://xarray.pydata.org/en/stable/examples/ERA5-GRIB-example.html

[iainrussell]

Another thought, if cfgrib is complaining because of a mixture of GRIB1 and GRIB2 messages, you can separate them using the filter_by_keys argument described here: https://github.com/ecmwf/cfgrib Untested suggestion: g1 = xr.open_dataset('mydata.grib', engine='cfgrib', backend_kwargs={'filter_by_keys': {'edition': '1'}}) g2 = xr.open_dataset('mydata.grib', engine='cfgrib', backend_kwargs={'filter_by_keys': {'edition': '2'}}) Then you can save them into separate netCDF files if that's useful.

[sylvielamythepaut]

Hi,

I just created a shared forder on Dropbox . I put 1 set of data ( in Grib and NetCDF)

This is a 10 days forecast for 5 parameters: 2t : 2m temperature lcc, mcc, hcc : low, Medium, High Cloud Cover tp : Total precipitation ( I think they are accumulated .. May be we need to do a bit of processing on this one ) That should allow us to start

[whatnick]

I have downloaded and verified that this data can be loaded successfully. What are the license conditions for this data. If this data can be CC licensed we could place it in our Github under LFS and use as test data in CI. I have put my exploratory code in a notebook here. https://gist.github.com/whatnick/0e2821f1abacba8d57c1ebdeea6487d7

More details on GIT LFS are available here. https://git-lfs.github.com/ . If a suitable licence can be obtained then please feel free to make a PR to blenderNC with this data in say a data folder.

[EsperanzaCuartero]

Hi Tisham,

I've already sent an invite for you to create a repository under ESoWC space GitHub. I've used the GitHub handle you provided in the form https://github.com/josuemtzmo

Thanks.

Best Esperanza

---

From: Tisham Dhar @.> Sent: Saturday, May 29, 2021 8:53 AM To: esowc/challenges_2021 @.> Cc: Esperanza Cuartero @.>; Author @.> Subject: Re: [esowc/challenges_2021] Challenge #31 -ECMWF forecast data in 3D with Blender (#4)

I have downloaded and verified that this data can be loaded successfully. What are the license conditions for this data. If this data can be CC licensed we could place it in our Github under LFS and use as test data in CI. https://gist.github.com/whatnick/0e2821f1abacba8d57c1ebdeea6487d7https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgist.github.com%2Fwhatnick%2F0e2821f1abacba8d57c1ebdeea6487d7&data=04%7C01%7Cesperanza.cuartero%40ecmwf.int%7C02dcd458111c4cabb9f608d9227f4909%7C21b711c6aab74d369ffbac0357bc20ba%7C1%7C0%7C637578752340146347%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=lq%2BLpcZEDXDc6FJ6K8QY2uarjQYzxJltRVokiSfnRrc%3D&reserved=0

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fesowc%2Fchallenges_2021%2Fissues%2F4%23issuecomment-850798597&data=04%7C01%7Cesperanza.cuartero%40ecmwf.int%7C02dcd458111c4cabb9f608d9227f4909%7C21b711c6aab74d369ffbac0357bc20ba%7C1%7C0%7C637578752340146347%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zz71dmKo7qsWoxbtlY3V7msVD67GOEQ7%2Bp3Ft94A1gE%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALENNJO347Y7HX3AEEFAYDTTQCTSBANCNFSM4WW66S3Q&data=04%7C01%7Cesperanza.cuartero%40ecmwf.int%7C02dcd458111c4cabb9f608d9227f4909%7C21b711c6aab74d369ffbac0357bc20ba%7C1%7C0%7C637578752340156291%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2FzGVereRq0ggbjQFUGK2CUBUvTpyv1KY2NNsFYLMbbk%3D&reserved=0.

[sylvielamythepaut]

I do not think there is any issue with the license, but I will check and come back to you.

[josuemtzmo]

Hi Tisham, I've already sent an invite for you to create a repository under ESoWC space GitHub. I've used the GitHub handle you provided in the form https://github.com/josuemtzmo Thanks. Best Esperanza …

Hello Esperanza, I've accepted the invitation, I've created an issue to draft the README file Issue 51.