h5web

h5web copied to clipboard

Thanks for this wonderful tool; The VS Code extension is super handy.

Is your feature request related to a problem?

No

Requested solution or feature

I know GRIB files are a completely different data format, but there are conversion tools to convert GRIB2 to NetCDF4. I wondered if it were possible to extend h5web to support GRIB2 files, either reading GRIB2 files directly, or converting them to HDF5 before reading them.

Thanks for considering this idea.

Hi @blaylockbk! In therory, H5Web is architected in a way that makes this possible.

The simplest approach from the current state of the codebase would be to write a back-end server with the same API as h5grove but for reading GRIB files instead of HDF5 files. You could then plug in H5GroveProvider in the front-end and it should "just work".

If you're more after an h5wasm, serverless approach, then this would require writing your own data provider. We've always had the goal to make this possible (outside of the codebase) by exporting all the right things from @h5web/app, so this would be a good opportunity to actually do it. What your provider does would then be entirely up to you: it could convert GRIB files to NetCDF4 or read GRIB files directly.

Hello @blaylockbk

I assume your end goal is to be able to read GRIB2 files in the VSCode extension. For this, we would need a way to read GRIB2 files in JS.

I looked into this for a bit but I struggle to find up-to-date information about the GRIB2 format and ways to read GRIB2 in JS.

Could you offer some guidance on this ? What do you usually use yourself to read GRIB2 files ?

Yes, that is the main goal. If VS Code extension requires JS to read the data, I'm not certain there is an easy way to do that.

I typically use the python package pygrib or cfgrib to read GRIB2 files. The cfgrib package relies on eccodes to read GRIB2 files.

Here are some references on ECMWF's eccodes tool:

• https://confluence.ecmwf.int/display/ECC/What+is+ecCodes
• https://confluence.ecmwf.int/display/OIFS/How+to+convert+GRIB+to+netCDF

Thanks for the links :slightly_smiling_face: . So, if we don't a way to do it in JS, we could still write a back-end provider in Python with pygrib.

I will continue investigating.

All right, I may have found an alternative way to solve this.

Thanks to the link you provided, I was able to convert a GRIB file (test file from pillow) into a netCDF4 file.

grib_to_netcdf -o WAlaska.nc -k 3 WAlaska.wind.7days.grb

(the -k 3 is needed to get netCDF4 instead of regular netCDF)

As the VSCode extension supports netCDF4, I was able to read this file with H5Web in VSCode :tada:

Could you try the previous process with one of your own GRIB files and report how it went ?

As per discussions here and in https://github.com/usnistgov/h5wasm/issues/39#issuecomment-1338076045, GRIB support seems too far out of scope of the H5Web VS Code extension. This would need to be developed as a separate extension. I'm going to close this issue, but in case someone wants to take this on, I see two approaches:

• Investigate the feasibility of compiling a GRIB parsing library to WebAssembly to build a similar extension to vscode-h5web.
• Develop a server (in Python or whatever) to read GRIB files, with a similar to API h5grove, and then start this server from a VS Code extension. The webview of the extension would then be able to render the H5Web viewer with the H5GroveProvider.