python-rasterstats icon indicating copy to clipboard operation
python-rasterstats copied to clipboard

netCDF support

Open perrygeo opened this issue 10 years ago • 4 comments

netCDF support in GDAL/rasterio leaves something to be desired.

The netcdf4-python module would be a big improvement and allow for native access to netcdf files as well as GRIB2 and OpenDAP, in a more robust way.

Most likely depends on #73

perrygeo avatar Sep 16 '15 15:09 perrygeo

Possibly relevant: we've been working on a mostly netCDF focused library called clover that builds on top of the netcdf4 library as well as rasterio, etc. We work with netCDF a lot, and found that netcdf4 gave us what we needed at a lower level, but we needed a bunch of utility functions that sit on top of that to get our work done. (warning: it is still early version and documentation is totally lacking).

What are you specifically going for? Simple I/O with netCDF? We'd love to talk further; coupling netCDF data structures with raster concepts is of high interest to us.

xray may also be worth checking out for your needs too.

brendan-ward avatar Sep 16 '15 16:09 brendan-ward

@brendan-ward I'll have to check out clover. I think netCDF direct might work as all I need to do is get the metadata to construct an affine transform and do indexed I/O. All of the window logic, bounds->index transforms, etc. happens internal to rasterstats already.

Of course the tricky part with netcdf files tends to be the longitude wrapping (0 to 360), subdatasets and the inverted y axis - GDAL is horrible at handling all of that. How does clover manage those issues?

perrygeo avatar Sep 16 '15 16:09 perrygeo

Clover handles those issues in abstractions that I put on top of coordinate dimensions:

However, the documentation for working with those abstractions is totally lacking. Best I've got for you at the moment is code that uses them for converting flat rasters to netCDF

These assume that there are variables in the netCDF file with the pixel centroid coordinates, so that we can turn them into bounds and cell size, and figure out direction. Also assumes that pixels are equally spaced, which is sometimes not the case in netCDF depending on where you get things.

I've not yet had to deal with longitude wrapping (most of our stuff is bounded to US) or subdatasets. netCDF lets you do lots of beautiful, complex, sometimes nonstandard things; I've only had to deal with a small subset of the possibilities.

Also worth checking out is ocgis

brendan-ward avatar Sep 16 '15 17:09 brendan-ward

That looks great! The y-direction and affine methods will come in very handy. When I get around to this I'll definitely give clover a whirl.

Currently I'm just handling all of it with VRTs to deal with the quirks of the GDAL netcdf driver. It's quite amazing that you can handle most things like y direction, longitude wrapping, transforms, etc in an xml wrapper! But quite disappointing that you need to in the first place.

perrygeo avatar Sep 16 '15 17:09 perrygeo