xMIP icon indicating copy to clipboard operation
xMIP copied to clipboard

Add support for unit conversion

Open DamienIrving opened this issue 2 years ago • 4 comments

Just wanted to make a quick note here that if you're thinking about adding unit conversion functionality to cmip6_preprocessing (@jbusecke mentioned this in a presentation I attended earlier today), the xclim package has put a lot of work into implementing pint in the context of xarray data arrays (for details see xclim.core.units).

Others might have a different experience, but I certainly found the xclim implementation of pint easier to implement into my xarray workflows than pint-xarray.

DamienIrving avatar Jul 22 '21 01:07 DamienIrving

Oh I wasnt even aware of that! I have been fighting with pint-xarray over in #160, but that would currently only correct units for depth. The knockout question for me is: Does this implementation handle wrapping of dask arrays?

cc @TomNicholas

jbusecke avatar Jul 22 '21 14:07 jbusecke

Hi @DamienIrving - thanks for showing us xclim!

I think the aims of the packages are fairly different: if I'm understanding correctly then xclim provides convenience functions for converting back and forth and operating on pint Quantities, but doesn't actually aim for full wrapping of pint arrays within xarray objects? If so then ideally a combination of upstream work in xarray, pint-xarray, and cf-xarray should eventually be able to replace much of xclim's functionality in a more general and integrated way.

For example xclim has a pint_multiply function that accepts an xarray.DataArray and a pint.Quantity, but thanks to @keewis' work upstream in xarray then if you quantify the dataarray you should be able to just directly multiply it with the pint.Quantity and it just work!

We also have to be mindful of separation of concerns: wrapping general numpy-like arrays =/= pint helper functionality =/= decoding cf conventions, and different users will want different combinations of these features.

That said it would be great to make use of your work in xclim, as Deepak suggested!

I certainly found the xclim implementation of pint easier to implement into my xarray workflows than pint-xarray.

If you have any specific issues then please raise them over in the pint-xarray issue tracker!

TomNicholas avatar Jul 22 '21 17:07 TomNicholas

No problem, @TomNicholas.

I totally agree that a combination of upstream work in xarray, pint-xarray, and cf-xarray should eventually be able to replace much of xclim's functionality in a more general and integrated way. I kinda just wanted to flag the xclim efforts in the unit conversion space because I made the switch from working with iris to xarray recently and stumbled across multiple options (xclim, metpy and pint-xarray) when it comes to unit conversion (I'm not involved in the development of any of those packages). I found it quicker and easier to pick up and go with the xclim functionality, but it is on my to-do list to come back to pint-xarray, so when I do I'll be able to better express any thoughts about specific issues.

DamienIrving avatar Jul 23 '21 00:07 DamienIrving

FYI the custom unit formats PR was merged into pint so now it should be possible to register xclim.core.units.pint2cfunits as "cfunits" or "cf" and get something like f"{quantity:.3f#cfunits}" to work (documentation is still missing, though)

see hgrecco/pint#1375 and hgrecco/pint#1371

keewis avatar Oct 10 '21 09:10 keewis