DimensionalData.jl icon indicating copy to clipboard operation
DimensionalData.jl copied to clipboard

Conversions from xarray to `DimTree`

Open rafaqz opened this issue 7 months ago • 2 comments

@JamesWrigley it might be good to try converting from xarray DataTree to our DimTree? and also from their Dataset to DimTree?

While DimTree is not covered by semver (now) we can still make changes to it to make sure these conversions can happen.

rafaqz avatar Apr 28 '25 08:04 rafaqz

Yep I think that makes sense, though I haven't actually used either of them 😅

and also from their Dataset to DimTree?

Wouldn't that be from Dataset to DimStack? That's already implemented in the PythonCall extension.

JamesWrigley avatar Apr 28 '25 09:04 JamesWrigley

Yes no pressure! Was just a thought reading the other PR, that we can actually convert more xarray objects now.

And yes it would be nice to have from Dataset directly to a single layered DimTree because it's much lighter on the compiler than converting to a DimStack and then to a DimTree. Say it the Datset has 1000 layers, it may be hundreds of times fast to go to DimTree directly.

DimStack is fast to index and use in inner loops (essentially a free compile-time abstraction) but its demanding on the compiler the first time you make a stack if its not relatively small, DimTree is slow to use (like an xarray object) but inherantly type unstable and doesn't compile for each combination of layers, so thousands of layers compiles the same as 5.

rafaqz avatar Apr 28 '25 09:04 rafaqz