Tom White

I agree that deferring some of the imports could help. I think numba compilation is still a significant part of this, see #363, although we now have numba caching on...

> I also wonder if PEP484 forward references might help with delaying imports: https://legacy.python.org/dev/peps/pep-0484/#forward-references It would be great if we could take advantage of forward references. Do you think xarray...

Pandas is not used very much in the codebase, so it might be possible to import it lazily. Similarly, the distance API is pretty niche so making that lazier would...

There are some docs in the Array API tests README: https://github.com/data-apis/array-api-tests/blob/master/README.md, but not much more than that I'm afraid. I saw the issue you are hitting in https://github.com/cubed-dev/cubed/commit/a4020f35c78dcfeb232c5e09bc78c5b6f8afcc4e, and fixed...

I tracked down the problem - this seems to fix it: https://github.com/pydata/xarray/pull/9326

The Xarray issue has been merged now so it might be worth seeing if it fixes your original issue @norlandrhagen.

Great! > Is it worth pinning `xarray-cubed` to and above the next release of Xarray? That might be a good idea.

This should be done at the Zarr-level, but concatenating >100 stores is not something we have tried yet. The work I've done on Cubed might be helpful here: you could...

Another approach is free-threaded Python, which is just ramping up (but is going to take some time): * [Free-threaded CPython is ready to experiment with!](https://labs.quansight.org/blog/free-threaded-python-rollout) * [py-free-threading.github.io](https://py-free-threading.github.io/)

> FWIW, I believe any ML framework will have to have special cases for randomness, given the constraints of hardware (GPUs/TPUs). I agree. That's why random number generation is not...