Isaac Virshup

> But obviously propagating the dimension_names in a transparent and consistent way. Doesn't that potentially raise problems already? I think it's reasonable to let people do concatenation like numpy arrays...

> Xarray already has a array wrapper type that does all this except concatentation and metadata handling: https://github.com/pydata/xarray/issues/5081. @dcherian I think the indexing capabilities of LazilyIndexedArray may be too powerful...

> > You still need to actually create the correct json reference file > > if you need a manifest, someone has to make this, and kerchunk already can for...

Hey @djarecka, I've been looking into using kerchunk for this very case! I'd really like to have this as core AnnData functionality this year. But, idk if it's possible right...

For a little more context: [here is the hdf5 representation of sparse arrays being used](https://anndata.readthedocs.io/en/latest/fileformat-prose.html#sparse-arrays). It's not native hdf5, it's a "convention". But it's very similar to what in the...

That also makes sense. Tbh I had gotten started with the pinning while narrowing down the bug.

Thanks for the response. I'm not sure what the specific implementation is, but that example doesn't seem to set the number of threads pyarrow sees. I'll demonstrate: ## Using threadpoolctl...

> An alternative would be to allow custom controllers as requested here `#137`. I believe I prompted that 😆

Great! Thanks @ogrisel and @jeremiedbb! I'm a little unfamiliar with linking, as I've avoided learning much C++, but have given this a shot. It seems to work, but there's something...

Ah, I think I'm starting to see. I think I'm getting a dynlib for all matching files as the expectation is that I am setting the threads directly using the...