Severin Dicks
Severin Dicks
This won't work for large scale datasets, eg. for Lake Tahoe each batch is 10Million cells. This will lead to a OOM error
@MPebworthEpana Your approach technically works, but it's not ideal. Rechunking and sorting the array seem to trigger persistence, which causes the entire dataset to be loaded into memory. On top...
Please try setting 'init_pos="random"'. This usually works for me.
I don't know actually. I have ran into this too for very large datasets (1M or more). I will forward this to the rapids team at NVIDIA.
I updated the title so people know how to fix it. I'll leave this open for know. I'll also update UMAP so that I has a new default with auto...
There will be a further update with rapids-24.10. I hope this will completely eliminate the problem.
https://github.com/scverse/rapids_singlecell/pull/124 I also have some issues testing for cuda-12
This also doesn't work for F-Arrays
I'm currently testing `kvikio` with `zarr` and we need to very careful with the compression. Some compressions break kvikio. We can only use `numcodecs.LZ4()` or no compression. I would add...
Reminder: Check Benchmarks