[WIP] Support for scipy.sparse.{csr,csc}_array
Starting support for scipy.sparse csc_array and csr_array. These are the new array-like interfaces for sparse matrices from scipy.
Fixes #746, #859
Still very WIP
TODO:
- [ ] Figure out compat with scipy<1.11
- [ ] Demo of how to load an array from disk
- [ ] Support for
.X - [ ] Tests (figure out which ones need adding)
- [ ] Fix concatenation tests
- [ ] Fix view tests
- [ ] Test for indices dtype after concatenation (e.g. #859)
- [ ] Release note
Codecov Report
Attention: Patch coverage is 80.00000% with 25 lines in your changes are missing coverage. Please review.
Project coverage is 83.89%. Comparing base (
0a768fc) to head (10ad4c9).
Additional details and impacted files
@@ Coverage Diff @@
## main #1028 +/- ##
==========================================
- Coverage 84.18% 83.89% -0.29%
==========================================
Files 35 35
Lines 5607 5675 +68
==========================================
+ Hits 4720 4761 +41
- Misses 887 914 +27
| Files | Coverage Δ | |
|---|---|---|
| src/anndata/_core/index.py | 93.24% <100.00%> (+0.04%) |
:arrow_up: |
| src/anndata/_core/views.py | 84.93% <100.00%> (+0.87%) |
:arrow_up: |
| src/anndata/_io/specs/methods.py | 88.00% <ø> (ø) |
|
| src/anndata/utils.py | 84.12% <100.00%> (ø) |
|
| src/anndata/_core/anndata.py | 84.71% <81.81%> (-0.12%) |
:arrow_down: |
| src/anndata/_core/sparse_dataset.py | 94.09% <85.71%> (-0.63%) |
:arrow_down: |
| src/anndata/compat/__init__.py | 81.73% <75.00%> (-1.60%) |
:arrow_down: |
| src/anndata/_core/merge.py | 82.72% <82.60%> (-0.20%) |
:arrow_down: |
| src/anndata/tests/helpers.py | 83.81% <63.15%> (-2.36%) |
:arrow_down: |
- [x] backed sparse array (instead of matrix)
Ok @ivirshup I think this is read for a review. This PR should only focus on the use of the object in the AnnData class and not on reading. That will come next, but here there is writing functionality (which made sense to me to put here)..
The test failures are from "FAILED anndata/tests/test_backed_sparse.py::test_append_overflow_check[csr_array-zarr] - numpy.core._exceptions._ArrayMemoryError: Unable to allocate 16.0 GiB for a..." - we could xfail these as well, but it seems like since this works for matrix, maybe there's a bug here?
sorry for the force push, VS code messed things up and I simply undid the mess to get a clean merge commit.
No worries @flying-sheep - much appreciated!
Demo of how to load an array from disk
Probably belongs in the follow up PR given that we did not implement that functionality here