Test failures as of 0.11.4 on NixOS

[samuela]

Please make sure these conditions are met

• [x] I have checked that this issue has not already been reported.
• [x] I have confirmed this bug exists on the latest version of anndata.
• [ ] (optional) I have confirmed this bug exists on the master branch of anndata.

Report

I'm seeing a number of test failures as of the latest version, 0.11.4,

=========================== short test summary info ============================
FAILED tests/test_concatenate.py::test_concat_dask_sparse_matches_memory[outer-only] - AssertionError:
FAILED tests/test_concatenate.py::test_concat_dask_sparse_matches_memory[outer-first] - AssertionError:
FAILED tests/test_concatenate.py::test_concat_dask_sparse_matches_memory[outer-same] - AssertionError:
FAILED tests/test_concatenate.py::test_concat_dask_sparse_matches_memory[outer-unique] - AssertionError:
FAILED tests/test_concatenate.py::test_concat_dask_sparse_matches_memory[outer-None] - AssertionError:
= 5 failed, 3265 passed, 23 skipped, 74 xfailed, 324 warnings in 89.90s (0:01:29) =

Full logs available here: https://gist.github.com/samuela/cb1e21408c78ff89726fcb2caf213532

Versions

n/a

[ilan-gold]

Hi @samuela thank you for the issue. Please fill out the Versions section because I cannot reproduce your issue and would very much so like to be able to! That information would include:

1. Dependencies
2. Python version
3. Your system information
4. Commit hash of anndata

Also, perhaps some context on why you're running the 0.11.x tests might help as well, perhaps there is something about your environment we are not capturing here.

[samuela]

Hi @ilan-gold , happy to fill in some more details! This is on version 0.11.4, the latest at the time of writing, currently 361325fc621887bf4f381e9412b150fcff599ff7. Python version is 3.12.11. I am running NixOS on an x86_64 machine.

For context these issues were uncovered in pursuit of packaging anndata in Nixpkgs (https://github.com/NixOS/nixpkgs/pull/418562). You can reproduce by cloning nixpkgs, checking out the linked PR, and running nix-build -A python3Packages.anndata.

[ilan-gold]

@samuela I don't think I can run nixos. If there were some way to get around this, I would test (i.e., you have some instructions or similar for how I can test this on non-nixos).

Short of that, if the OS can be recognized, we would accept a PR to add a xfail for nixos to that test.

[samuela]

It should be possible to reproduce these failures on a non-NixOS operating system by using the Nix package manager, but I have not tried this myself yet.

[ilan-gold]

Hmmm, what are the build instructions for the package manager then? The directions you gave above didn't work (nix-build -A python3Packages.anndata presumably requires some sort of build step since it didn't "just work"?):

nixpkgs on  master via 🐍 v3.12.3 (venv) on ☁️   [email protected]
❯ gh pr checkout 418562
Updating files: 100% (3462/3462), done.
branch 'samuela/scanpy' set up to track 'origin/samuela/scanpy'.
Switched to a new branch 'samuela/scanpy'


A new release of gh is available: 2.55.0 → 2.74.2
To upgrade, run: brew upgrade gh
https://github.com/cli/cli/releases/tag/v2.74.2

(base)
nixpkgs on  samuela/scanpy via 🐍 v3.12.3 (venv) on ☁️   [email protected] took 6s
❯ nix-build -A python3Packages.anndata
zsh: command not found: nix-build

I also looked at the CONTRIBUTING.md but didn't see anything.

[ilan-gold]

Separately I noticed you skipped a bunch of cupy tests: https://github.com/NixOS/nixpkgs/pull/418562/files#diff-37f218763d6e4b82c1cd424b93b2c85840540a85f595523447c98e8a4df5d69cR115-R136

But the ones that require cupy are skipped by our pytest setup on environments that don't install cupy. Is this behavior also not working?

[samuela]

❯ nix-build -A python3Packages.anndata zsh: command not found: nix-build

It looks like the nix tools are not in PATH. What installation steps did you follow? Sometimes it requires opening a new terminal session for all of the env vars to kick in.

Printing the version info should at least work:

skainswo@skainswo:~/nixpkgs$ nix-build --version
nix-build (Nix) 2.24.12

But the ones that require cupy are skipped by our pytest setup on environments that don't install cupy. Is this behavior also not working?

Ah interesting, IIRC I was seeing some errors attempting to import cupy in these tests. I did not realize they should be auto-skipping.

[ilan-gold]

@samuela I followed your instructions and when that didn't work looked at CONTRIBUTING.md. Going back to https://github.com/scverse/anndata/issues/2017#issuecomment-2997520250 and

It looks like the nix tools are not in PATH

How would I get nix tools in my path by doing "cloning nixpkgs, checking out the linked PR, and running nix-build -A python3Packages.anndata"? It seems like there's a step missing (make or export PATH=./some/path/to/some/file or something similar?). I'm sorry if I'm being dumb but I have never used nix and don't really know much about it.

[samuela]

Apologies for the confusion! Prerequisite to the instructions is installing nix itself: https://nixos.org/download/.

Nix is the build tool itself and Nixpkgs is the package collection, analogous to pip and PyPI.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. Please add a comment if you want to keep the issue open. Thank you for your contributions!

[flying-sheep]

I think it’s worth is to investigate this, maybe it can help us get a lid on imprecision.

Also I really don’t like rtol and atol. The testing output makes no sense to me, because e.g. calling the function with atol=0 can still succeed, yet if it fails, both atol=0 and the arrays’ absolute tolerance (very much not 0) will be reported.

I think the newer functions like assert_array_almost_equal_nulp are more intuitive to understand what’s going on.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. Please add a comment if you want to keep the issue open. Thank you for your contributions!