fix: get poetry2nix working again with recent nixpkgs

[cpcloud]

[cpcloud]

This has turned into an odyssey.

[JaniM]

"We do it not because it's easy, but because we thought it'd be easy"

[cpcloud]

I'm currently trying to understand whether it's even possible to use cython3 in any poetry2nix environment that has a transitive dependency on pyyaml (see https://github.com/yaml/pyyaml/issues/601). I might push that investigation to another day/PR.

[cpcloud]

Default checking of runtime deps, while I can see the appeal, has made it more difficult to work in the twilight zone that is Python packaging :(

[francocalvo]

Hey there! I'm using poetry2nix and I'm affected by the issue. I can't build an app which has pyyaml. Is there any workaround for a bit until this is fixed?

[cpcloud]

Hey there! I'm using poetry2nix and I'm affected by the issue. I can't build an app which has pyyaml. Is there any workaround for a bit until this is fixed?

Not that I know of.

[cpcloud]

@K900 would you mind giving this a review?

[cpcloud]

Building locally with an aarch64-darwin machine via rosetta to work out any x86_64 kinks.

[cpcloud]

Ok, the aggregate darwin build is now passing locally for me. As long as it eventually becomes green here, I'll merge this PR in.

[jugendhacker]

If I get it right, this PR should fix building pyyaml. But somehow I still get the following error while building something depending on pyyaml (through apprise):

error: attribute 'cython_0' missing

       at /nix/store/vv0w2xpfxrqm48qwbvxqla6j1zvi10a4-source/overrides/default.nix:147:6:

          146|
          147|     {
             |      ^
          148|       addBuildSystem = attr: drv: addBuildSystem' { inherit self drv attr; };
       Did you mean one of cython_3 or cython?

[cpcloud]

You have to update your nixpkgs. There was a rename of cython < 3 to cython that makes it infeasible to support versions of nixpkgs prior to the rename.

[jugendhacker]

Oh okay, I already thought about it, but replaced it in the wrong place. I needed to build the whole nixosSystem inside my flake from recent/unstable nixpkgs

[EBADBEEF]

I also got the error: attribute 'cython_0' missing.

Cython is a static compiler for python. They went from version number 0.29.x to 3.0 recently (https://pypi.org/project/Cython/#history). In nixpkgs, the earlier default version was '0.29.x', now the default version is '3.0.x'. The commit is here: https://github.com/NixOS/nixpkgs/commit/528354e66c26cf947b04cc94db7185f5d127ca31

I was able to get my build to work on nixos-23.11 by adding to my overrides, like:

inherit (poetry2nix) mkPoetryEnv defaultPoetryOverrides;
cythonOverrides = (final: prev: {
  cython_0 = null;
});
blah = mkPoetryEnv {
  ...
  overrides = [ defaultPoetryOverrides cythonOverrides ];
};

I guess none of my packages really depend on cython ;-)! I also experimented a bit and tried to swap the values in the overlay but that leads to infinite recursion.

I also tried using the external nixpkgs python.pkgs.cython in the overlay and that worked even though that seems like it might cause trouble (but at least one package built like this, heh):

(final: prev: {
  cython_0 = pkgs.python311.pkgs.cython;
  cython_3 = pkgs.python311.pkgs.cython_3;
})

I do wonder how this interacts with existing nixpkgs pythonPackages. Haven't really understood how poetry2nix approaches it. I see all the normal nixpkgs pythonPackages are available in the mkPoetryEnv. But if I do have a package in my poetry.lock it seems like it totally overrides any existing pythonPackages entry (which is what I want and would expect).