pdm: missing `editables` module when entering devShell

[abathur]

I couldn't find other reports of this, so maybe I'm just holding it wrong, but after switching to the pdm module I started getting this traceback when I enter a devShell:

Traceback (most recent call last):
  File "/nix/store/kjg1yk7k0fflpzjnnpy745rb0haacii7-editable.py", line 229, in <module>
    make_editable(
  File "/nix/store/kjg1yk7k0fflpzjnnpy745rb0haacii7-editable.py", line 65, in make_editable
    editable_dist_info = make_dist_info(
  File "/nix/store/kjg1yk7k0fflpzjnnpy745rb0haacii7-editable.py", line 112, in make_dist_info
    dist_info_name = backend.prepare_metadata_for_build_editable(tmp_dir)
  File "/nix/store/nq1m0pmqh4yzncm311dj5pn8bzzn05hh-python3-3.9.19-env/lib/python3.9/site-packages/pdm/backend/__init__.py", line 87, in prepare_metadata_for_build_editable
    from pdm.backend.editable import EditableBuilder
  File "/nix/store/nq1m0pmqh4yzncm311dj5pn8bzzn05hh-python3-3.9.19-env/lib/python3.9/site-packages/pdm/backend/editable.py", line 7, in <module>
    from editables import EditableProject
ModuleNotFoundError: No module named 'editables'

I'm able to work around it by adding editables it to the app's python dependencies.

[JonBoyleCoding]

What do you mean by app's python dependencies - in the pyproject.toml or in the nix derivation somewhere?

[JonBoyleCoding]

What do you mean by app's python dependencies - in the pyproject.toml or in the nix derivation somewhere?

Never mind - I figured it out (at least for me). Place it in the mkDerivation in the module (in the examples in this repo it would be the default.nix file)

[purepani]

Curious, I've never had that happen before. I'll see if I can debug it later.

[abathur]

@JonBoyleCoding Were you just asking me to clarify, or does your response indicate that you've run into the same error?

I really was hoping someone would know what this was and just tell me I was holding it wrong, so we might be able to help run this to ground by finding a minimal repro.

I took a few quick swings at doing so this morning but haven't yet been able to isolate one, and I've got other fires to fight so I may not be able to revisit for a bit.

[JonBoyleCoding]

@JonBoyleCoding Were you just asking me to clarify, or does your response indicate that you've run into the same error?

I really was hoping someone would know what this was and just tell me I was holding it wrong, so we might be able to help run this to ground by finding a minimal repro.

I took a few quick swings at doing so this morning but haven't yet been able to isolate one, and I've got other fires to fight so I may not be able to revisit for a bit.

I encountered the same error and was asking if you could provide specifics on how you got around it. Thankfully I did manage to get around it for now!

[abathur]

Gotcha. I did something a little more targeted, but what you're saying sounds fine for a workaround.

I don't really love some specifics of the Nix half of this and will probably refactor later, but for reference I just added a new optional/dev dependency group to pyproject.toml containing only "editables". This part looks like:

[tool.pdm.dev-dependencies]
# workaround
editables = [
    "editables"
]

Back in flake.nix I appended dependencies from that group to the buildInputs for only the devShell so that editables didn't also leak into the container image. In barebones form, that looked something like:

let 
  get_dev_packages = groups: with groups;
        editables.packages //
        other_group_i_need.packages;
in {
  devShells.default = (pkgs.mkShell rec {
    inputsFrom = [
      packages.blah.devShell
    ];

    buildInputs = with pkgs; [
      # ...
    ] ++ map (x: (pkgs.lib.head (pkgs.lib.attrValues x)).public) (
      pkgs.lib.attrValues (get_dev_packages packages.blah.config.groups)
    );
  });
}

Is your project public, or is it generally fairly small/simple? This may be an easy fix for @purepani if we have a minimal reproduction at hand. My project's private and somewhat large so it'll take me some time to extract something minimal from it.

[purepani]

I ended up getting the error today while working so I can figure it out based on that.

[JonBoyleCoding]

It will be public in a few days, but not yet. However if you've already got the error then presumably you can go forward with that.

[phaer]

@purepani Awesome, if you take a look! I could do as well, if you want and push a reproducer :)

[purepani]

I'll take a look after work today!

[purepani]

Looks like this may have been a bug in pdm-backend: https://github.com/pdm-project/pdm-backend/issues/242. It was fixed in the most recent update from a few weeks ago, so I made a PR in nixpkgs to update the backend: https://github.com/NixOS/nixpkgs/pull/323292 (though I haven't tested it with @phaer should we wait for this to be merged and hit nixpkgs or fix it here while we wait?

[purepani]

Even better looks like it was already commited into staging 5 days ago: https://github.com/NixOS/nixpkgs/pull/320924

[phaer]

So we just have to wait until https://nixpk.gs/pr-tracker.html?pr=320924 hits nixpkgs-unstable and update the inputs then :)

[purepani]

Yep exactly I did test it by adding an overlay to update it, and it did work correctly so just until it gets merged(I wouldnt recommend this since it took a long time to build since it rebuilds a bunch of stuff)

[purepani]

This is now fixed as far as I can tell!