`nix build` fails for a Python project if `click` dependency is installed from a wheel

[akaihola]

In examples/packages/languages/python-local-development/pyproject.toml, the dependencies are:

dependencies = [
  "requests",
  "click @ git+https://github.com/pallets/click.git@main"
]

This builds just fine, and also this seems to work:

dependencies = [
  "requests",
  "click @ git+https://github.com/pallets/[email protected]"
]

But if I change the click dependency to e.g. "click==8.1.7", "click<8" or just "click", I get this:

error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'nix-shell'
         whose name attribute is located at /nix/store/rhg90jpryc286xn9xjy6qjiaap6pjgdc-source/pkgs/stdenv/generic/make-derivation.nix:331:7

       … while evaluating attribute 'buildInputs' of derivation 'nix-shell'

         at /nix/store/rhg90jpryc286xn9xjy6qjiaap6pjgdc-source/pkgs/stdenv/generic/make-derivation.nix:378:7:

          377|       depsHostHost                = elemAt (elemAt dependencies 1) 0;
          378|       buildInputs                 = elemAt (elemAt dependencies 1) 1;
             |       ^
          379|       depsTargetTarget            = elemAt (elemAt dependencies 2) 0;

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: assertion '((pyproject != null) -> (format == null))' failed

       at /nix/store/rhg90jpryc286xn9xjy6qjiaap6pjgdc-source/pkgs/development/interpreters/python/mk-python-derivation.nix:156:1:

          155|
          156| assert (pyproject != null) -> (format == null);
             | ^
          157|

Other dependencies from PyPI that I have tried seem to work fine with and without a version specifier.

[DavHau]

I think that issue was fixed by @phaer recently. Can you update your dream2nix version an check again?

[akaihola]

I think that issue was fixed by @phaer recently.

I guess you mean the fix in PR #1001? I actually was already running a version from two days after the fix. Updating to today's main didn't seem to help either:

dream2nix/examples/packages/languages/python-local-development/flake.nix

{
  description = "Dream2nix example flake";
  inputs = {
    dream2nix.url = "github:nix-community/dream2nix/main";
    nixpkgs.follows = "dream2nix/nixpkgs";
  };
  # ...
}

dream2nix/examples/packages/languages/python-local-development/pyproject.toml

# ...
dependencies = [
  "requests",
  "click"
]
# ...

$ nix run '.#default.lock' && nix develop

warning: Git tree '/home/akaihola/dream2nix' is dirty
warning: updating lock file '/home/akaihola/dream2nix/examples/packages/languages/python-local-development/flake.lock':
• Updated input 'dream2nix':
    'github:nix-community/dream2nix/1b5e01219a32324c8f6889fe1f4db933ec7932f6' (2024-06-29)
  → 'github:nix-community/dream2nix/46b6ee6b0bf6e601a0765bffbbd5af9006caa2dc' (2024-07-04)
warning: Git tree '/home/akaihola/repos/os/nixos/dream2nix' is dirty
Collecting pip==23.1
  Using cached pip-23.1-py3-none-any.whl.metadata (4.1 kB)
Collecting wheel==0.40.0
  Using cached wheel-0.40.0-py3-none-any.whl.metadata (2.1 kB)
Using cached pip-23.1-py3-none-any.whl (2.1 MB)
Using cached wheel-0.40.0-py3-none-any.whl (64 kB)
Installing collected packages: wheel, pip
  Attempting uninstall: pip
    Found existing installation: pip 24.0
    Uninstalling pip-24.0:
      Successfully uninstalled pip-24.0
Successfully installed pip-23.1 wheel-0.40.0

[notice] A new release of pip is available: 23.1 -> 24.1.1
[notice] To update, run: python -m pip install --upgrade pip
Collecting pdm-backend
  Using cached pdm_backend-2.3.1-py3-none-any.whl (104 kB)
Collecting requests
  Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Collecting click
  Using cached click-8.1.7-py3-none-any.whl (97 kB)
Collecting charset-normalizer<4,>=2 (from requests)
  Using cached charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Collecting idna<4,>=2.5 (from requests)
  Using cached idna-3.7-py3-none-any.whl (66 kB)
Collecting urllib3<3,>=1.21.1 (from requests)
  Using cached urllib3-2.2.2-py3-none-any.whl (121 kB)
Collecting certifi>=2017.4.17 (from requests)
  Using cached certifi-2024.7.4-py3-none-any.whl (162 kB)
Would install certifi-2024.7.4 charset-normalizer-3.3.2 click-8.1.7 idna-3.7 pdm-backend-2.3.1 requests-2.32.3 urllib3-2.2.2
WARNING: There was an error checking the latest version of pip.
lock file written to /home/akaihola/dream2nix/examples/packages/languages/python-local-development/lock.json
Add this file to git if flakes is used.
warning: Git tree '/home/akaihola/dream2nix' is dirty
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'nix-shell'
         whose name attribute is located at /nix/store/rhg90jpryc286xn9xjy6qjiaap6pjgdc-source/pkgs/stdenv/generic/make-derivation.nix:331:7

       … while evaluating attribute 'buildInputs' of derivation 'nix-shell'

         at /nix/store/rhg90jpryc286xn9xjy6qjiaap6pjgdc-source/pkgs/stdenv/generic/make-derivation.nix:378:7:

          377|       depsHostHost                = elemAt (elemAt dependencies 1) 0;
          378|       buildInputs                 = elemAt (elemAt dependencies 1) 1;
             |       ^
          379|       depsTargetTarget            = elemAt (elemAt dependencies 2) 0;

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: assertion '((pyproject != null) -> (format == null))' failed

       at /nix/store/rhg90jpryc286xn9xjy6qjiaap6pjgdc-source/pkgs/development/interpreters/python/mk-python-derivation.nix:156:1:

          155|
          156| assert (pyproject != null) -> (format == null);
             | ^
          157|

[phaer]

$ nix run '.#default.lock' && nix develop

I think using && here isn't ideal. Most of the log comes from nix run .#default.lock which succeeds and correctly locks the latest version of click.

The error is triggered by the second command, while trying to evaluate the devshell. We probably could use an assertion to trigger better errors here or at least include it in the docs but the reason is:

overrides.click.buildPythonPackage.pyproject = true; in default.nix, if you remove that line it should work. This is because using a VCS url like in the example builds from a git clone which includes a pyproject.toml. Using a released version like you do, locks a .whl feel from pypi which does not include a pyproject.toml (and format = "wheel" in the buildPythonpackage arguments)

[goyalyashpal]

updates:

2. this issue seems to be there no matter how many individual deps are removed. it proceeds to completion only after emptying the project.dependencies in pyproject.toml file
3. the issue persists even after removing "cs50", from the pyproject.toml's project.dependencies

expand for error console log

$ nix develop   # or, nix run
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'nix-shell'
         whose name attribute is located at /nix/store/HASH...-source/pkgs/stdenv/generic/make-derivation.nix:336:7

       … while evaluating attribute 'buildInputs' of derivation 'nix-shell'

         at /nix/store/HASH...-source/pkgs/stdenv/generic/make-derivation.nix:383:7:

          382|       depsHostHost                = elemAt (elemAt dependencies 1) 0;
          383|       buildInputs                 = elemAt (elemAt dependencies 1) 1;
             |       ^
          384|       depsTargetTarget            = elemAt (elemAt dependencies 2) 0;

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: assertion '((pyproject != null) -> (format == null))' failed

       at /nix/store/HASH...-source/pkgs/development/interpreters/python/mk-python-derivation.nix:156:1:

          155|
          156| assert (pyproject != null) -> (format == null);
             | ^
          157|

---

I am facing exact same issue. i am using a custom override package, defined in-part as follows:

# cs50.nix
buildPythonPackage rec {
    pname = "cs50";
    version = "9.4.0";

    src = fetchPypi {
        inherit pname version;
        hash = "sha256-1KGHbo+pVwA+vw2I6aha9s53USO/3tcUgMF+yPv8YSk=";
    };

and trying to use a python package not in nixpkgs as d2nix-local-override:

# my_d2_package.nix

{ config, lib, dream2nix, ... }:
...
{
    ...
    deps = {nixpkgs, ...}: {
        # python = nixpkgs.python3;
        python = nixpkgs.python3.override {
            # self = python;
            packageOverrides = pyfinal: pyprev: {
                cs50 = pyfinal.callPackage ./cs50.nix {};
            };
        };
    };

also, maybe related: https://github.com/nix-community/home-manager/issues/4679