Poetry Installs packages when running container instead of during build

[brittonr]

Describe the bug When creating a python poetry based container using the devenv container command, the poetry dependencies are installed when the container is ran and not when building the container. Additionally when running the container I'm getting this message: /nix/store/mnjksxz8alkwf6pa74a9j04h93p3y2sy-coreutils-9.3/bin/ln: failed to create symbolic link '/root/poetry_test/.venv': No such file or directory Creating virtualenv poetry-test in /.venv Using virtualenv: /.venv To reproduce I have created an example repo here that reproduces the issue. devenv_poetry_container_test

Version

0.6.2

[domenkozar]

This requires a bit of fiddling, but it can be fixed.

[SebastianCallh]

I am experiencing something similar using python and venv. Here's a small devenv.nix that when run with devenv container run test (or copied to the docker image registry and run with docker, which is really what I want to do so I can expose ports, mount volumes etc) causes "Python interpreter changed, rebuilding Python venv..." to appear and tqdm to be reinstalled upon executing the container.

{ ... }:
{
  languages.python = {
    enable = true;
    version = "3.12";
    venv = {
      enable = true;
      requirements = ''
        tqdm
      '';
    };
  };

  containers.test.name = "python-rebuilds-deps";
  processes.test.exec = ''
    python -c 'from tqdm import tqdm; import time; [time.sleep(0.2) for i in tqdm(range(10))]'
  '';
}

This makes it very unappealing to use devenv for building production containers (especially with heavy dependencies like torch), which is unfortunate because I love everything else about it. What kind of fiddling would be required for the python dependencies to be installed only when the container is built?

[SebastianCallh]

I tried using only pkgs.python312Packages and when I do everything works as expected. Of course, this is not as easy for python developers to use as poetry or venv would be.

{ pkgs, config, ... }:
{
  packages = with pkgs; [
    python312
    python312Packages.tqdm
  ];

  containers.processes.copyToRoot = null;
  containers.test = {
    name = "works-as-expected";
    startupCommand = config.processes.test.exec;
  };
  
  processes.test.exec = ''
    ${pkgs.python312}/bin/python -c 'from tqdm import tqdm; import time; [time.sleep(0.2) for i in tqdm(range(10))]'
  '';
}

[ento]

My messy attempt:

• Set the container's copyToRoot to a derivation that copies my app's source and sets up a virtualenv with dependencies installed. This approach necessitates the use of --impure. I think an alternative is to use poetry2nix but haven't tried that path. The image size was bigger than I'd liked (something like 880MB) and I think I'll try again after #1367 is resolved.
• Set entrypoint to a script that runs my app after poetry install --only-root.

simplified devenv.nix

There might be lines that aren't really necessary.

{ pkgs, config, inputs, self, ... }:
{
  languages.python.enable = true;
  languages.python.package = pkgs.python310;
  languages.python.poetry.enable = true;
  languages.python.poetry.activate.enable = true;
  languages.python.poetry.install.enable = !config.container.isBuilding;

  containers.main.name = "main";
  containers.main.startupCommand = "--help";
  containers.main.entrypoint = let
    cfg = config.languages.python;
    bash = "${pkgs.bash}/bin/bash";
    entrypoint = pkgs.writeScript "entrypoint" ''
      #!${bash}

      # Installing the root app during build resulted in a path like
      # /build/ to be written to .pth
      ${cfg.poetry.package}/bin/poetry -C /app install --only-root
      ${cfg.poetry.package}/bin/poetry -C /app run mycli "''${@}"
    '';
  in [ entrypoint ];
  containers.main.copyToRoot =
    let
      cfg = config.languages.python;
      homeDir = "/app";
      nix2container = inputs.nix2container.packages.${pkgs.stdenv.system};
      app = pkgs.stdenv.mkDerivation {
        name = "build-poetry-deps";
        src = self;
        buildPhase = ''
          export XDG_CONFIG_HOME=$TMPDIR/.config
          ${cfg.poetry.package}/bin/poetry config virtualenvs.in-project true --local
          ${cfg.poetry.package}/bin/poetry config virtualenvs.create true --local
          ${cfg.poetry.package}/bin/poetry config cache-dir $TMPDIR/.cache --local
          ${cfg.poetry.package}/bin/poetry install --only main --no-root --no-directory --no-interaction --compile
        '';
        installPhase = ''
          mkdir -p $out/tmp
          mkdir -p $out${homeDir}
          cp -R ./. $out${homeDir}/
        '';
      };
    in (nix2container.nix2container.buildLayer {
      copyToRoot = app;
    });