thoth-station
[RFE] support installing into a target venv
Is your feature request related to a problem? Please describe.
I'm using micropipenv in a multi-stage container build process where the first 'build' stage creates a Python virtual environment directory, which is copied into the final stage.
I'm currently relying on what feels like trick/hack in order to have the system-wide micropipenv install into my target virtual environment:
RUN python3 -m pip install micropipenv
RUN python3 -m venv /opt/app-root/venv \
&& source /opt/app-root/venv/bin/activate \
&& /usr/bin/python3 -m micropipenv install --deploy
... which works fine, but I wonder whether this is the best way to do this and/or whether it might break in the future.
Describe the solution you'd like It would be nice if I could do this:
RUN python3 -m micropipenv install --deploy --target-venv=/opt/app-root/venv
It would be super nice if that would work on a target virtual environment that was created --without-pip, so that my target venv doesn't need pip and its dependencies. But that's probably a lot more work.
Describe alternatives you've considered I think the hack I stumbled upon is logically the same as doing this:
RUN \
PATH=/opt/app-root/venv/bin \
/usr/bin/python3 -m pipenv install --deploy
... which works because micropipenv invokes whichever pip is first on the PATH, and the virtual environment's pip installs into the virtual environment. So maybe I should just switch to doing that.
Thank you for the report. Micropipenv depends heavily on pip so it's only a matter of how you make micropipenv use the right pip in the virtual environment you want or how you manage to configure pip to do what you want. Your solutions are correct.
Another possible solution from the top of my head might be this: Create a virtual environment without pip and then set PIP_PYTHON (doc) to the path of the Python interpreter in the virtual environment. That way, when micropipenv runs system pip, the system pip should install packages inside the virtual environment.
Let me know if that works for you and we can document it.
I gave PIP_PYTHON a go and it works, thanks!
(Grumble, I looked for this years ago, and tried --root, --prefix and --target before giving up and assuming that Pip always installed into its own Python environment. Now I know that the option I was looking for all along was --python!)
Here's what I came up with:
ARG PYTHON_SUFFIX=3.11
COPY pyproject.toml poetry.lock .
# Build the app's wheel.
COPY src src
RUN python${PYTHON_SUFFIX} -m build -w -v
# Create the runtime virtual environment for the app.
#
RUN \
python${PYTHON_SUFFIX} -m venv \
--without-pip \
/opt/app-root/venv
# Cause subsequent pip invocations to install into the runtime virtual
# environment.
#
ENV PIP_PYTHON=/opt/app-root/venv/bin/python
# Install dependencies and the app's built wheel.
RUN \
MICROPIPENV_PIP_BIN=pip${PYTHON_SUFFIX} \
python${PYTHON_SUFFIX} -m micropipenv \
install --deploy
RUN python${PYTHON_SUFFIX} -m pip install --no-deps dist/*.whl
I'm using this to build & deploy a project that uses Poetry together with the dependencies from the lockfile, which is why I'm both building & installing a wheel and installing the dependencies in two separate steps.
Sure.
I was trying to show examples of this working using the --python= option to pip install, however there's a problem with the order of the command line arguments that micropipenv constructs:
$ micropipenv install -- --python=/tmp/venv91/bin/python
[...]
ERROR: The --python option must be placed before the pip subcommand name
Failed to install requirements, it's highly recommended to use a lock file to to make sure correct dependencies are installed in the correct order
This is because micropipenv constructs the following command line:
pip install -r requirements.txt --disable-pip-version-check --python=/tmp/venv91/bin/python
... but pip wants --python to come before install. Grr.
So I'll just document the use of the PYTHON_PIP environment variable for the time being...