readthedocs-docker-images icon indicating copy to clipboard operation
readthedocs-docker-images copied to clipboard

Include `pipx` with proper configuration

Open huxuan opened this issue 1 year ago • 7 comments

Though we already have asdf, it might be better to also have pipx so that we can install Python Applications directly from PyPI when needed.

Current workaround in .readthedocs.yaml:

      - asdf plugin add pipx
      - asdf install pipx latest
      - asdf global pipx latest
      - PIPX_BIN_DIR=$READTHEDOCS_VIRTUALENV_PATH/bin pipx install <package>

The mostly inconvenience is that we have to specify the PIPX_BIN_DIR to make it actually work. It would be great if we can have it properly configured just like asdf.

huxuan avatar Jun 23 '24 04:06 huxuan

Hi @huxuan! Thanks for opening this issue. Can you expand a little more on this use case? In particular, what's the different between running pipx and just the regular pip to install packages?

humitos avatar Jun 25 '24 07:06 humitos

Can you expand a little more on this use case? In particular, what's the different between running pipx and just the regular pip to install packages?

Generally if possible, I would like to install those standalone tools in isolated virtual environments. And currently, there are at least three of them:

  1. pdm: manage and install plugins and theme for building documentation
  2. mypy: generate mypy report and include them in the generated documentation in HTML format
  3. coverage: same as mypy but generating coverage report

All of these tools have nothing to do with building documentation, and installing them with pip will introduce a lot of redundant dependencies and increase the possibility of dependency conflicts and build failures.

huxuan avatar Jun 25 '24 09:06 huxuan

Looks like you can also install pipx from apt https://pipx.pypa.io/latest/installation/ (https://docs.readthedocs.io/en/stable/config-file/v2.html#build-apt-packages).

stsewd avatar Jun 25 '24 15:06 stsewd

Looks like you can also install pipx from apt https://pipx.pypa.io/latest/installation/ (https://docs.readthedocs.io/en/stable/config-file/v2.html#build-apt-packages).

Wow, that is cool! I did not notice that configuration. Even though that does not ensure to use the latest version of pipx, it should be a better solution than the workaround I mentioned in the issue description. Thanks!

huxuan avatar Jun 26 '24 01:06 huxuan

Hi, sorry for heads up again.

TL;DR;

Can we add /home/docs/.local/bin in PATH here? https://github.com/readthedocs/readthedocs-docker-images/blob/4d356fa1586f74bd39c56c4789f383686f1d7ed9/Dockerfile#L137

I can send a pull request if it is OK.


I tried to use the build.apt_packages, it still suffers on the following PATH issue. And you can refer to the full build log here: https://readthedocs.org/projects/ss-python/builds/24818740/

⚠️ Note: '/home/docs/.local/bin' is not on your PATH environment variable.

One solution could be running pipx ensurepath and source ~/.bashrc explicitly. Otherwise, we have to specify the PIPX_BIN_DIR when installing packages, do you have any ideas about better solutions?

The currently workaround I use is as following which is already better than the previous version. :-)

build:
  apt_packages:
    - pipx
  jobs:
    pre_create_environment:
      - PIPX_BIN_DIR=$READTHEDOCS_VIRTUALENV_PATH/bin pipx install mypy[reports]==1.10.1
      - PIPX_BIN_DIR=$READTHEDOCS_VIRTUALENV_PATH/bin pipx install pdm==2.15.4

huxuan avatar Jun 26 '24 11:06 huxuan

@huxuan can you try running asdf reshim python after asdf global pipx latest? I think that should remove the need for defining PIPX_BIN_DIR environment variable.

humitos avatar Jun 27 '24 09:06 humitos

@huxuan can you try running asdf reshim python after asdf global pipx latest? I think that should remove the need for defining PIPX_BIN_DIR environment variable.

Currently, I want to avoid using asdf, the build.apt_packages works quite well except for the PATH issue.

huxuan avatar Jun 27 '24 09:06 huxuan