Install error from bad python3-distutils on Ubuntu 20.04 and Python 3.9

[staticdev]

• [x] I am on the latest Poetry version.

• [x] I have searched the issues of this repo and believe that this is not a duplicate.

• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Ubuntu 20.04.5 (focal)

• Poetry version: 1.2.0

• Link of a Gist with the contents of your pyproject.toml file: D/C

Issue

Running:

$ curl -sSL https://install.python-poetry.org | python3 -
Retrieving Poetry metadata

# Welcome to Poetry!

This will download and install the latest version of Poetry,
a dependency and package manager for Python.

It will add the `poetry` command to Poetry's bin directory, located at:

/home/user/.local/bin

You can uninstall at any time by executing this script with the --uninstall option,
and these changes will be reverted.

Installing Poetry (1.2.0): An error occurred. Removing partial environment.
Poetry installation failed.
See /home/user/poetry-installer-error-017x_k1p.log for error logs.

Traceback on log:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/user/.local/share/pypoetry/venv/lib/python3.8/site-packages/pip/__main__.py", line 29, in <module>
    from pip._internal.cli.main import main as _main
  File "/home/user/.local/share/pypoetry/venv/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 9, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/home/user/.local/share/pypoetry/venv/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/home/user/.local/share/pypoetry/venv/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 8, in <module>
    from pip._internal.cli import cmdoptions
  File "/home/user/.local/share/pypoetry/venv/lib/python3.8/site-packages/pip/_internal/cli/cmdoptions.py", line 24, in <module>
    from pip._internal.cli.parser import ConfigOptionParser
  File "/home/user/.local/share/pypoetry/venv/lib/python3.8/site-packages/pip/_internal/cli/parser.py", line 12, in <module>
    from pip._internal.configuration import Configuration, ConfigurationError
  File "/home/user/.local/share/pypoetry/venv/lib/python3.8/site-packages/pip/_internal/configuration.py", line 26, in <module>
    from pip._internal.utils.logging import getLogger
  File "/home/user/.local/share/pypoetry/venv/lib/python3.8/site-packages/pip/_internal/utils/logging.py", line 29, in <module>
    from pip._internal.utils.misc import ensure_dir
  File "/home/user/.local/share/pypoetry/venv/lib/python3.8/site-packages/pip/_internal/utils/misc.py", line 42, in <module>
    from pip._internal.locations import get_major_minor_version
  File "/home/user/.local/share/pypoetry/venv/lib/python3.8/site-packages/pip/_internal/locations/__init__.py", line 67, in <module>
    from . import _distutils
  File "/home/user/.local/share/pypoetry/venv/lib/python3.8/site-packages/pip/_internal/locations/_distutils.py", line 20, in <module>
    from distutils.cmd import Command as DistutilsCommand
ModuleNotFoundError: No module named 'distutils.cmd'

Traceback:

  File "<stdin>", line 892, in main
  File "<stdin>", line 528, in run
  File "<stdin>", line 549, in install
  File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "<stdin>", line 621, in make_env
  File "<stdin>", line 607, in make_env
  File "<stdin>", line 339, in make
  File "<stdin>", line 362, in pip
  File "<stdin>", line 359, in python
  File "<stdin>", line 352, in run

[staticdev]

With sudo apt-get install python3-distutils it works. Should this be required, explained in docs and/or in the error message?

[neersighted]

I am not of the opinion this is worth documenting, except as maybe a FAQ entry... distutils is a part of the Python standard library, and a Python missing distutils is a defective Python. Distros may do what they want, but I've found it impossible for the Poetry project to stay on top of every way the modify the base install (let alone come up with a mitigation for each one).

We could add a FAQ entry with some common gotchas (missing venv and distutils backtraces) as well as suggestions for where to look (e.g. "look for your distro's packaging of the stdlib venv module and install that"), but it's going to be impossible to maintain detailed instructions for the matrix of Python and Ubuntu versions.

Similarly, I think it will clutter up the install directions too much to add sometimes-necessary workarounds for a couple operating systems -- I don't think it's unfair of Poetry to assume we have a fully functional Python interpreter available with an unmolested standard library.

[staticdev]

I am not of the opinion this is worth documenting, except as maybe a FAQ entry... distutils is a part of the Python standard library, and a Python missing distutils is a defective Python. Distros may do what they want, but I've found it impossible for the Poetry project to stay on top of every way the modify the base install (let alone come up with a mitigation for each one).

We could add a FAQ entry with some common gotchas (missing venv and distutils backtraces) as well as suggestions for where to look (e.g. "look for your distro's packaging of the stdlib venv module and install that"), but it's going to be impossible to maintain detailed instructions for the matrix of Python and Ubuntu versions.

Similarly, I think it will clutter up the install directions too much to add sometimes-necessary workarounds for a couple operating systems -- I don't think it's unfair of Poetry to assume we have a fully functional Python interpreter available with an unmolested standard library.

I agree but users do not expect one of the most used distros in LTS to have a "defective" python. Even if it is not Poetry's fault, maybe it IS worth documenting somewhere.

If you think it is an implicit requirement, then the best place would be after install instructions (https://python-poetry.org/docs/#installation) or in some troubleshooting page.

[neersighted]

I'm leaning towards splitting the FAQ into a high-level/conceptual FAQ, and a technical FAQ (with common troubleshooting/footguns consolidated there)... Maybe troubleshooting would be a good name for a technical FAQ page :laughing:

[a-l-e-c]

Just ran into this issue with my Debian 11. Only recently set it up with VSCodium & Docker Engine and even tried sudo apt install --reinstall python3 which still didn't install distutils... finally just did sudo apt install python3-dev to be sure I no longer have similar issues in the future.

This seems relevant which seems that it should be mentioned for Ubuntu/Debian users

[neersighted]

Just ran into this issue with my Debian 11. Only recently set it up with VSCodium & Docker Engine and even tried sudo apt install --reinstall python3 which still didn't install distutils... finally just did sudo apt install python3-dev to be sure I no longer have similar issues in the future.

This seems relevant which seems that it should be mentioned for Ubuntu/Debian users

I'm not sure how that issue is related, except as a similar situation casued by this mispackaging of Python... They seem to have come to the same conclusion we have.

sudo apt install --reinstall python3-distutils should be the minimal command to fix things on Ubuntu 22.04/Debian Bullseye (11), but sudo apt install python3-dev will pull in everything optional removed from the Python packages, including the C API headers.

[marcstreeter]

Does deprecation of distutils have any effect on this https://peps.python.org/pep-0632/ ?