Installation fails on MacOS

[WilliamVenner]

trafficstars

Python 3.8 was installed by Command Line Tools (Xcode)

dyld[58557]: Library not loaded: @executable_path/../Python3
  Referenced from: /Users/william.venner/Library/Application Support/pypoetry/venv/bin/Python3
  Reason: tried: '/Users/william.venner/Library/Application Support/pypoetry/venv/bin/../Python3' (no such file), '/usr/local/lib/Python3' (no such file), '/usr/lib/Python3' (no such file)

Traceback:

  File "<stdin>", line 892, in main
  File "<stdin>", line 530, in run

Is this a case sensitivity issue?

MacOS Monterey 12.4 on M1 Pro

For anyone stumbling on this

I found installing poetry from homebrew worked fine.

[ifeeltiredboss]

It's the same for MacOS Monterey 12.3 on Intel. Perhaps some errors in paths?

➜  ~ which python3
/usr/bin/python3

[ifeeltiredboss]

Installing python with homebrew and then restarting the install script works fine. But I think it shouldn't be necessary to install another python before installing poetry.

[neersighted]

Looks like macOS's Python is broken. OS-provided Pythons are always touch and go/partially broken, so this is a low priority, but it would be nice to figure out.

[branchvincent]

The issue here is the macOS-provided python is not relocatable, so venv creates a broken python when using --copies / symlinks=False as our installer does. See https://github.com/python/cpython/issues/82886#issuecomment-1093846287

A simple reproduction:

$ /usr/bin/python3 -m venv --copies --without-pip venv && ./venv/bin/python3
dyld[97998]: Library not loaded: '@executable_path/../Python3'
  Referenced from: '/private/tmp/venv/bin/python3'
  Reason: tried: '/private/tmp/venv/bin/../Python3' (no such file), '/usr/local/lib/Python3' (no such file), '/usr/lib/Python3' (no such file)
fish: Job 1, './venv/bin/python3' terminated by signal SIGABRT (Abort)

Perhaps we should try to detect + handle this case and fallback to symlinks=True

[cjolowicz]

This problem is not limited to Command Line Tools. It applies to any Python on macOS, provided it's built as a framework (--enable-framework). For example, this includes official binaries from python.org and Homebrew. It excludes pyenv which uses the default POSIX build configuration.

From #34 :

The problem appears to be that the installer creates the virtual environment with copies instead of symlinks. This is a good default for Windows, but not for macOS. On macOS, the copied interpreter binary references the Python dynamic library using a path including the full version number. When Homebrew upgrades Python, this path disappears. By contrast, virtual environments created with the default strategy use a symbolic link that remains stable for all releases of a Python feature version (major.minor). Upgrading Python to a maintenance release seamlessly and implicitly upgrades any associated virtual environments.

[cjolowicz]

Looking a bit further into this, there are some differences between these Pythons. According to the CPython issue linked above, Apple's Python uses a relative path to the dynamic library, which always breaks with symlinks=False. Homebrew uses an absolute path, which breaks when you upgrade to a new release of Python. Official binaries from python.org are probably fine (haven't tested this).

In any case, I'd recommend to use symlinks=True except on Windows. This is also the default that the python -m venv command uses.

[windoverwater]

FYI - on my 2020 M1 mini with the latest MacOS, poetry does not install from scratch. FYI - homebrew is installed for other stuff besides python and miniconda is installed for all things python. I am trying to switch a few projects from conda to poetry.

% which python3 /usr/bin/python3 % ls -l /usr/bin/python3 -rwxr-xr-x 76 root wheel 167120 Aug 24 04:59 /usr/bin/python3*

% python3 install-poetry.py
Retrieving Poetry metadata

Welcome to Poetry!

This will download and install the latest version of Poetry, a dependency and package manager for Python.

It will add the poetry command to Poetry's bin directory, located at:

/Users/sandy/.local/bin

You can uninstall at any time by executing this script with the --uninstall option, and these changes will be reverted.

Installing Poetry (1.2.1): Creating environment Traceback (most recent call last): File "/Users/sandy/repos/github.com/install.python-poetry.org/install-poetry.py", line 940, in sys.exit(main()) File "/Users/sandy/repos/github.com/install.python-poetry.org/install-poetry.py", line 919, in main return installer.run() File "/Users/sandy/repos/github.com/install.python-poetry.org/install-poetry.py", line 550, in run self.install(version) File "/Users/sandy/repos/github.com/install.python-poetry.org/install-poetry.py", line 571, in install with self.make_env(version) as env: File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/contextlib.py", line 117, in enter return next(self.gen) File "/Users/sandy/repos/github.com/install.python-poetry.org/install-poetry.py", line 643, in make_env raise e File "/Users/sandy/repos/github.com/install.python-poetry.org/install-poetry.py", line 629, in make_env yield VirtualEnvironment.make(env_path) File "/Users/sandy/repos/github.com/install.python-poetry.org/install-poetry.py", line 309, in make builder = venv.EnvBuilder(clear=True, with_pip=True, symlinks=False) File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/venv/init.py", line 66, in init self.symlinks = should_use_symlinks(symlinks) File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/venv/init.py", line 31, in should_use_symlinks raise Exception("This build of python cannot create venvs without using symlinks") Exception: This build of python cannot create venvs without using symlinks

% git logpp -1 * c8c3ce4 - (HEAD -> main, origin/main, origin/HEAD) (34 hours ago) <pre-commit-ci[bot]> [pre-commit.ci] pre-commit autoupdate % python3 --version Python 3.9.6

[abhinavdayal]

python3 -m venv $VENV_PATH $VENV_PATH/bin/pip install -U pip setuptools $VENV_PATH/bin/pip install poetry

Using these to install poetry works fine. However this must be repeated for each new project.

[neersighted]

I'm not sure what you mean @abhinavdayal -- Poetry should not be installed in the project environment. The manual or pipx install directions on https://python-poetry.org/docs work just fine in the mean time.

[jemshit]

So pipx is preferred way instead of altering macOS's default python ?

[neersighted]

It works and many users may prefer it -- however, it's another tool. It would be nice to fix this issue, but it will take a moderate refactor of the installer to do it in a clean way -- no one has put the time in yet.

[cjuracek]

Hi all, just wanted to see if there's a consensus answer for this problem? I'm trying to help someone get Poetry installed and we're hitting this roadblock. We tried to use a Python installed via Pyenv with no luck. Any help appreciated!

[neersighted]

If you're using a pyenv Python you are hitting something else as pyenv does not set --enable-framework by default. I suggest reaching out on Discussions or Discord for support, or creating a detailed issue report if you've troubleshot things sufficiently and think you're hitting a new bug.

[cjuracek]

@neersighted This may be internal relating to where we're pointing pip to install packages. I'll report back

Update: Problem solved - responsibility 50-50:

• When using a Pyenv Python: Our bad - it's related to our default pip.conf
• When using OS Python: Seems to be a script related problem. I'd encourage anyway who encounters this to try out Pyenv (which may fix it). It's a great tool to manage multiple Python verisons.

[neersighted]

The reason the OS Python fails for you @cjuracek is this issue; --copies does not work with framework (non-relocatable) builds of Python.

[urbanonymous]

So, just use brew right?

[neersighted]

We provide no support for Homebrew installations; feel free to use Homebrew, but please don't open issues against Poetry unless you have reproduced on an install using this script/pipx/pip+venv. pipx and pip+venv work just fine with framework-based Pythons; #79 complicates the fix I had in mind for this issue, and I will need to spend some time reworking it with the new information in mind (or, we may just drop --copies entirely).

[EddyDavies]

For anyone struggling with this error using pyenv I just needed to add eval "$(pyenv init --path)" to my ~/.zshrc (or ~/.bashrc and then it worked

[ralexx]

It would be nice to fix this issue, but it will take a moderate refactor of the installer to do it in a clean way -- no one has put the time in yet.

I modified the install script so it will accept an argument --use-system-python that calls venv.Builder(..., symlinks=True) only on macOS where the Python executable path starts with "/Applications/Xcode.app". The modified script installs Poetry 1.3.2 successfully on my macOS 13.2 where the system Python is 3.9.6., and reproduces the Exception("This build of python cannot create venvs without using symlinks") if the new argument is omitted.

This doesn't attempt to address #79 or even a more generalized macOS use case (personally I'm going for pyenv for any/every other use case) but I'm happy to submit a PR if @python-poetry/triage wants one.

[ikebo]

In a conclusion: 1, download the install script, curl -sSL https://install.python-poetry.org > poetry_install.py 2, change the symlink to True on line 317 3, python3 poetry_install.py

that's it

[shekharidentv]

In a conclusion: 1, download the install script, curl -sSL https://install.python-poetry.org > poetry_install.py 2, change the symlink to True on line 317 3, python3 poetry_install.py

that's it

it installed the poetry but the path of poetry is hidden. `~ ❯ python3 poetry_install.py Retrieving Poetry metadata

The latest version (1.5.1) is already installed. ~ ❯ poetry --version zsh: command not found: poetry`

[b0le]

Looking a bit further into this, there are some differences between these Pythons. According to the CPython issue linked above, Apple's Python uses a relative path to the dynamic library, which always breaks with symlinks=False. Homebrew uses an absolute path, which breaks when you upgrade to a new release of Python.

@cjolowicz I can actually confirm that Poetry installation script fails when using Python 3.11.4 installed by the latest Homebrew's [email protected] formula (version 3.11.4_1 as of 2023-08-05), i.e. it's not just the upgrade that breaks due to version numbers being used in paths but a clean install of Poetry seems to be failing as well.

The Poetry install scripts generates the following pyvenv.cfg:

% cat ~/Library/Application\ Support/pypoetry/venv/pyvenv.cfg
home = /opt/homebrew/opt/[email protected]/bin
include-system-site-packages = false
version = 3.11.4
executable = /opt/homebrew/Cellar/[email protected]/3.11.4_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11
command = /opt/homebrew/opt/[email protected]/bin/python3.11 -m venv --copies --clear /Users/b0le/Library/Application Support/pypoetry/venv

Using otool to check for shared library used by the python3.11 binary referred to above shows a relative path ../Python being used similar to what was observed with the Python version shipped with macOS:

% otool -L /opt/homebrew/Cellar/[email protected]/3.11.4_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11
/opt/homebrew/Cellar/[email protected]/3.11.4_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11:
	@loader_path/../Python (compatibility version 3.11.0, current version 3.11.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)

Interesting enough I was previously able to install Poetry 1.4.2 back in April 2023 using the Python interpreter installed by Homebrew's [email protected] formula version 3.11.3. As the behavior of install-poetry.py hasn't changed in regards to venv creation I am assuming that the Homebrew Python interpreter has become non-relocatable since 3.11.3.

@neersighted commented the following on August 24, 2022:

The reason the OS Python fails for you @cjuracek is this issue; --copies does not work with framework (non-relocatable) builds of Python.

Poetry installation now seems to be broken with both the system default Python and the Homebrew version as well. As expected symlinks=True solves the issue and installation proceeds as expected. Using symlinks for venv creation really should become the default behavior on macOS.

[ckabalan]

Macbook M1 on Ventura with Homebrew-installed Python 3.11. Follow the instructions in @ikebo 's comment for a fix.

I fought with this for like 4 hours over two days, causing me to literally uninstall homebrew, all formulae, and reinstall them all chasing this dyld error. All to no avail, until I found @ralexx 's comment and it worked immediately.

Seems like this should be simple enough to catch the error and re-try with symlink=True if there is not a more suitable solution.

[adrianrfreedman]

In a conclusion: 1, download the install script, curl -sSL https://install.python-poetry.org > poetry_install.py 2, change the symlink to True on line 317 3, python3 poetry_install.py that's it

it installed the poetry but the path of poetry is hidden. `~ ❯ python3 poetry_install.py Retrieving Poetry metadata

The latest version (1.5.1) is already installed. ~ ❯ poetry --version zsh: command not found: poetry`

➜  ~ python3 poetry_install.py 
Retrieving Poetry metadata

# Welcome to Poetry!
...
Installing Poetry (1.5.1): Installing Poetry
Installing Poetry (1.5.1): Done

Poetry (1.5.1) is installed now. Great!

To get started you need Poetry's bin directory (/Users/bob/.local/bin) in your `PATH`
environment variable.

Add `export PATH="/Users/bob/.local/bin:$PATH"` to your shell configuration file.

Alternatively, you can call Poetry explicitly with `/Users/bob/.local/bin/poetry`.

You can test that everything is set up by executing:

`poetry --version`

It's important to check the output from the install? Specifically Add `export PATH="/Users/bob/.local/bin:$PATH"` to your shell configuration file.`.

Remember, after you add this line, you either need to open a new terminal or run source ~/<your_shell_config_file>.

[vincer]

A one-liner of @ikebo 's solution:

curl -sSL https://install.python-poetry.org | sed 's/symlinks=False/symlinks=True/' | python3 -

[dream-dasher]

Just to add some context, in case it impacts prioritization or maintainer interest: these issues make it much more difficult to get other workplace devs to use Poetry. Already an uphill battle. Macs being common as the type of computer companies give to employees (at least in my area).

[iftekhariasif]

I am getting the following error => My Python version is => Python 3.11.5 I ran the command => curl -sSL https://install.python-poetry.org | python3 - Can anyone help?

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1286, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1332, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1281, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1041, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 979, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1458, in connect
    self.sock = self._context.wrap_socket(self.sock,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1108, in _create
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1379, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 945, in <module>
  File "<stdin>", line 923, in main
  File "<stdin>", line 524, in run
  File "<stdin>", line 786, in get_version
  File "<stdin>", line 847, in _get
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 519, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)>

[Kevin-free]

A one-liner of @ikebo 's solution:

curl -sSL https://install.python-poetry.org | sed 's/symlinks=False/symlinks=True/' | python3 -

Thanks! I have been stuck at Installing Poetry (1.6.1): Installing Poetry. After waiting for a long time, there is no progress. The network is normal. How to troubleshoot and solve it?

Mac OS 12.6.3

[dream-dasher]

I ran the command => curl -sSL https://install.python-poetry.org | python3 - Can anyone help?

@iftekhariasif : see the post two comments before yours - using sed to switch symlink setting

[luceat-lux-vestra]

I think it should be fixed. homebrew way is just an alternative way.