Using Poetry behind a company proxy

[incoggnito]

• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] I have searched the documentation and believe that my question is not covered.

Feature Request

I would have different ideas for everyone who wants to use poetry through a company proxy-server:

1. Check the proxy settings in the pip.ini settings and use them
2. If the poetry secondary source is true and the primary (pypi) throws an https error use the secondary
3. Add the proxy settings to the global poetry config

If there is no other solution yet and if it is also desired by others, I would try to add that myself. Thanks for any reply.

PS: I had already started a discussion on the subject, if that is more appropriate. Link

[incoggnito]

I found a workaround on win10 (git bash) and unix: Add this to the .bashrc:

$ export HTTP_PROXY="http://<user>:<pass>@<server>:<port>"
$ export HTTPS_PROXY="http://<user>:<pass>@<server>:<port>"

I currently work on a sample pull request for the first 2 points.

[MajorDallas]

There's a limitation to using the http_proxy env variables: If the proxy doesn't "hairpin" requests to internal sites, then any dependencies hosted on eg. an internal git repo can't be installed.

The ideal solution in that case would be global and per-dependency proxy settings that can work with http_proxy.

[alexKiew]

@incoggnito Hello. I have Raspberry Pi4 behind company - proxy. poetry install bring me proxy error. Do you know solution?

[humbinal]

who can i use my own pypi url(mirror) in proxy? like pip --index

[nesb1]

I found the solution. You need simply add this in pyproject.toml:

[[tool.poetry.source]] name = "proxy" url = "your proxy url here" default = true

[alicederyn]

That's not a solution, that's a workaround :( Poetry should use the pip configuration. Absolute worst case, it should have global configuration of its own, but I can't find any in https://python-poetry.org/docs/configuration/. Changing the checked in pyproject.toml of e.g. an open-source project you're working on simply isn't an option.

[djbrown]

This is very unpleasant..

I also tried different ways to not have the mirror url in the pyproject.toml but I could'nt find a way.. Neither via Local Configuration (poetry.toml) nor "user global" Config Directory (~/.config/pypoetry/pyproject.toml).

@sdispater can we please get a way to specify pypi mirror/repository/source as configured from outside the pyproject.toml? also highly relevant for https://github.com/python-poetry/poetry/issues/559

[neersighted]

Duplicate #1632. Please refrain from directly pinging project contributors in the future. It's unnecessary as those who are interested will get notifications for every issue, and inconsiderate to those who have opted out of notifications for every single issue.

[gwangjinkim]

Same here. Behind a company proxy. Proxy data should not have to be added into pyproject.toml - this is bad for security. And this is unnecessarily publishing information which other people don't need. There should be a way to globally define a proxy - or give the proxy as an argument.

[HuM4NoiD]

@neersighted how is this a duplicate of #1632? #1632 caters to the fact that a "mirror"/"proxy" repository should be used globally. As far as I understand this issue is about using http_proxy or https_proxy. This issue should be reopened.

[mdecano-av]

Hi, was there an actual fix for the issue? The proxy has been set in my environment variables and the ide of choice. I've also tried to add the information on the project.toml but no success.

[F18]

@neersighted how is this a duplicate of #1632? #1632 caters to the fact that a "mirror"/"proxy" repository should be used globally. As far as I understand this issue is about using http_proxy or https_proxy. This issue should be reopened.

I would agree. The issue seems to be that there is no equivalent in Poetry to something like pip config --site set global.proxy. When behind a company proxy, this inhibits the ability to install packages, even from pypi.org. As such, this issue does not seem to be a duplicate of #1632, since it is unrelated to replacing the URL of a source.

[gwangjinkim]

There should be both:

• a global or local config for proxy handling for poetry
• the possibility to add a --proxy "<proxyurl>" argument for a poetry call which gets propagated to pip inside poetry. And also propagated to any git calls. - esecially for shutting of proxy for certain cases. While in Linux no_proxy env variable does that. In Windows, no_proxy gets ignored.

[dimbleby]

since #1632 does not have either a proposal or an implementation it is strange to judge it "very inefficient and complicated".

As far as I know - I have not checked - HTTP_PROXY and HTTPS_PROXY environment variables will be respected by poetry: poetry is essentially just calling through to requests and I don't believe it does anything to mask those variables.

If you need something more sophisticated than that... well this is open source. The way that things happen is: someone who is motivated and able shows up and makes them happen. Is that you?

[F18]

I agree, @dimbleby. Someone in the open source community, with the right motivation and talent to fix this, could certainly do so.

Unfortunately, the HTTP_PROXY and HPPTS_PROXY do not seem to be respected by poetry, through to requests.

My only comment or request would be to re-open this. Since this issue is currently marked as a duplicate of the unrelated issue #1632, the chances of it getting fixed by and for the poetry community are certainly reduced.

[dimbleby]

I'm pretty sure you're wrong, and those variables are respected. I don't have a real proxy to play with: but if I set HTTPS_PROXY to a bogus value (and clear poetry's cache) then that causes poetry to fail:

...

  ProxyError

  HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/google-auth/ (Caused by ProxyError('Unable to connect to proxy', ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7fd666aefe20>, 'Connection to 1.2.3.4 timed out. (connect timeout=15)')))

[F18]

Okay, so if that's the case, then it is quite possible that something else is causing poetry to fail behind a proxy. Here are some findings:

• when using pip with the HTTP(S)_PROXY variables defined, installing pypi.org packages behind the proxy is successful

• when using pip without the HTTP(S)_PROXY variables defined, installing pypi.org packages behind the proxy fails (unless I use the --proxy option or I have my pip.ini defined)

• when using poetry with OR without the HTTP(S)_PROXY variables defined, installing pypi.org packages behind the proxy fails (and there is no equivalent --proxy "<proxyurl>" option in poetry)

• when using poetry with a bogus HTTP(S)_PROXY, installing pypi.org packages fails

Note: other users (in comments above) have reported that despite having the HTTP(S)_PROXY variables defined, they are unable to install packages using poetry behind a proxy.

[F18]

Quick question to the smart people that make these kinds of decisions: is it a crazy notion to re-open this issue based on the above discussions and findings?

If you need more info or testing, please let us know. It looks like there are several people interested in this issue. Unfortunately, I do not have the talent to fix this myself (at this stage), but if we re-open the issue, maybe some brillant contributor out there will conquer it once and for all.

[Gr3at]

I forked the poetry repo to implement the functionality, but it seems to be provided out of the box.

as it seems poetry install respects the http(s)_proxy.

Here is an example .gitlab-ci.yml to indicate the functionality in a reproducible environment.

# .gitlab-ci.yml
image: python:3.10-alpine

stages:
    - test

pytest-job:
    stage: test
    before_script:
        - pip list
        - export http_proxy=$COMPANY_PROXY
        - export https_proxy=$COMPANY_PROXY
        - echo "Installing poetry..."
        - pip install poetry
        - echo "Installing requirements..."
        - poetry install
        - echo "Preparing db"
        # - poetry run make
        - poetry run python src/manage.py makemigrations
        - poetry run python src/manage.py migrate
    script:
        - echo "Running tests..."
        - poetry run pytest -vv
        - echo "Testing complete."
    cache:
        key: venv-cache
        paths:
            - .venv

FYI: the outputs of pip list and pip install poetry is the following

$ pip list
Package    Version
---------- -------
pip        23.0.1
setuptools 65.5.1
wheel      0.40.0

$ pip install poetry
Collecting poetry
  Downloading poetry-1.5.1-py3-none-any.whl (225 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 225.2/225.2 kB 4.2 MB/s eta 0:00:00
Collecting cachecontrol[filecache]<0.13.0,>=0.12.9
....

[yhyu13]

can confirm only pip config set global.proxy xxx works for petry, env var all not working

[Gr3at]

can confirm only pip config set global.proxy xxx works for petry, env var all not working

Can you provide a reproducible example like the one above?

[jalwn]

poetry add does not respect the proxy set in pip config

im on windows 11

[Philaeux]

For people finding this issue, having a problem to run poetry behind a HTTP company proxy on Windows.

I used to set my proxy for pip using pip config set global.proxy <company http proxy>, which was not taken into account by poetry. Instead, you need to set your HTTP_PROXY environment variables.

To do so in powershell:

$env:HTTP_PROXY = "<your company HTTP proxy>"
$env:HTTPS_PROXY = "<your company HTTPS proxy>"
# Now it should work
poetry add SQLAlchemy

You can also do Windows > Edit environment variable to add these there.

A problem appears if you want to use a company repository / company mirror repository (so without http proxy) and public PYPI (with http proxy) as you cannot setup "per repository proxy".

A solution to this would be to have the option per repository in pyproject.tom. If a login is required, the auth information should be some local conf outside project conf.

pyproject.toml (in your git)

[[tool.poetry.source]]
name = "company-repository"
priority = "default"
url = "<company repository url>"

[[tool.poetry.source]]
name = "pypi"
priority = "primary"
proxy = "<company http proxy>
auth = "str-key" (optional, use if proxy has auth)

config.toml (on your machine)

[[proxies.str-key]]
username = "<company http proxy username>"
password = "<company http proxy password>"

This duality "company-repository" + "pypi repository with http proxy" must be very common and should be answered by poetry for adoption.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.