uv `uv add` and `uv lock` fail on corporate network

I'm running into some problems trying to use uv add and uv lock commands on a machine running behind a corporate proxy.

I have configured internal pypi mirrors in my ~/.config/uv/uv.toml file:

[pip]
index-url = "<pypi_mirrror_url>"

I then run through the following "hello world" example and get some errors along the way:

➜  uv init hello-uv             
Initialized project `hello-uv` at `/home/develop/hello-uv`

➜  cd hello-uv
 
➜  uv add requests 
Using Python 3.12.3 interpreter at: /home/linuxbrew/.linuxbrew/opt/[email protected]/bin/python3.12
Creating virtualenv at: .venv
⠼ requests==1.2.3                                                                                                                                   error: Failed to download and build `requests==1.2.3`
  Caused by: Failed to extract archive
  Caused by: Invalid gzip header

➜  cat pyproject.toml
[project]
name = "hello-uv"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
    "requests",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

➜  uv lock           
⠼ requests==1.2.3                                                                                                                                   error: Failed to download and build `requests==1.2.3`
  Caused by: Failed to extract archive
  Caused by: Invalid gzip header

➜  uv pip compile pyproject.toml
Resolved 5 packages in 1.66s
# This file was autogenerated by uv via the following command:
#    uv pip compile pyproject.toml
certifi==2024.7.4
    # via requests
charset-normalizer==3.3.2
    # via requests
idna==3.8
    # via requests
requests==2.32.3
    # via hello-uv (pyproject.toml)
urllib3==2.2.2
    # via requests

The interesting thing is that the uv pip compile pyproject.toml command succeeds, whereas the uv add and uv lock commands fail.

If I subsequently delete the ~/.config/uv/uv.toml file, then the uv pip compile pyproject.toml command fails with the same error:

➜  uv pip compile pyproject.toml
⠸ requests==1.2.3                                                                                                                                   error: Failed to download and build `requests==1.2.3`
  Caused by: Failed to extract archive
  Caused by: Invalid gzip header

This would seem to imply that the uv add and uv lock commands are not respecting the index-url configuration.

Aug 23 '24 23:08 maxfirman

Can you try removing the [pip] header from your uv.toml?

Aug 23 '24 23:08 charliermarsh

Settings under [pip] only apply to uv pip; settings at the top-level apply to everything (uv sync, uv lock, and uv pip et al).

Aug 23 '24 23:08 charliermarsh

Brilliant, that worked! Thanks

Aug 23 '24 23:08 maxfirman

@charliermarsh why do we fail with this weird error? because the proxy returns a weird response?

Aug 23 '24 23:08 zanieb

(@maxfirman if you can get more details about what the response from the proxy is that'd be helpful for improving the error message for other users, uv will say more about the requests with RUST_LOG=trace uv add --verbose ...)

Aug 23 '24 23:08 zanieb

uv_add_failure.log @zanieb log file attached

Aug 24 '24 00:08 maxfirman

What kind of response is returned when you GET https://files.pythonhosted.org/packages/61/79/efc316760a906763de872d7328c9bf8c5af28708a35fdae57fbb4ee005f7/requests-1.2.3.tar.gz?

Aug 24 '24 01:08 zanieb

curl_pythonhosted.log @zanieb see attached log. The request goes via our proxy and gets redirected to a our Artifactory which ultimately returns a 404. This looks likely to be a proxy misconfiguration on our end. It would definitely be better to have uv fail immediately and surface the 404 error.

Aug 24 '24 19:08 maxfirman

Running into (I think) similar:

RUST_LOG=trace uv pip install --verbose --native-tls https://my.host.com/mylib/archive/refs/tags/v5.1.15.tar.gz

Logs (edited to provide corrected logs)

DEBUG Using Python 3.11.9 environment at .venv
TRACE Checking lock for `.venv` at `.venv/.lock`
DEBUG Acquired lock for `.venv`
DEBUG At least one requirement is not satisfied: https://my.host.com/mylib/archive/refs/tags/v5.1.15.tar.gz
DEBUG Using request timeout of 30s
TRACE Checking lock for `/.../Caches/uv/sdists-v6/url/7581192789a2c104` at `/.../Caches/uv/sdists-v6/url/7581192789a2c104/.lock`
DEBUG Acquired lock for `/.../Caches/uv/sdists-v6/url/7581192789a2c104`
TRACE No cache entry exists for /.../Caches/uv/sdists-v6/url/7581192789a2c104/revision.http
DEBUG No cache entry for: https://my.host.com/mylibx/archive/refs/tags/v5.1.15.tar.gz
TRACE Sending fresh GET request for https://my.host.com/mylib/archive/refs/tags/v5.1.15.tar.gz
TRACE Handling request for https://my.host.com/mylib/archive/refs/tags/v5.1.15.tar.gz
TRACE Request for https://my.host.com/mylib/archive/refs/tags/v5.1.15.tar.gz is unauthenticated, checking cache
TRACE No credentials in cache for URL https://my.host.com/mylib/archive/refs/tags/v5.1.15.tar.gz
TRACE Attempting unauthenticated request for https://my.host.com/mylib/archive/refs/tags/v5.1.15.tar.gz
TRACE checkout waiting for idle connection: ("https", my.host.com)
DEBUG starting new connection: https://my.host.com/    
TRACE Http::connect; scheme=Some("https"), host=Some("my.host.com"), port=None
DEBUG connecting to 10.52.226.221:443
DEBUG connected to 10.52.226.221:443
TRACE http1 handshake complete, spawning background dispatcher task
TRACE checkout dropped for ("https", my.host.com)
TRACE put; add idle connection for ("https", my.host.com)
DEBUG pooling idle connection for ("https", my.host.com)
DEBUG redirecting 'https://my.host.com/mylib/archive/refs/tags/v5.1.15.tar.gz' to 'https://...%2Farchive%2Frefs%2Ftags%2Fv5.1.15.tar.gz'    
TRACE take? ("https", my.host.com): expiration = Some(90s)
DEBUG reuse idle connection for ("https", my.host.com)
TRACE idle interval checking for expired
TRACE cached request https://my.host.com/mylib/archive/refs/tags/v5.1.15.tar.gz is not storable because its response has a 'no-store' cache-control directive
DEBUG Downloading source distribution: https://my.host.com/mylib/archive/refs/tags/v5.1.15.tar.gz
DEBUG Released lock at `/.../Caches/uv/sdists-v6/url/7581192789a2c104/.lock`
DEBUG Released lock at `/.../.venv/.lock`
error: Failed to extract archive
  Caused by: Invalid gzip header

Dec 11 '24 06:12 brendan-morin

Ok in my case it appears credential oriented, and the credentials are not being read from netrc properly (though they seem to work fine for other commands). I confirmed this with some gzip commands:

> curl -L https://my.host.com/mylib/archive/refs/tags/v5.1.15.tar.gz -o package.tar.gz
> gzip -t package.tar.gz                                                                                                                                        

gzip: package.tar.gz: not in gzip format

> cat package.tar.gz

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
...

Basically, it just ends up grabbing the html for our internal "please sign in to access GitHub" page (the redirect log above should have cued me in). Looking at my .netrc, it seems my credentials should be passed through, but aren't for some reason:

machine my.host.com
login username
password ghp_token

I can confirm if I sub into the url that it does work.

uv pip install --native-tls https://username:[email protected]/mylib/archive/refs/tags/v5.1.15.tar.gz

Do we know why it wouldn't be picking up netrc in this case? Is this expected behavior?

Dec 11 '24 17:12 brendan-morin