uv icon indicating copy to clipboard operation
uv copied to clipboard

Use users pip.conf

Open msharp9 opened this issue 1 year ago • 41 comments

Pip allows a user to set configuration settings inside a pip.conf, usually stored at ~/.config/pip/pip.conf. For example, setting a default index-url. uv seems to ignore these settings completely.

This is a problem since enterprise companies typically have their own PyPI images and don't want users to use the public PyPI for security reasons. For example, users who use AWS codeartifact have this config file updated automatically when they run aws login. https://docs.aws.amazon.com/codeartifact/latest/ug/python-configure-pip.html

msharp9 avatar Feb 16 '24 00:02 msharp9

Thanks for the clear issue!

zanieb avatar Feb 16 '24 00:02 zanieb

I was just about to raise an issue for this exact use case. We also use codeartifact for a few in house packages but mostly pull public PyPi packages via AWS codeartifact.

Anyway +1 to this. Would love to use uv if this can be supported.

owenlamont avatar Feb 16 '24 07:02 owenlamont

We also set index-url in our venv by pip config --site set install.index-url "http://path/to/internal/pypi/server"

kkpattern avatar Feb 20 '24 10:02 kkpattern

BTW, for Windows this is %APPDATA%\pip\pip.ini, see https://pip.pypa.io/en/stable/topics/configuration/#location

schlamar avatar Feb 22 '24 09:02 schlamar

Can you perhaps add to the README instructions on how to set the index-url? My company uses a proxy repo and it's not obvoius how to read from it. Do we have to supply --index-url every time we call python -m pip uv? That seems a bit cumbersome.

mcrumiller avatar Feb 25 '24 15:02 mcrumiller

Hi @mcrumiller you can use UV_INDEX_URL instead. We may want to add all of our environment variables to the README, but they are currently listed in the --help output e.g.

  -i, --index-url <INDEX_URL>
          The URL of the Python package index (by default: https://pypi.org/simple)
          
          [env: UV_INDEX_URL=]

zanieb avatar Feb 25 '24 15:02 zanieb

The environment variables are nice, but reading the pip config file would be very good for enterprises. It would allow a seamless transition to uv without installing additional configuration on the machines.

PetterS avatar Mar 01 '24 12:03 PetterS

We'd like the system config to be considered as well (/etc/xdg/pip/pip.conf on Ubuntu). Our organisation configure a local mirror for all dev machines & users this way.

edwardpeek-crown avatar Mar 04 '24 21:03 edwardpeek-crown

The environment variables are nice, but reading the pip config file would be very good for enterprises. It would allow a seamless transition to uv without installing additional configuration on the machines.

Exactly, this is what I was expecting from uv, a drop-in replacement with a major performance boost.

RichardDally avatar Mar 05 '24 09:03 RichardDally

This is still under consideration, but the comment at https://github.com/astral-sh/uv/issues/1789#issuecomment-1978810917 explains our current stance well.

zanieb avatar Mar 05 '24 15:03 zanieb

Hi @mcrumiller you can use UV_INDEX_URL instead. We may want to add all of our environment variables to the README, but they are currently listed in the --help output e.g.


  -i, --index-url <INDEX_URL>

          The URL of the Python package index (by default: https://pypi.org/simple)

          

          [env: UV_INDEX_URL=]

I've been trying environment variables but it raises an error related to Unexpected fragment on URL: contact. I've seen issues that have been closed for this, I'm a little bit puzzled here. Using version 0.1.12 on Windows on private Artifactory repository.

RichardDally avatar Mar 13 '24 14:03 RichardDally

@RichardDally could you please create a new issue if this is not related to pip.conf support?

zanieb avatar Mar 22 '24 19:03 zanieb

Okay, we are officially considering support for this feature. Here is my proposal.

Require explicit selection of configuration files

pip supports discovery of configuration files at various locations

  • Do not support automated discovery of pip.conf
  • Do not support global / user / site
  • Use of pip.conf requires opt-in e.g. --pip-config <path> or UV_PIP_CONFIG=<path>

Only support index configuration

pip supports all of the command-line options via the configuration file

  • Do not support flags like no-compile, etc.
    • Unknown flags are ignored without warning
    • Flags we support in the CLI but not in the config file result in a warning in verbose mode
  • Support for:
    • find-links, index-url, extra-index-url
  • Possible support for native-tls toggle

Do not support configuration per command

pip supports separate configuration for each sub-command like install

  • All configuration must be either in a [global] declaration or in the top-level
  • Declarations in the top-level will override those in a [global] section
    • We could alternatively error or avoid checking [global] at all if a top-level option exists
  • Configuration per command will be silently ignored
  • If configuration file is provided without [global] or top-level options we will warn it is empty

zanieb avatar Mar 22 '24 19:03 zanieb

Require explicit selection of configuration files

pip supports discovery of configuration files at various locations

  • Do not support automated discovery of pip.conf
  • Do not support global / user / site
  • Use of pip.conf requires opt-in e.g. --pip-config <path> or UV_PIP_CONFIG=<path>

Maybe UV can auto discovery pip.conf in a virtual environment?

UV_INDEX_URL is actually enough for most use cases. I think many want uv to support pip.conf because they want uv as a drop-in replacement for pip. Without auto-discovery pip.conf from a virtual env, this cannot be achieved.

Also, in our project's CI job, there're multiple stage install different dependencies into the same .venv. For example, the bootstrap script may setup the .venv and install all the essential dependencies. Then in a later stage, the test script will install all the test related dependencies into the .venv.

By auto-picking up the pip.conf in the virtual environment, all the jobs in the following stages don't need to worry about which index it should use, which is very convenient for us.

kkpattern avatar Mar 23 '24 08:03 kkpattern

Maybe UV can auto discovery pip.conf in a virtual environment?

What exactly do this mean? There's a pip.conf file in the .venv folder?

zanieb avatar Mar 29 '24 16:03 zanieb

What exactly do this mean? There's a pip.conf file in the .venv folder?

Yes. If you activate a .venv then run pip config --site set install.index-url "http://pypi.internal.com", there will be a pip.conf in the .venv containing following content:

[install]
index-url = http://pypi.internal.com

Later pip commands will automatically pick up this config file.

kkpattern avatar Mar 29 '24 17:03 kkpattern

Why not just export an environment variable selecting the configuration file? e.g. UV_PIP_CONFIG=./.venv/pip.conf?

zanieb avatar Mar 29 '24 17:03 zanieb

It's convenient when we want to install some libs into a .venv across multiple sessions.

For example, a developer may set up a local .venv for development and use it for a relatively long time. Then, one day, they may need to update or install a new lib from an internal Pypi index. Since pip can automatically pick up the pip.conf file in the .venv folder, they don't need to remember what the index is or where the pip.conf is located.

This is not a deal breaker. We already use uv in our project and are very happy with it. But I think it's a nice-to-have feature.

kkpattern avatar Mar 29 '24 17:03 kkpattern

Thanks for the additional details! I don't think we'd launch this feature with support for that, but I'm not fully opposed. I worry about sometimes supporting inference, but having it work in the most-scoped context makes some sense.

zanieb avatar Mar 29 '24 17:03 zanieb

Just as another viewpoint for this conversation, as someone who has helped set up a lot of different users Python environments a lot of users get confused by all the different places pip configuration can go, e.g. environmental variables, multiple user directories, multiple system directories, venv directories etc.

I've encountered many time someone not being able to find where a setting was set, and have a broken environment until someone assisted them.

notatallshaw avatar Mar 29 '24 19:03 notatallshaw

Do not support automated discovery of pip.conf Do not support global / user / site

Maybe one default location can be supported without causing any confusion? I would pick user config as the one config if I had to choose. I see how not picking any discovery is a simple solution, but perhaps there is a benefit if this choice is done well.

With environment variables the issue is e.g. I use fish shell, but sometimes things are run through bash. Then I have to maintain two configs and export variables there. Even for one shell its an extra step that sometimes could be avoided.

ikamensh avatar Apr 03 '24 14:04 ikamensh

The intent is to introduce our own configuration in the future and we'll support automatic discovery of that. We're just not particularly comfortable reading other tool's configuration files without opt-in.

zanieb avatar Apr 03 '24 16:04 zanieb

Since I first created the issue there's been a lot of conversation, this is what I would suggest as I've thought about everyone's different viewpoints.

UV should create it's own configuration file and provide automatic discovery for it. There should then be another tool that converts pip configs to uv configs, e.g pip.conf -> uv.conf. This second tool could provide a simple extensible configuration to add default locations. Collect all pip.conf, and create one combined uv.conf to the auto discovery location.

If UV then added support for plugins, then all a user would have to do is pip install both, say, pip install uv uv-pip-conf. This would make the user experience feel much more like UV is actually a drop in replacement to pip and pip-tools since no flags or environment variables would have to be set.

This would be similar to how flake8 plugins work, simply pip install flake8 and flake8-import-order as an example, and now flake8 automatically checks your codes imports along with other syntax.

msharp9 avatar Apr 03 '24 20:04 msharp9

@zanieb Slightly off topic, but would you be willing to share an expected timeline for the config file feature? It’s the one thing preventing me from immediately swapping all of my projects to uv. 😅 Many thanks, love the project.

d-miketa avatar Apr 03 '24 21:04 d-miketa

Hey @d-miketa — we don't have a timeline yet. I think it will be relatively straight-forward to implement but I'm giving time for feedback on my proposal which only has four upvotes compared to the ~50 on the original post.

zanieb avatar Apr 03 '24 23:04 zanieb

As stated by @PetterS not having built-in support for /etc/pip.conf will be detrimental for Enterprise adoption.

In our case we drop a configuration in /etc/pip.conf using Ansible, we then symlink this file to /etc/xdg/pip/pip.conf for Ubuntu based systems.

After that any Python package installed using ansible.pip module automatically uses the configuration from /etc/pip.conf, we don't have to expose ENV's or write extra config files.

gaby avatar Apr 14 '24 23:04 gaby

That requirement sounds more like you need a persistent configuration file rather than a pip.conf specifically - or am I misunderstanding?

charliermarsh avatar Apr 14 '24 23:04 charliermarsh

@charliermarsh In enterprise environments /etc/pip.conf is used to managed several things:

  • Index-url
  • Search-index
  • HTTP/HTTPS proxies
  • Custom CA's

User are just able to do "pip install" without having to configure anything since it's managed by the platform via /etc/pip.conf.

gaby avatar Apr 14 '24 23:04 gaby

Any progress on this? It's critical for enterprise environments.

astronautas avatar Apr 18 '24 11:04 astronautas

The intent is to introduce our own configuration in the future and we'll support automatic discovery of that. We're just not particularly comfortable reading other tool's configuration files without opt-in

Then uv will stay indefinitely a non-viable product for enterprise solutions.

/etc/pip.conf or even sometimes pip.conf files getting cp'd inside of the venv folder during the dependency installation phase is an industry-wide practice/trend, for better or for worse.

Regardless, huge fan of the product and thank you immensely for your time and hard work. Just sad to see this will only stay on personal projects.

CostcoFanboy avatar Apr 20 '24 19:04 CostcoFanboy