pip icon indicating copy to clipboard operation
pip copied to clipboard

The priority issue regarding the download mirror address for pip

Open pptfz opened this issue 1 month ago • 8 comments

When I use the following command, what I understand is that the -i option indicates the source from which I want to download the package. --extra-index-url refers to the alternative source behind it.

However, the actual test result I got was that I always accessed the source after the "--extra-index-url" parameter, and never downloaded from the source after the "-i" parameter.

I don't understand why this is the case.Could help me solve this problem? Thank you.

RUN --mount=type=cache,target=/root/.cache/pip \
    pip install --default-timeout=120 --retries=10 \
      -r requirements.txt \
      -i https://mydomain.com/repository/pypi_group/simple \
     --extra-index-url https://pypi.org/simple/

pptfz avatar Nov 28 '25 11:11 pptfz

The indexes specified by --index-url and --extra-index-url are treated equally. There's no priority or ordering between them, pip can download a package from either location. This is considered safe, because you should never have the same wheel containing different code on two different indexes. But if you're trying to get better performance, for example by preferring a local repository over a remote one, these options won't help you (and indeed, pip doesn't offer that functionality).

pfmoore avatar Nov 28 '25 11:11 pfmoore

The indexes specified by --index-url and --extra-index-url are treated equally. There's no priority or ordering between them, pip can download a package from either location. This is considered safe, because you should never have the same wheel containing different code on two different indexes. But if you're trying to get better performance, for example by preferring a local repository over a remote one, these options won't help you (and indeed, pip doesn't offer that functionality).--index-url--extra-index-url 指定的索引被同等对待。它们之间没有优先级或顺序,pip 可以从任一位置下载包。这被认为是安全的,因为您永远不应该在两个不同的索引上使用包含不同代码的相同轮子。但是,如果您想获得更好的性能,例如通过选择本地存储库而不是远程存储库,这些选项将无法帮助您(事实上,pip 不提供该功能)。

Thank you for your response. It seems that I misunderstood. However, there is still one aspect that I don't quite understand. When both the --index-url and --extra-index-url parameters are specified, the result of my experiment here is that the package is always downloaded from the source specified after --extra-index-url. I'm not quite clear about this point.

pptfz avatar Dec 01 '25 06:12 pptfz

You should consider that as unreliable. It’s not documented behaviour, so it’s subject to change without notice. It may even change without there being a change in pip (for example, if the URLs were checked in alphabetical order, a change in the URL you specify could change the order).

Basically, if you rely on the ordering, you are accepting the risk that it could change without warning. I’d strongly recommend that you don’t do that.

pfmoore avatar Dec 01 '25 08:12 pfmoore

You should consider that as unreliable. It’s not documented behaviour, so it’s subject to change without notice. It may even change without there being a change in pip (for example, if the URLs were checked in alphabetical order, a change in the URL you specify could change the order).您应该认为这是不可靠的。它不是记录在案的行为,因此如有更改,恕不另行通知。它甚至可能会在 pip 没有变化的情况下发生变化(例如,如果按字母顺序检查 URL,则您指定的 URL 的更改可能会更改顺序)。

Basically, if you rely on the ordering, you are accepting the risk that it could change without warning. I’d strongly recommend that you don’t do that.基本上,如果您依赖订单,您就接受了订单可能会在没有警告的情况下发生更改的风险。我强烈建议你不要这样做。

Thank you for your reply. That means when installing pip, you should only write one option. You shouldn't write both --index-url and --extra-index-url at the same time. However, I noticed that in the command help, it states that an additional option is provided, but it doesn't specify the specific usage scenarios.

Image

pptfz avatar Dec 01 '25 09:12 pptfz

No, you can use multiple options, that works perfectly well. But what you must ensure is that any package present on multiple indexes is identical on all of them, so that it doesn't matter which index pip chooses. This should be true in all real cases - after all, version 2.0 of package foo should be the same regardless of where you get it from!

The point of having --extra-index-url is to specify an index that serves additional packages, not packages that already exist on the main index.

I'm not entirely sure why it matters to you which index pip uses when fetching a package that's available on both indexes that you specify. If you can explain that, maybe I can give you clearer information.

pfmoore avatar Dec 01 '25 15:12 pfmoore

FYI this is basically a duplicate of https://github.com/pypa/pip/issues/8606

You shouldn't write both --index-url and --extra-index-url at the same time.

You can absolutely use both if you are comfortable that either could be used to source any package. For example if they are mirrors of each other, or two private indexes that you control.

Pip may add prioritization options in the future, e.g. https://github.com/pypa/pip/issues/8606#issuecomment-2206811503

notatallshaw avatar Dec 01 '25 15:12 notatallshaw

No, you can use multiple options, that works perfectly well. But what you must ensure is that any package present on multiple indexes is identical on all of them, so that it doesn't matter which index pip chooses. This should be true in all real cases - after all, version 2.0 of package foo should be the same regardless of where you get it from!不,您可以使用多个选项,效果非常好。但您必须确保多个索引上存在的任何包在所有索引上都是相同的,因此 pip 选择哪个索引并不重要。在所有实际情况下都应该如此 - 毕竟,无论您从哪里获得 foo 包的 2.0 版本都应该是相同的!

The point of having --extra-index-url is to specify an index that serves additional packages, not packages that already exist on the main index.拥有 --extra-index-url 的目的是指定一个为其他包提供服务的索引,而不是主索引上已存在的包。

I'm not entirely sure why it matters to you which index pip uses when fetching a package that's available on both indexes that you specify. If you can explain that, maybe I can give you clearer information.我不完全确定为什么在获取您指定的两个索引上可用的包时,pip 使用哪个索引对您很重要。如果你能解释一下,也许我可以给你更清晰的信息。

I'm not entirely sure why it matters to you which index pip uses when fetching a package that's available on both indexes that you specify. If you can explain that, maybe I can give you clearer information.

Thank you for your reply. Suppose I have a private package developed within my company and it has been uploaded to the company's private repository. Then, when I execute pip install, I only need to specify one index? --index-url should be followed by the address of the internal private repository. If you need to install other public packages, it will default to downloading from the official source of pip. Am I understanding this correctly?

Regarding the issue you mentioned, I initially misunderstood that the priority of --index-url is higher than --extra-index-url. I started using these two options because we had our own internal pip repository, which cached certain packages to improve the speed of package downloads during Python project compilation. Thus, I used both --index-url and --extra-index-url simultaneously. My original thought was that --index-url pointed to our internal repository for faster downloads, while --extra-index-url specified a fallback source to fetch packages when the internal repository lacked them or when the cache might have expired. This was my initial understanding

I'm not entirely sure why it matters to you which index pip uses when fetching a package that's available on both indexes that you specify. If you can explain that, maybe I can give you clearer information.

pptfz avatar Dec 02 '25 02:12 pptfz

FYI this is basically a duplicate of #8606仅供参考,这基本上是 #8606 的重复

You shouldn't write both --index-url and --extra-index-url at the same time.您不应该同时写入 --index-url--extra-index-url

You can absolutely use both if you are comfortable that either could be used to source any package. For example if they are mirrors of each other, or two private indexes that you control.如果您愿意使用两者中的任何一个来获取任何软件包,那么您绝对可以使用两者。例如,如果它们是彼此的镜像,或者是您控制的两个私有索引。

Pip may add prioritization options in the future, e.g. #8606 (comment)Pip 将来可能会添加优先级选项,例如 第8606章 (评论)

Thank you very much for your reply. I just noticed these two options recently, and I mistakenly thought there was a priority issue.

pptfz avatar Dec 02 '25 07:12 pptfz