rules_python icon indicating copy to clipboard operation
rules_python copied to clipboard

`rules_python_publish_deps` causing pip extension resolution to always require network access

Open meteorcloudy opened this issue 4 months ago • 12 comments

🐞 bug report

Affected Rule

https://github.com/bazel-contrib/rules_python/blob/b40d96aba36d675c60b03424aa22f31c09e0ea4f/MODULE.bazel#L63-L75

Is this a regression?

Yes, after https://github.com/bazel-contrib/rules_python/commit/b9b0948234216cf43bb898eec078b1c71a37c1f9, moving from pip_internal to pip.

Description

The rules_python_publish_deps python hub uses the same module extension (pip) end users use, which causes the module extension resolution to always require network access. In offline build without an up-to-date lockfile, the module extension resolution will fail even though rules_python_publish_deps is never required by the end user.

This was discovered in https://github.com/bazelbuild/bazel/pull/26163, where upgrading rules_python broke Bazel's bootstrap build.

🔬 Minimal Reproduction

🔥 Exception or Error




[1A[K[31m[1mERROR: [0m/tmp/bazel_8RGzY3p4/out/external/rules_python+/python/private/pypi/simpleapi_download.bzl:130:14: Traceback (most recent call last):
	File "/tmp/bazel_8RGzY3p4/out/external/rules_python+/python/private/pypi/extension.bzl", line 615, column 25, in _pip_impl
		mods = parse_modules(module_ctx)
	File "/tmp/bazel_8RGzY3p4/out/external/rules_python+/python/private/pypi/extension.bzl", line 490, column 36, in parse_modules
		out = _create_whl_repos(
	File "/tmp/bazel_8RGzY3p4/out/external/rules_python+/python/private/pypi/extension.bzl", line 155, column 50, in _create_whl_repos
		requirements_by_platform = parse_requirements(
	File "/tmp/bazel_8RGzY3p4/out/external/rules_python+/python/private/pypi/parse_requirements.bzl", line 180, column 36, in parse_requirements
		index_urls = get_index_urls(
	File "/tmp/bazel_8RGzY3p4/out/external/rules_python+/python/private/pypi/extension.bzl", line 470, column 79, in lambda
		get_index_urls = lambda ctx, distributions: simpleapi_download(
	File "/tmp/bazel_8RGzY3p4/out/external/rules_python+/python/private/pypi/simpleapi_download.bzl", line 130, column 14, in simpleapi_download
		_fail("Failed to download metadata for {} for from urls: {}".format(
Error in fail: Failed to download metadata for ["nh3", "idna", "rich", "zipp", "mdurl", "twine", "certifi", "keyring", "pkginfo", "rfc3986", "urllib3", "docutils", "pygments", "requests", "jaraco-classes", "jaraco-context", "markdown-it-py", "more-itertools", "readme-renderer", "jaraco-functools", "backports-tarfile", "requests-toolbelt", "charset-normalizer", "importlib-metadata", "cffi", "jeepney", "pycparser", "cryptography", "secretstorage", "pywin32-ctypes"] for from urls: ["https://pypi.org/simple"]

🌍 Your Environment

Operating System:

  
macOS, Linux, Windows
  

Output of bazel version:

  
8.2.1
  

Rules_python version:

  
1.3.1
  

Anything else relevant?

This might be working as intended, but should rules_python_publish_deps be dev_dependency?

meteorcloudy avatar May 28 '25 10:05 meteorcloudy