rules_python
rules_python copied to clipboard
bazelbuild
Release notes on bzlmod changes.
Here are the git logs to dig through
commit 1c581242c25925a1bd6764608877ec170724f11d
Author: Chris Love <[email protected]>
Date: Thu Jun 15 10:58:54 2023 -0600
feat(bzlmod)!: Calling pip multiple times allowing for multiple Python versions (#1254)
Implementing the capability to call pip bzlmod extension multiple times.
We can now call the pip extension with the same hub name multiple times.
This allows a user to have multiple different requirement files based on
the Python version.
With workspace, we need incompatible aliases because you get
@pip//tabulate or @pip_tabulate//:pkg.
The requirement() macro hides this, but changing the flag becomes a
breaking change if you've manually referenced things. With bzlmod,
though, the @pip_tabulate style isn't a realistic option (you'd have to
use_repo everything, which is a huge pain).
So we have chosen to have `@pip//tabulate`.
This commit implements that capability for pip.parse to determine the
Python version from programmatically
the provided interpreter.
See
https://github.com/bazelbuild/rules_python/blob/76aab0f91bd614ee1b2ade310baf28c38695c522/python/extensions/pip.bzl#L88
For more in-depth implementation details.
INTERFACE CHANGE::
- pip.parse argument python_version is introduced but not required. When
possible, it is
inferred.
BREAKING CHANGE:
* `pip.parse()` extension: the `name` attribute is renamed `hub_name`.
This is to reflect
that the user-provided name isn't unique to each `pip.parse()` call. We
now have a hub that is created, and each
pip.parse call creates a spoke.
* `pip.parse()` extension: the `incompatible_generate_aliases` arg is
removed; behavior
has changed to assume it is always True.
* `pip.parse()` calls are collected under the same `hub_name` to support
multiple Python versions under the same resulting repo name (the hub
name0.
Close: #1229
diff --git a/examples/bzlmod_build_file_generation/MODULE.bazel b/examples/bzlmod_build_file_generation/MODULE.bazel
index fab2a26..6bc5880 100644
--- a/examples/bzlmod_build_file_generation/MODULE.bazel
+++ b/examples/bzlmod_build_file_generation/MODULE.bazel
@@ -44,12 +44,6 @@ bazel_dep(name = "gazelle", version = "0.30.0", repo_name = "bazel_gazelle")
# its methods can be invoked to create module extension tags.
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
-# This name is passed into python.toolchain and it's use_repo statement.
-# We also use the same name for python.host_python_interpreter.
-PYTHON_NAME = "python_3_9"
-
-INTERPRETER_NAME = "interpreter"
-
# We next initialize the python toolchain using the extension.
# You can set different Python versions in this block.
python.toolchain(
@@ -58,16 +52,6 @@ python.toolchain(
python_version = "3.9",
)
-# The interpreter extension discovers the platform specific Python binary.
-# It creates a symlink to the binary, and we pass the label to the following
-# pip.parse call.
-interpreter = use_extension("@rules_python//python/extensions:interpreter.bzl", "interpreter")
-interpreter.install(
- name = INTERPRETER_NAME,
- python_name = PYTHON_NAME,
-)
-use_repo(interpreter, INTERPRETER_NAME)
-
# Use the extension, pip.parse, to call the `pip_repository` rule that invokes
# `pip`, with `incremental` set. The pip call accepts a locked/compiled
# requirements file and installs the dependencies listed within.
@@ -77,10 +61,7 @@ use_repo(interpreter, INTERPRETER_NAME)
# operating systems, we have requirements for each.
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
- name = "pip",
- # When using gazelle you must use set the following flag
- # in order for the generation of gazelle dependency resolution.
- incompatible_generate_aliases = True,
+ hub_name = "pip",
# The interpreter_target attribute points to the interpreter to
# use for running pip commands to download the packages in the
# requirements file.
@@ -89,7 +70,7 @@ pip.parse(
# is used for both resolving dependencies and running tests/binaries.
# If this isn't specified, then you'll get whatever is locally installed
# on your system.
- python_interpreter_target = "@{}//:python".format(INTERPRETER_NAME),
+ python_version = "3.9",
requirements_lock = "//:requirements_lock.txt",
requirements_windows = "//:requirements_windows.txt",
)
commit b228f6047671abcf4c78ea7318916218fb098831
Author: Richard Levasseur <[email protected]>
Date: Thu Jun 8 07:50:20 2023 -0700
fix(bzlmod)!: Remove ability to specify toolchain repo name. (#1258)
The main reasons this is removed is because if modules choose different
names for the same toolchain, only one of the two toolchains (which are,
hopefully, identical) will be used. Which toolchain is used depends on
the module graph dependency ordering.
Furthermore, as of #1238, only one repo per version is created; others
are ignored. This means a module doesn't know if the name it chooses
will result in a repo being created with that name.
Instead, the toolchain repos are named by rules_python:
`python_{major}_{minor}`. These repo names are currently part of the
public API, since they end up referenced in MODULE config (to wire the
toolchain interpreter to pip).
BREAKING CHANGES
This removes the `name` arg from `python.toolchain()`. Users will need
to remove such usages from their `MODULE.bazel` and update their
`use_repo()` statements. If keeping the custom repo name is necessary,
then repo mappings can be used. See #1232 for additional migration
steps, commands, and information.
diff --git a/examples/bzlmod_build_file_generation/MODULE.bazel b/examples/bzlmod_build_file_generation/MODULE.bazel
index d69dd7d..fab2a26 100644
--- a/examples/bzlmod_build_file_generation/MODULE.bazel
+++ b/examples/bzlmod_build_file_generation/MODULE.bazel
@@ -46,14 +46,13 @@ python = use_extension("@rules_python//python/extensions:python.bzl", "python")
# This name is passed into python.toolchain and it's use_repo statement.
# We also use the same name for python.host_python_interpreter.
-PYTHON_NAME = "python"
+PYTHON_NAME = "python_3_9"
INTERPRETER_NAME = "interpreter"
# We next initialize the python toolchain using the extension.
# You can set different Python versions in this block.
python.toolchain(
- name = PYTHON_NAME,
configure_coverage_tool = True,
is_default = True,
python_version = "3.9",
commit 148622aa92fdd5afcaf9f153d7bb6afce713e553
Author: Chris Love <[email protected]>
Date: Wed May 31 19:48:28 2023 -0600
feat(bzlmod): Moving register.toolchains internal (#1238)
This commit moves the register.toolchains bzlmod call to inside
of rules_python. Instead of a user having to call register.toolchains
in their MODULE.bazel, rules_python/MODULE.bazel calls it
on the internal hub.
This is a breaking change if you are using register.toolchains inside
of submodules. Using register.toolchains inside of submodules is
not recommended anyways. This is now broken because we are not
creating a repo for every Python version toolchain. All of the
toochain calls exist now in the hub's repo BUILD.bazel file.
diff --git a/examples/bzlmod_build_file_generation/MODULE.bazel b/examples/bzlmod_build_file_generation/MODULE.bazel
index 45a1318..d69dd7d 100644
--- a/examples/bzlmod_build_file_generation/MODULE.bazel
+++ b/examples/bzlmod_build_file_generation/MODULE.bazel
@@ -48,38 +48,17 @@ python = use_extension("@rules_python//python/extensions:python.bzl", "python")
# We also use the same name for python.host_python_interpreter.
PYTHON_NAME = "python"
-PYTHON_TOOLCHAINS = PYTHON_NAME + "_toolchains"
-
INTERPRETER_NAME = "interpreter"
# We next initialize the python toolchain using the extension.
# You can set different Python versions in this block.
python.toolchain(
- # This name is used in the various use_repo statements
- # below, and in the local extension that is in this
- # example.
name = PYTHON_NAME,
configure_coverage_tool = True,
is_default = True,
python_version = "3.9",
)
-# Import the python repositories generated by the given module extension
-# into the scope of the current module.
-# All of the python3 repositories use the PYTHON_NAME as there prefix. They
-# are not catenated for ease of reading.
-use_repo(
- python,
- PYTHON_NAME,
- PYTHON_TOOLCHAINS,
-)
-
-# Register an already-defined toolchain so that Bazel can use it during
-# toolchain resolution.
-register_toolchains(
- "@{}//:all".format(PYTHON_TOOLCHAINS),
-)
-
# The interpreter extension discovers the platform specific Python binary.
# It creates a symlink to the binary, and we pass the label to the following
# pip.parse call.
commit 60c61e51646f610e7886d6bc1eaddb154f80ad44
Author: Chris Love <[email protected]>
Date: Fri May 26 14:20:35 2023 -0600
feat(bzlmod): Allowing multiple python.toolchain extension calls (#1230)
We do this work for two reasons.
First, we must support Module dependencies and sub-modules using
`python.toolchain`. There are already two known instances of sub-modules
setting up a Python toolchain and colliding with another module
(nanobind and rules_testing both run into this).
Second, the upcoming multi-version support is going to work by having
each `python.toolchain()` call register its particular version with the
extra toolchain constraint. This also helps unify the version-aware and
non-version-aware code paths (the non-version aware paths are just
version-aware with a single version registered as the default)
This commit implements various business logic in the toolchain class.
Toolchains in Sub Modules
It will create a toolchain in a sub-module if the toolchain
of the same name does not exist in the root module. The extension stops
name
clashing between toolchains in the root module and sub-modules.
You cannot configure more than one toolchain as the default toolchain.
Toolchain set as the default version.
This extension will not create a toolchain in a sub-module
if the sub-module toolchain is marked as the default version. If you
have
more than one toolchain in your root module, you need to set one of the
toolchains as the default version. If there is only one toolchain, it
is set as the default toolchain.
See #1229 for more information
diff --git a/examples/bzlmod_build_file_generation/MODULE.bazel b/examples/bzlmod_build_file_generation/MODULE.bazel
index 179fe1b..45a1318 100644
--- a/examples/bzlmod_build_file_generation/MODULE.bazel
+++ b/examples/bzlmod_build_file_generation/MODULE.bazel
@@ -46,7 +46,11 @@ python = use_extension("@rules_python//python/extensions:python.bzl", "python")
# This name is passed into python.toolchain and it's use_repo statement.
# We also use the same name for python.host_python_interpreter.
-PYTHON_NAME = "python3"
+PYTHON_NAME = "python"
+
+PYTHON_TOOLCHAINS = PYTHON_NAME + "_toolchains"
+
+INTERPRETER_NAME = "interpreter"
# We next initialize the python toolchain using the extension.
# You can set different Python versions in this block.
@@ -56,6 +60,7 @@ python.toolchain(
# example.
name = PYTHON_NAME,
configure_coverage_tool = True,
+ is_default = True,
python_version = "3.9",
)
@@ -63,12 +68,16 @@ python.toolchain(
# into the scope of the current module.
# All of the python3 repositories use the PYTHON_NAME as there prefix. They
# are not catenated for ease of reading.
-use_repo(python, PYTHON_NAME, "python3_toolchains")
+use_repo(
+ python,
+ PYTHON_NAME,
+ PYTHON_TOOLCHAINS,
+)
# Register an already-defined toolchain so that Bazel can use it during
# toolchain resolution.
register_toolchains(
- "@python3_toolchains//:all",
+ "@{}//:all".format(PYTHON_TOOLCHAINS),
)
# The interpreter extension discovers the platform specific Python binary.
@@ -76,10 +85,10 @@ register_toolchains(
# pip.parse call.
interpreter = use_extension("@rules_python//python/extensions:interpreter.bzl", "interpreter")
interpreter.install(
- name = "interpreter_python3",
+ name = INTERPRETER_NAME,
python_name = PYTHON_NAME,
)
-use_repo(interpreter, "interpreter_python3")
+use_repo(interpreter, INTERPRETER_NAME)
# Use the extension, pip.parse, to call the `pip_repository` rule that invokes
# `pip`, with `incremental` set. The pip call accepts a locked/compiled
@@ -102,7 +111,7 @@ pip.parse(
# is used for both resolving dependencies and running tests/binaries.
# If this isn't specified, then you'll get whatever is locally installed
# on your system.
- python_interpreter_target = "@interpreter_python3//:python",
+ python_interpreter_target = "@{}//:python".format(INTERPRETER_NAME),
requirements_lock = "//:requirements_lock.txt",
requirements_windows = "//:requirements_windows.txt",
)
commit 46537cf32f8bf722a6be805821cb2ee17d7b936a
Author: Chris Love <[email protected]>
Date: Mon May 15 16:04:40 2023 -0600
feat(bzlmod)!: Move each bzlmod extension into its own file (#1226)
This commit refactors the files that contain the bzlmod
extensions.
- All extensions are moved under the new extensions folder
- Private extensions are moved under extensions/private
- All extension files are renamed to remove the _extension suffix
- pip and internal_deps extensions are moved to their own file
This commit organizes the extensions better and also follows the
best practice of having a single extension per file. Having each
extension in its own file allows them to use some additional features
while helping avoid backwards incompatible changes.
## BREAKING CHANGES
This splits `//python:extensions.bzl`, which previously held the
`python`
and `pip` extensions, into separate files (`python.bzl` and `pip.bzl`,
respectively). Unfortunately, moving the location of the extensions is a
breaking change due to how bzlmod extension identity works (see
https://bazel.build/external/extension#extension_identity). Fortunately,
by moving to one extension per file, we shouldn't have to ever do this
again.
Users must update the file path in their `use_repo()` statements as
follows:
* `use_extension("@rules_python//python:extensions.bzl", "python")` ->
`use_extension("@rules_python//python/extensions:python.bzl", "python")`
* `use_extension("@rules_python//python:extensions.bzl", "pip")` ->
`use_extension("@rules_python//python/extensions:pip.bzl", "pip")`
The following `sed` commands should approximate the necessary changes:
```
sed 'sXuse_extension("@rules_python//python:extensions.bzl", "python")Xuse_extension("@rules_python//python/extensions:python.bzl", "python")X'`
sed 'sXuse_extension("@rules_python//python:extensions.bzl", "pip")Xuse_extension("@rules_python//python/extensions:pip.bzl", "pip")X'`
```
See `examples/bzlmod_build_file_generation/MODULE.bazel` for an example
of the new paths.
diff --git a/examples/bzlmod_build_file_generation/MODULE.bazel b/examples/bzlmod_build_file_generation/MODULE.bazel
index d59fbb3..179fe1b 100644
--- a/examples/bzlmod_build_file_generation/MODULE.bazel
+++ b/examples/bzlmod_build_file_generation/MODULE.bazel
@@ -42,7 +42,7 @@ bazel_dep(name = "gazelle", version = "0.30.0", repo_name = "bazel_gazelle")
# The following stanze returns a proxy object representing a module extension;
# its methods can be invoked to create module extension tags.
-python = use_extension("@rules_python//python:extensions.bzl", "python")
+python = use_extension("@rules_python//python/extensions:python.bzl", "python")
# This name is passed into python.toolchain and it's use_repo statement.
# We also use the same name for python.host_python_interpreter.
@@ -74,7 +74,7 @@ register_toolchains(
# The interpreter extension discovers the platform specific Python binary.
# It creates a symlink to the binary, and we pass the label to the following
# pip.parse call.
-interpreter = use_extension("@rules_python//python:interpreter_extension.bzl", "interpreter")
+interpreter = use_extension("@rules_python//python/extensions:interpreter.bzl", "interpreter")
interpreter.install(
name = "interpreter_python3",
python_name = PYTHON_NAME,
@@ -88,7 +88,7 @@ use_repo(interpreter, "interpreter_python3")
# You can instead check this `requirements.bzl` file into your repo.
# Because this project has different requirements for windows vs other
# operating systems, we have requirements for each.
-pip = use_extension("@rules_python//python:extensions.bzl", "pip")
+pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
name = "pip",
# When using gazelle you must use set the following flag
commit ccea92a3ad6f9204a172d306a6b1c4cb18e41cee
Author: Chris Love <[email protected]>
Date: Mon May 15 10:24:43 2023 -0600
feat(bzlmod): Cleaning up interpreter resolution (#1218)
This commit cleans up the use of "canonical resolution" of
the Python interpreter. When the extension toolchains run
it collects a list of the interpreters and then
uses the hub_repo rule to create a map of names and the
interpreter labels.
Next, we then use the interpreter_extension that, creates
reports that have symlinks pointing to the different interpreter
binaries.
The user can then pass in a label to the pip call for the
specific hermetic interpreter.
diff --git a/examples/bzlmod_build_file_generation/MODULE.bazel b/examples/bzlmod_build_file_generation/MODULE.bazel
index 781b0cb..d59fbb3 100644
--- a/examples/bzlmod_build_file_generation/MODULE.bazel
+++ b/examples/bzlmod_build_file_generation/MODULE.bazel
@@ -48,9 +48,6 @@ python = use_extension("@rules_python//python:extensions.bzl", "python")
# We also use the same name for python.host_python_interpreter.
PYTHON_NAME = "python3"
-# This is the name that is used for the host interpreter
-PYTHON_INTERPRETER = PYTHON_NAME + "_host_interpreter"
-
# We next initialize the python toolchain using the extension.
# You can set different Python versions in this block.
python.toolchain(
@@ -66,37 +63,46 @@ python.toolchain(
# into the scope of the current module.
# All of the python3 repositories use the PYTHON_NAME as there prefix. They
# are not catenated for ease of reading.
-use_repo(python, PYTHON_NAME)
-use_repo(python, "python3_toolchains")
-use_repo(python, PYTHON_INTERPRETER)
+use_repo(python, PYTHON_NAME, "python3_toolchains")
-# Register an already-defined toolchain so that Bazel can use it during toolchain resolution.
+# Register an already-defined toolchain so that Bazel can use it during
+# toolchain resolution.
register_toolchains(
"@python3_toolchains//:all",
)
-# Use the pip extension
-pip = use_extension("@rules_python//python:extensions.bzl", "pip")
+# The interpreter extension discovers the platform specific Python binary.
+# It creates a symlink to the binary, and we pass the label to the following
+# pip.parse call.
+interpreter = use_extension("@rules_python//python:interpreter_extension.bzl", "interpreter")
+interpreter.install(
+ name = "interpreter_python3",
+ python_name = PYTHON_NAME,
+)
+use_repo(interpreter, "interpreter_python3")
-# Use the extension to call the `pip_repository` rule that invokes `pip`, with `incremental` set.
-# Accepts a locked/compiled requirements file and installs the dependencies listed within.
+# Use the extension, pip.parse, to call the `pip_repository` rule that invokes
+# `pip`, with `incremental` set. The pip call accepts a locked/compiled
+# requirements file and installs the dependencies listed within.
# Those dependencies become available in a generated `requirements.bzl` file.
# You can instead check this `requirements.bzl` file into your repo.
# Because this project has different requirements for windows vs other
# operating systems, we have requirements for each.
+pip = use_extension("@rules_python//python:extensions.bzl", "pip")
pip.parse(
name = "pip",
# When using gazelle you must use set the following flag
# in order for the generation of gazelle dependency resolution.
incompatible_generate_aliases = True,
- # The interpreter attribute points to the interpreter to use for running
- # pip commands to download the packages in the requirements file.
+ # The interpreter_target attribute points to the interpreter to
+ # use for running pip commands to download the packages in the
+ # requirements file.
# As a best practice, we use the same interpreter as the toolchain
# that was configured above; this ensures the same Python version
# is used for both resolving dependencies and running tests/binaries.
# If this isn't specified, then you'll get whatever is locally installed
# on your system.
- python_interpreter_target = "@" + PYTHON_INTERPRETER + "//:python",
+ python_interpreter_target = "@interpreter_python3//:python",
requirements_lock = "//:requirements_lock.txt",
requirements_windows = "//:requirements_windows.txt",
)
commit 0912bba35d0f3e341c7beae206b21e9ca5b8322e
Author: Chris Love <[email protected]>
Date: Thu May 4 14:08:34 2023 -0600
feat(bzlmod): expose platform-agnostic repo target for toolchain interpreter (#1155)
This exposes a new repo and target, `@{name}_host_interpreter//:python`,
created by `python.toolchain()`, that points to the host OS's Python
interpreter for that particular toolchain.
This solves two problems:
1. `pip.parse()` can now refer to the same interpreter used in the
toolchains
2. There is now a canonical, public, way to refer to the host OS Python
interpreter
for repository rules.
The above were _sort of_ possible for users to do already, but it
required them to write much more configuration and extension code to do
so. This moves that sort of boilerplate into our code so they have a
simpler configuration.
Also:
- removing bzlmod support in the build_file_generation example; making
examples work
with both WORKSPACE and MODULE is a pain, hence splitting them.
- adding an example of bzlmod and gazelle
- improved documentation in the pip arguments
Closes: https://github.com/bazelbuild/rules_python/issues/1161
diff --git a/examples/bzlmod_build_file_generation/MODULE.bazel b/examples/bzlmod_build_file_generation/MODULE.bazel
new file mode 100644
index 0000000..781b0cb
--- /dev/null
+++ b/examples/bzlmod_build_file_generation/MODULE.bazel
@@ -0,0 +1,113 @@
+# This file replaces the WORKSPACE file when using bzlmod.
+
+# module declares certain properties of the Bazel module represented by the current Bazel repo.
+# These properties are either essential metadata of the module (such as the name and version),
+# or affect behavior of the current module and its dependents.
+module(
+ name = "example_bzlmod_build_file_generation",
+ version = "0.0.0",
+ compatibility_level = 1,
+)
+
+# The following stanza defines the dependency rules_python.
+# For typical setups you set the version.
+# See the releases page for available versions.
+# https://github.com/bazelbuild/rules_python/releases
+bazel_dep(name = "rules_python", version = "0.0.0")
+
+# The following loads rules_python from the file system.
+# For usual setups you should remove this local_path_override block.
+local_path_override(
+ module_name = "rules_python",
+ path = "../..",
+)
+
+# The following stanza defines the dependency rules_python_gazelle_plugin.
+# For typical setups you set the version.
+# See the releases page for available versions.
+# https://github.com/bazelbuild/rules_python/releases
+bazel_dep(name = "rules_python_gazelle_plugin", version = "0.0.0")
+
+# The following starlark loads the gazelle plugin from the file system.
+# For usual setups you should remove this local_path_override block.
+local_path_override(
+ module_name = "rules_python_gazelle_plugin",
+ path = "../../gazelle",
+)
+
+# The following stanza defines the dependency for gazelle
+# See here https://github.com/bazelbuild/bazel-gazelle/releases/ for the
+# latest version.
+bazel_dep(name = "gazelle", version = "0.30.0", repo_name = "bazel_gazelle")
+
+# The following stanze returns a proxy object representing a module extension;
+# its methods can be invoked to create module extension tags.
+python = use_extension("@rules_python//python:extensions.bzl", "python")
+
+# This name is passed into python.toolchain and it's use_repo statement.
+# We also use the same name for python.host_python_interpreter.
+PYTHON_NAME = "python3"
+
+# This is the name that is used for the host interpreter
+PYTHON_INTERPRETER = PYTHON_NAME + "_host_interpreter"
+
+# We next initialize the python toolchain using the extension.
+# You can set different Python versions in this block.
+python.toolchain(
+ # This name is used in the various use_repo statements
+ # below, and in the local extension that is in this
+ # example.
+ name = PYTHON_NAME,
+ configure_coverage_tool = True,
+ python_version = "3.9",
+)
+
+# Import the python repositories generated by the given module extension
+# into the scope of the current module.
+# All of the python3 repositories use the PYTHON_NAME as there prefix. They
+# are not catenated for ease of reading.
+use_repo(python, PYTHON_NAME)
+use_repo(python, "python3_toolchains")
+use_repo(python, PYTHON_INTERPRETER)
+
+# Register an already-defined toolchain so that Bazel can use it during toolchain resolution.
+register_toolchains(
+ "@python3_toolchains//:all",
+)
+
+# Use the pip extension
+pip = use_extension("@rules_python//python:extensions.bzl", "pip")
+
+# Use the extension to call the `pip_repository` rule that invokes `pip`, with `incremental` set.
+# Accepts a locked/compiled requirements file and installs the dependencies listed within.
+# Those dependencies become available in a generated `requirements.bzl` file.
+# You can instead check this `requirements.bzl` file into your repo.
+# Because this project has different requirements for windows vs other
+# operating systems, we have requirements for each.
+pip.parse(
+ name = "pip",
+ # When using gazelle you must use set the following flag
+ # in order for the generation of gazelle dependency resolution.
+ incompatible_generate_aliases = True,
+ # The interpreter attribute points to the interpreter to use for running
+ # pip commands to download the packages in the requirements file.
+ # As a best practice, we use the same interpreter as the toolchain
+ # that was configured above; this ensures the same Python version
+ # is used for both resolving dependencies and running tests/binaries.
+ # If this isn't specified, then you'll get whatever is locally installed
+ # on your system.
+ python_interpreter_target = "@" + PYTHON_INTERPRETER + "//:python",
+ requirements_lock = "//:requirements_lock.txt",
+ requirements_windows = "//:requirements_windows.txt",
+)
+
+# Imports the pip toolchain generated by the given module extension into the scope of the current module.
+use_repo(pip, "pip")
+
+# This project includes a different module that is on the local file system.
+# Add the module to this parent project.
+bazel_dep(name = "other_module", version = "", repo_name = "our_other_module")
+local_path_override(
+ module_name = "other_module",
+ path = "other_module",
+)
commit 9dd944e963807b02b21489ad286715b60aec8c84
Author: Ignas Anikevicius <[email protected]>
Date: Tue Jul 11 02:19:09 2023 +0900
feat(gazelle): support multiple requirements files in manifest generation (#1301)
For certain workflows it is useful to calculate the integrity hash of
the manifest file based on a number of requirements files. The
requirements locking is usually done by executing a script on each
platform and having gazelle manifest generator be aware that more than
one requirements file may affect the outcome (e.g. the wheels that get
passed to modules map may come from multi_pip_parse rule) is generally
useful.
This change modifies the generation macro to concatenate the
requirements
files into one before passing it to the manifest generator.
diff --git a/examples/bzlmod_build_file_generation/BUILD.bazel b/examples/bzlmod_build_file_generation/BUILD.bazel
index 498969b..c5e27c2 100644
--- a/examples/bzlmod_build_file_generation/BUILD.bazel
+++ b/examples/bzlmod_build_file_generation/BUILD.bazel
@@ -49,7 +49,10 @@ gazelle_python_manifest(
name = "gazelle_python_manifest",
modules_mapping = ":modules_map",
pip_repository_name = "pip",
- requirements = "//:requirements_lock.txt",
+ requirements = [
+ "//:requirements_lock.txt",
+ "//:requirements_windows.txt",
+ ],
# NOTE: we can use this flag in order to make our setup compatible with
# bzlmod.
use_pip_repository_aliases = True,
commit 148622aa92fdd5afcaf9f153d7bb6afce713e553
Author: Chris Love <[email protected]>
Date: Wed May 31 19:48:28 2023 -0600
feat(bzlmod): Moving register.toolchains internal (#1238)
This commit moves the register.toolchains bzlmod call to inside
of rules_python. Instead of a user having to call register.toolchains
in their MODULE.bazel, rules_python/MODULE.bazel calls it
on the internal hub.
This is a breaking change if you are using register.toolchains inside
of submodules. Using register.toolchains inside of submodules is
not recommended anyways. This is now broken because we are not
creating a repo for every Python version toolchain. All of the
toochain calls exist now in the hub's repo BUILD.bazel file.
diff --git a/examples/bzlmod_build_file_generation/BUILD.bazel b/examples/bzlmod_build_file_generation/BUILD.bazel
index 05a15cc..498969b 100644
--- a/examples/bzlmod_build_file_generation/BUILD.bazel
+++ b/examples/bzlmod_build_file_generation/BUILD.bazel
@@ -7,7 +7,6 @@
# requirements.
load("@bazel_gazelle//:def.bzl", "gazelle")
load("@pip//:requirements.bzl", "all_whl_requirements")
-load("@python//:defs.bzl", py_test_with_transition = "py_test")
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
load("@rules_python_gazelle_plugin//:def.bzl", "GAZELLE_PYTHON_RUNTIME_DEPS")
@@ -72,13 +71,6 @@ gazelle(
gazelle = "@rules_python_gazelle_plugin//python:gazelle_binary",
)
-py_test_with_transition(
- name = "test_with_transition",
- srcs = ["__test__.py"],
- main = "__test__.py",
- deps = [":bzlmod_build_file_generation"],
-)
-
# The following targets are created and maintained by gazelle
py_library(
name = "bzlmod_build_file_generation",
commit 60c61e51646f610e7886d6bc1eaddb154f80ad44
Author: Chris Love <[email protected]>
Date: Fri May 26 14:20:35 2023 -0600
feat(bzlmod): Allowing multiple python.toolchain extension calls (#1230)
We do this work for two reasons.
First, we must support Module dependencies and sub-modules using
`python.toolchain`. There are already two known instances of sub-modules
setting up a Python toolchain and colliding with another module
(nanobind and rules_testing both run into this).
Second, the upcoming multi-version support is going to work by having
each `python.toolchain()` call register its particular version with the
extra toolchain constraint. This also helps unify the version-aware and
non-version-aware code paths (the non-version aware paths are just
version-aware with a single version registered as the default)
This commit implements various business logic in the toolchain class.
Toolchains in Sub Modules
It will create a toolchain in a sub-module if the toolchain
of the same name does not exist in the root module. The extension stops
name
clashing between toolchains in the root module and sub-modules.
You cannot configure more than one toolchain as the default toolchain.
Toolchain set as the default version.
This extension will not create a toolchain in a sub-module
if the sub-module toolchain is marked as the default version. If you
have
more than one toolchain in your root module, you need to set one of the
toolchains as the default version. If there is only one toolchain, it
is set as the default toolchain.
See #1229 for more information
diff --git a/examples/bzlmod_build_file_generation/BUILD.bazel b/examples/bzlmod_build_file_generation/BUILD.bazel
index c667f1e..05a15cc 100644
--- a/examples/bzlmod_build_file_generation/BUILD.bazel
+++ b/examples/bzlmod_build_file_generation/BUILD.bazel
@@ -7,7 +7,7 @@
# requirements.
load("@bazel_gazelle//:def.bzl", "gazelle")
load("@pip//:requirements.bzl", "all_whl_requirements")
-load("@python3//:defs.bzl", py_test_with_transition = "py_test")
+load("@python//:defs.bzl", py_test_with_transition = "py_test")
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
load("@rules_python_gazelle_plugin//:def.bzl", "GAZELLE_PYTHON_RUNTIME_DEPS")
commit 0912bba35d0f3e341c7beae206b21e9ca5b8322e
Author: Chris Love <[email protected]>
Date: Thu May 4 14:08:34 2023 -0600
feat(bzlmod): expose platform-agnostic repo target for toolchain interpreter (#1155)
This exposes a new repo and target, `@{name}_host_interpreter//:python`,
created by `python.toolchain()`, that points to the host OS's Python
interpreter for that particular toolchain.
This solves two problems:
1. `pip.parse()` can now refer to the same interpreter used in the
toolchains
2. There is now a canonical, public, way to refer to the host OS Python
interpreter
for repository rules.
The above were _sort of_ possible for users to do already, but it
required them to write much more configuration and extension code to do
so. This moves that sort of boilerplate into our code so they have a
simpler configuration.
Also:
- removing bzlmod support in the build_file_generation example; making
examples work
with both WORKSPACE and MODULE is a pain, hence splitting them.
- adding an example of bzlmod and gazelle
- improved documentation in the pip arguments
Closes: https://github.com/bazelbuild/rules_python/issues/1161
diff --git a/examples/bzlmod_build_file_generation/BUILD.bazel b/examples/bzlmod_build_file_generation/BUILD.bazel
new file mode 100644
index 0000000..c667f1e
--- /dev/null
+++ b/examples/bzlmod_build_file_generation/BUILD.bazel
@@ -0,0 +1,103 @@
+# Load various rules so that we can have bazel download
+# various rulesets and dependencies.
+# The `load` statement imports the symbol for the rule, in the defined
+# ruleset. When the symbol is loaded you can use the rule.
+
+# The following code loads the base python requirements and gazelle
+# requirements.
+load("@bazel_gazelle//:def.bzl", "gazelle")
+load("@pip//:requirements.bzl", "all_whl_requirements")
+load("@python3//:defs.bzl", py_test_with_transition = "py_test")
+load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
+load("@rules_python//python:pip.bzl", "compile_pip_requirements")
+load("@rules_python_gazelle_plugin//:def.bzl", "GAZELLE_PYTHON_RUNTIME_DEPS")
+load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest")
+load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping")
+
+# This stanza calls a rule that generates targets for managing pip dependencies
+# with pip-compile.
+compile_pip_requirements(
+ name = "requirements",
+ extra_args = ["--allow-unsafe"],
+ requirements_in = "requirements.in",
+ requirements_txt = "requirements_lock.txt",
+ requirements_windows = "requirements_windows.txt",
+)
+
+# This repository rule fetches the metadata for python packages we
+# depend on. That data is required for the gazelle_python_manifest
+# rule to update our manifest file.
+modules_mapping(
+ name = "modules_map",
+ exclude_patterns = [
+ "^_|(\\._)+", # This is the default.
+ "(\\.tests)+", # Add a custom one to get rid of the psutil tests.
+ ],
+ wheels = all_whl_requirements,
+)
+
+# Gazelle python extension needs a manifest file mapping from
+# an import to the installed package that provides it.
+# This macro produces two targets:
+# - //:gazelle_python_manifest.update can be used with `bazel run`
+# to recalculate the manifest
+# - //:gazelle_python_manifest.test is a test target ensuring that
+# the manifest doesn't need to be updated
+# This target updates a file called gazelle_python.yaml, and
+# requires that file exist before the target is run.
+# When you are using gazelle you need to run this target first.
+gazelle_python_manifest(
+ name = "gazelle_python_manifest",
+ modules_mapping = ":modules_map",
+ pip_repository_name = "pip",
+ requirements = "//:requirements_lock.txt",
+ # NOTE: we can use this flag in order to make our setup compatible with
+ # bzlmod.
+ use_pip_repository_aliases = True,
+)
+
+# Our gazelle target points to the python gazelle binary.
+# This is the simple case where we only need one language supported.
+# If you also had proto, go, or other gazelle-supported languages,
+# you would also need a gazelle_binary rule.
+# See https://github.com/bazelbuild/bazel-gazelle/blob/master/extend.rst#example
+# This is the primary gazelle target to run, so that you can update BUILD.bazel files.
+# You can execute:
+# - bazel run //:gazelle update
+# - bazel run //:gazelle fix
+# See: https://github.com/bazelbuild/bazel-gazelle#fix-and-update
+gazelle(
+ name = "gazelle",
+ data = GAZELLE_PYTHON_RUNTIME_DEPS,
+ gazelle = "@rules_python_gazelle_plugin//python:gazelle_binary",
+)
+
+py_test_with_transition(
+ name = "test_with_transition",
+ srcs = ["__test__.py"],
+ main = "__test__.py",
+ deps = [":bzlmod_build_file_generation"],
+)
+
+# The following targets are created and maintained by gazelle
+py_library(
+ name = "bzlmod_build_file_generation",
+ srcs = ["lib.py"],
+ visibility = ["//:__subpackages__"],
+ deps = ["@pip//tabulate"],
+)
+
+py_binary(
+ name = "bzlmod_build_file_generation_bin",
+ srcs = ["__main__.py"],
+ main = "__main__.py",
+ visibility = ["//:__subpackages__"],
+ deps = [":bzlmod_build_file_generation"],
+)
+
+py_test(
+ name = "bzlmod_build_file_generation_test",
+ srcs = ["__test__.py"],
+ main = "__test__.py",
+ deps = [":bzlmod_build_file_generation"],
+)
commit e5d9f10243f546d3796e2daa74266e3a13158d40
Author: Chris Love <[email protected]>
Date: Mon Jul 10 17:18:05 2023 -0600
feat(bzlmod): Allow bzlmod pip.parse to reference the default python toolchain and interpreter (#1303)
This commit defaults `the pip.parse` `python_version` attribute to the
default version of
Python, as configured by the `python.toolchain` extension. This allows a
user to use the
Python version set by rules_python or the root module. Also, this makes
setting the
attribute optional (as it has a default) and we automatically select the
interpreter.
Closes #1267
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index be9466d..df88ae8 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -11,6 +11,10 @@ local_path_override(
path = "../..",
)
+# Setting python.toolchain is optional as rules_python
+# sets a toolchain for you, using the latest supported version
+# of Python. We do recomend that you set a version here.
+
# We next initialize the python toolchain using the extension.
# You can set different Python versions in this block.
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
@@ -87,9 +91,10 @@ use_repo(pip, "whl_mods_hub")
# call.
# Alternatively, `python_interpreter_target` can be used to directly specify
# the Python interpreter to run to resolve dependencies.
+# Because we do not have a python_version defined here
+# pip.parse uses the python toolchain that is set as default.
pip.parse(
hub_name = "pip",
- python_version = "3.9",
requirements_lock = "//:requirements_lock_3_9.txt",
requirements_windows = "//:requirements_windows_3_9.txt",
# These modifications were created above and we
commit 523b9de1e9e8b1fc8cbfcb530ee8287bef13a736
Author: Chris Love <[email protected]>
Date: Mon Jul 10 16:11:35 2023 -0600
fix(bzlmod)!: Changing repository name "python_aliases" to "python_versions" (#1304)
I think this name is more informative for a public API. The
functionality it exposes are rules/macros that use a specific Python
version to be used. These aren't really aliases.
This commit renames "python_aliases" to "python_versions". This isn't
technically a breaking
change because bzlmod support is still beta, but we'll flag it as such
just in case.
BREAKING CHANGE:
* The `python_aliases` repo is renamed to `python_versions`. You will
need to either
update references from `@python_aliases` to `@python_versions`, or use
repo-remapping
to alias the old name (`use_repo(python,
python_aliases="python_versions")`)
Closes #1273
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index 96b05be..be9466d 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -35,7 +35,7 @@ python.toolchain(
# See the tests folder for various examples on using multiple Python versions.
# The names "python_3_9" and "python_3_10" are autmatically created by the repo
# rules based on the `python_version` arg values.
-use_repo(python, "python_3_10", "python_3_9", "python_aliases")
+use_repo(python, "python_3_10", "python_3_9", "python_versions")
# This extension allows a user to create modifications to how rules_python
# creates different wheel repositories. Different attributes allow the user
commit 42b72dbd6ea34753a43b3dd89ffff2520a978099
Author: Chris Love <[email protected]>
Date: Mon Jul 10 09:55:46 2023 -0600
feat(bzlmod): Implementing wheel annotations via whl_mods (#1278)
This commit implements a bzlmod extension that allows users to create
"annotations" for wheel builds. The wheel_builder.py accepts a JSON file
via a parameter called annotations; this extension creates those JSON
files. The pip extension accepts a Label -> String dict argument of the
JSON files.
This feature is renamed to `whl_mods` because the JSON files are handled
differently
and the name "annotations" is uninformative. This modifies the creation
of the BUILD
files and their content, and is much more than just adding some notes
about a whl.
The whl_mod extension wheel names and the wheel names in pip must match.
Closes: https://github.com/bazelbuild/rules_python/issues/1213
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index a61e094..96b05be 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -37,8 +37,50 @@ python.toolchain(
# rules based on the `python_version` arg values.
use_repo(python, "python_3_10", "python_3_9", "python_aliases")
+# This extension allows a user to create modifications to how rules_python
+# creates different wheel repositories. Different attributes allow the user
+# to modify the BUILD file, and copy files.
+# See @rules_python//python/extensions:whl_mods.bzl attributes for more information
+# on each of the attributes.
+# You are able to set a hub name, so that you can have different modifications of the same
+# wheel in different pip hubs.
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
+# Call whl_mods.create for the requests package.
+pip.whl_mods(
+ # we are using the appended_build_content.BUILD file
+ # to add content to the request wheel BUILD file.
+ additive_build_content_file = "//whl_mods:appended_build_content.BUILD",
+ data = [":generated_file"],
+ hub_name = "whl_mods_hub",
+ whl_name = "requests",
+)
+
+ADDITIVE_BUILD_CONTENT = """\
+load("@bazel_skylib//rules:write_file.bzl", "write_file")
+write_file(
+ name = "generated_file",
+ out = "generated_file.txt",
+ content = ["Hello world from build content file"],
+)
+"""
+
+# Call whl_mods.create for the wheel package.
+pip.whl_mods(
+ additive_build_content = ADDITIVE_BUILD_CONTENT,
+ copy_executables = {
+ "@@//whl_mods:data/copy_executable.py": "copied_content/executable.py",
+ },
+ copy_files = {
+ "@@//whl_mods:data/copy_file.txt": "copied_content/file.txt",
+ },
+ data = [":generated_file"],
+ data_exclude_glob = ["site-packages/*.dist-info/WHEEL"],
+ hub_name = "whl_mods_hub",
+ whl_name = "wheel",
+)
+use_repo(pip, "whl_mods_hub")
+
# To fetch pip dependencies, use pip.parse. We can pass in various options,
# but typically we pass requirements and the Python version. The Python
# version must have been configured by a corresponding `python.toolchain()`
@@ -50,12 +92,26 @@ pip.parse(
python_version = "3.9",
requirements_lock = "//:requirements_lock_3_9.txt",
requirements_windows = "//:requirements_windows_3_9.txt",
+ # These modifications were created above and we
+ # are providing pip.parse with the label of the mod
+ # and the name of the wheel.
+ whl_modifications = {
+ "@whl_mods_hub//:requests.json": "requests",
+ "@whl_mods_hub//:wheel.json": "wheel",
+ },
)
pip.parse(
hub_name = "pip",
python_version = "3.10",
requirements_lock = "//:requirements_lock_3_10.txt",
requirements_windows = "//:requirements_windows_3_10.txt",
+ # These modifications were created above and we
+ # are providing pip.parse with the label of the mod
+ # and the name of the wheel.
+ whl_modifications = {
+ "@whl_mods_hub//:requests.json": "requests",
+ "@whl_mods_hub//:wheel.json": "wheel",
+ },
)
# NOTE: The pip_39 repo is only used because the plain `@pip` repo doesn't
commit cc435943e92a32200e7620e234f058cc828d6f6e
Author: Richard Levasseur <[email protected]>
Date: Tue Jun 20 12:58:30 2023 -0700
cleanup: Typos, doc, and formatting cleanup in pip extension et al (#1275)
* Corrects some typos in docs
* Expands the user-facing documentation
* Fixes errors having newlines in the middle of them
* Renames some internal functions to be more self-describing.
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index e09e029..a61e094 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -34,16 +34,17 @@ python.toolchain(
# You only need to load this repositories if you are using multiple Python versions.
# See the tests folder for various examples on using multiple Python versions.
# The names "python_3_9" and "python_3_10" are autmatically created by the repo
-# rules based on the python_versions.
+# rules based on the `python_version` arg values.
use_repo(python, "python_3_10", "python_3_9", "python_aliases")
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
-# For each pip setup we call pip.parse. We can pass in various options
-# but typically we are passing in a requirements and an interpreter.
-# If you provide the python_version we will attempt to determine
-# the interpreter target automatically. Otherwise use python_interpreter_target
-# to override the lookup.
+# To fetch pip dependencies, use pip.parse. We can pass in various options,
+# but typically we pass requirements and the Python version. The Python
+# version must have been configured by a corresponding `python.toolchain()`
+# call.
+# Alternatively, `python_interpreter_target` can be used to directly specify
+# the Python interpreter to run to resolve dependencies.
pip.parse(
hub_name = "pip",
python_version = "3.9",
@@ -57,7 +58,8 @@ pip.parse(
requirements_windows = "//:requirements_windows_3_10.txt",
)
-# we use the pip_39 repo because entry points are not yet supported.
+# NOTE: The pip_39 repo is only used because the plain `@pip` repo doesn't
+# yet support entry points; see https://github.com/bazelbuild/rules_python/issues/1262
use_repo(pip, "pip", "pip_39")
bazel_dep(name = "other_module", version = "", repo_name = "our_other_module")
commit 1c581242c25925a1bd6764608877ec170724f11d
Author: Chris Love <[email protected]>
Date: Thu Jun 15 10:58:54 2023 -0600
feat(bzlmod)!: Calling pip multiple times allowing for multiple Python versions (#1254)
Implementing the capability to call pip bzlmod extension multiple times.
We can now call the pip extension with the same hub name multiple times.
This allows a user to have multiple different requirement files based on
the Python version.
With workspace, we need incompatible aliases because you get
@pip//tabulate or @pip_tabulate//:pkg.
The requirement() macro hides this, but changing the flag becomes a
breaking change if you've manually referenced things. With bzlmod,
though, the @pip_tabulate style isn't a realistic option (you'd have to
use_repo everything, which is a huge pain).
So we have chosen to have `@pip//tabulate`.
This commit implements that capability for pip.parse to determine the
Python version from programmatically
the provided interpreter.
See
https://github.com/bazelbuild/rules_python/blob/76aab0f91bd614ee1b2ade310baf28c38695c522/python/extensions/pip.bzl#L88
For more in-depth implementation details.
INTERFACE CHANGE::
- pip.parse argument python_version is introduced but not required. When
possible, it is
inferred.
BREAKING CHANGE:
* `pip.parse()` extension: the `name` attribute is renamed `hub_name`.
This is to reflect
that the user-provided name isn't unique to each `pip.parse()` call. We
now have a hub that is created, and each
pip.parse call creates a spoke.
* `pip.parse()` extension: the `incompatible_generate_aliases` arg is
removed; behavior
has changed to assume it is always True.
* `pip.parse()` calls are collected under the same `hub_name` to support
multiple Python versions under the same resulting repo name (the hub
name0.
Close: #1229
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index 40dfb6a..e09e029 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -11,16 +11,6 @@ local_path_override(
path = "../..",
)
-# This name is generated by python.toolchain(), and is later passed
-# to use_repo() and interpreter.install().
-PYTHON_NAME_39 = "python_3_9"
-
-INTERPRETER_NAME_39 = "interpreter_39"
-
-PYTHON_NAME_310 = "python_3_10"
-
-INTERPRETER_NAME_310 = "interpreter_310"
-
# We next initialize the python toolchain using the extension.
# You can set different Python versions in this block.
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
@@ -41,38 +31,34 @@ python.toolchain(
python_version = "3.10",
)
-# You only need to load this repositories if you are using muiltple Python versions.
-# See the tests folder for various examples.
-use_repo(python, PYTHON_NAME_39, "python_aliases")
+# You only need to load this repositories if you are using multiple Python versions.
+# See the tests folder for various examples on using multiple Python versions.
+# The names "python_3_9" and "python_3_10" are autmatically created by the repo
+# rules based on the python_versions.
+use_repo(python, "python_3_10", "python_3_9", "python_aliases")
-# The interpreter extension discovers the platform specific Python binary.
-# It creates a symlink to the binary, and we pass the label to the following
-# pip.parse call.
-interpreter = use_extension("@rules_python//python/extensions:interpreter.bzl", "interpreter")
-interpreter.install(
- name = INTERPRETER_NAME_39,
- python_name = PYTHON_NAME_39,
-)
+pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
-# This second call is only needed if you are using mulitple different
-# Python versions/interpreters.
-interpreter.install(
- name = INTERPRETER_NAME_310,
- python_name = PYTHON_NAME_310,
+# For each pip setup we call pip.parse. We can pass in various options
+# but typically we are passing in a requirements and an interpreter.
+# If you provide the python_version we will attempt to determine
+# the interpreter target automatically. Otherwise use python_interpreter_target
+# to override the lookup.
+pip.parse(
+ hub_name = "pip",
+ python_version = "3.9",
+ requirements_lock = "//:requirements_lock_3_9.txt",
+ requirements_windows = "//:requirements_windows_3_9.txt",
)
-use_repo(interpreter, INTERPRETER_NAME_39, INTERPRETER_NAME_310)
-
-pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
- name = "pip",
- # Intentionally set it false because the "true" case is already
- # covered by examples/bzlmod_build_file_generation
- incompatible_generate_aliases = False,
- python_interpreter_target = "@{}//:python".format(INTERPRETER_NAME_39),
- requirements_lock = "//:requirements_lock.txt",
- requirements_windows = "//:requirements_windows.txt",
+ hub_name = "pip",
+ python_version = "3.10",
+ requirements_lock = "//:requirements_lock_3_10.txt",
+ requirements_windows = "//:requirements_windows_3_10.txt",
)
-use_repo(pip, "pip")
+
+# we use the pip_39 repo because entry points are not yet supported.
+use_repo(pip, "pip", "pip_39")
bazel_dep(name = "other_module", version = "", repo_name = "our_other_module")
local_path_override(
commit b228f6047671abcf4c78ea7318916218fb098831
Author: Richard Levasseur <[email protected]>
Date: Thu Jun 8 07:50:20 2023 -0700
fix(bzlmod)!: Remove ability to specify toolchain repo name. (#1258)
The main reasons this is removed is because if modules choose different
names for the same toolchain, only one of the two toolchains (which are,
hopefully, identical) will be used. Which toolchain is used depends on
the module graph dependency ordering.
Furthermore, as of #1238, only one repo per version is created; others
are ignored. This means a module doesn't know if the name it chooses
will result in a repo being created with that name.
Instead, the toolchain repos are named by rules_python:
`python_{major}_{minor}`. These repo names are currently part of the
public API, since they end up referenced in MODULE config (to wire the
toolchain interpreter to pip).
BREAKING CHANGES
This removes the `name` arg from `python.toolchain()`. Users will need
to remove such usages from their `MODULE.bazel` and update their
`use_repo()` statements. If keeping the custom repo name is necessary,
then repo mappings can be used. See #1232 for additional migration
steps, commands, and information.
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index 24bb458..40dfb6a 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -11,13 +11,13 @@ local_path_override(
path = "../..",
)
-# This name is passed into python.toolchain and it's use_repo statement.
-# We also use the same value in the python.host_python_interpreter call.
-PYTHON_NAME_39 = "python_39"
+# This name is generated by python.toolchain(), and is later passed
+# to use_repo() and interpreter.install().
+PYTHON_NAME_39 = "python_3_9"
INTERPRETER_NAME_39 = "interpreter_39"
-PYTHON_NAME_310 = "python_310"
+PYTHON_NAME_310 = "python_3_10"
INTERPRETER_NAME_310 = "interpreter_310"
@@ -25,10 +25,6 @@ INTERPRETER_NAME_310 = "interpreter_310"
# You can set different Python versions in this block.
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
- # This name is used in the various use_repo statements
- # below, and in the local extension that is in this
- # example.
- name = PYTHON_NAME_39,
configure_coverage_tool = True,
# Only set when you have mulitple toolchain versions.
is_default = True,
@@ -41,7 +37,6 @@ python.toolchain(
# Note: we do not supporting using multiple pip extensions, this is
# work in progress.
python.toolchain(
- name = PYTHON_NAME_310,
configure_coverage_tool = True,
python_version = "3.10",
)
commit 148622aa92fdd5afcaf9f153d7bb6afce713e553
Author: Chris Love <[email protected]>
Date: Wed May 31 19:48:28 2023 -0600
feat(bzlmod): Moving register.toolchains internal (#1238)
This commit moves the register.toolchains bzlmod call to inside
of rules_python. Instead of a user having to call register.toolchains
in their MODULE.bazel, rules_python/MODULE.bazel calls it
on the internal hub.
This is a breaking change if you are using register.toolchains inside
of submodules. Using register.toolchains inside of submodules is
not recommended anyways. This is now broken because we are not
creating a repo for every Python version toolchain. All of the
toochain calls exist now in the hub's repo BUILD.bazel file.
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index bb4183b..24bb458 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -15,14 +15,10 @@ local_path_override(
# We also use the same value in the python.host_python_interpreter call.
PYTHON_NAME_39 = "python_39"
-PYTHON_39_TOOLCHAINS = PYTHON_NAME_39 + "_toolchains"
-
INTERPRETER_NAME_39 = "interpreter_39"
PYTHON_NAME_310 = "python_310"
-PYTHON_310_TOOLCHAINS = PYTHON_NAME_310 + "_toolchains"
-
INTERPRETER_NAME_310 = "interpreter_310"
# We next initialize the python toolchain using the extension.
@@ -50,25 +46,9 @@ python.toolchain(
python_version = "3.10",
)
-# use_repo imports one or more repos generated by the given module extension
-# into the scope of the current module. We are importing the various repos
-# created by the above python.toolchain calls.
-use_repo(
- python,
- PYTHON_NAME_39,
- PYTHON_39_TOOLCHAINS,
- PYTHON_NAME_310,
- PYTHON_310_TOOLCHAINS,
-)
-
-# This call registers the Python toolchains.
-# Note: there is work under way to move this code to within
-# rules_python, and the user won't have to make this call,
-# unless they are registering custom toolchains.
-register_toolchains(
- "@{}//:all".format(PYTHON_39_TOOLCHAINS),
- "@{}//:all".format(PYTHON_310_TOOLCHAINS),
-)
+# You only need to load this repositories if you are using muiltple Python versions.
+# See the tests folder for various examples.
+use_repo(python, PYTHON_NAME_39, "python_aliases")
# The interpreter extension discovers the platform specific Python binary.
# It creates a symlink to the binary, and we pass the label to the following
commit 60c61e51646f610e7886d6bc1eaddb154f80ad44
Author: Chris Love <[email protected]>
Date: Fri May 26 14:20:35 2023 -0600
feat(bzlmod): Allowing multiple python.toolchain extension calls (#1230)
We do this work for two reasons.
First, we must support Module dependencies and sub-modules using
`python.toolchain`. There are already two known instances of sub-modules
setting up a Python toolchain and colliding with another module
(nanobind and rules_testing both run into this).
Second, the upcoming multi-version support is going to work by having
each `python.toolchain()` call register its particular version with the
extra toolchain constraint. This also helps unify the version-aware and
non-version-aware code paths (the non-version aware paths are just
version-aware with a single version registered as the default)
This commit implements various business logic in the toolchain class.
Toolchains in Sub Modules
It will create a toolchain in a sub-module if the toolchain
of the same name does not exist in the root module. The extension stops
name
clashing between toolchains in the root module and sub-modules.
You cannot configure more than one toolchain as the default toolchain.
Toolchain set as the default version.
This extension will not create a toolchain in a sub-module
if the sub-module toolchain is marked as the default version. If you
have
more than one toolchain in your root module, you need to set one of the
toolchains as the default version. If there is only one toolchain, it
is set as the default toolchain.
See #1229 for more information
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index 145cebd..bb4183b 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -11,32 +11,89 @@ local_path_override(
path = "../..",
)
+# This name is passed into python.toolchain and it's use_repo statement.
+# We also use the same value in the python.host_python_interpreter call.
+PYTHON_NAME_39 = "python_39"
+
+PYTHON_39_TOOLCHAINS = PYTHON_NAME_39 + "_toolchains"
+
+INTERPRETER_NAME_39 = "interpreter_39"
+
+PYTHON_NAME_310 = "python_310"
+
+PYTHON_310_TOOLCHAINS = PYTHON_NAME_310 + "_toolchains"
+
+INTERPRETER_NAME_310 = "interpreter_310"
+
+# We next initialize the python toolchain using the extension.
+# You can set different Python versions in this block.
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
- name = "python3_9",
+ # This name is used in the various use_repo statements
+ # below, and in the local extension that is in this
+ # example.
+ name = PYTHON_NAME_39,
configure_coverage_tool = True,
+ # Only set when you have mulitple toolchain versions.
+ is_default = True,
python_version = "3.9",
)
-use_repo(python, "python3_9")
-use_repo(python, "python3_9_toolchains")
+# We are also using a second version of Python in this project.
+# Typically you will only need a single version of Python, but
+# If you need a different vesion we support more than one.
+# Note: we do not supporting using multiple pip extensions, this is
+# work in progress.
+python.toolchain(
+ name = PYTHON_NAME_310,
+ configure_coverage_tool = True,
+ python_version = "3.10",
+)
+
+# use_repo imports one or more repos generated by the given module extension
+# into the scope of the current module. We are importing the various repos
+# created by the above python.toolchain calls.
+use_repo(
+ python,
+ PYTHON_NAME_39,
+ PYTHON_39_TOOLCHAINS,
+ PYTHON_NAME_310,
+ PYTHON_310_TOOLCHAINS,
+)
+
+# This call registers the Python toolchains.
+# Note: there is work under way to move this code to within
+# rules_python, and the user won't have to make this call,
+# unless they are registering custom toolchains.
register_toolchains(
- "@python3_9_toolchains//:all",
+ "@{}//:all".format(PYTHON_39_TOOLCHAINS),
+ "@{}//:all".format(PYTHON_310_TOOLCHAINS),
)
+# The interpreter extension discovers the platform specific Python binary.
+# It creates a symlink to the binary, and we pass the label to the following
+# pip.parse call.
interpreter = use_extension("@rules_python//python/extensions:interpreter.bzl", "interpreter")
interpreter.install(
- name = "interpreter_python3_9",
- python_name = "python3_9",
+ name = INTERPRETER_NAME_39,
+ python_name = PYTHON_NAME_39,
+)
+
+# This second call is only needed if you are using mulitple different
+# Python versions/interpreters.
+interpreter.install(
+ name = INTERPRETER_NAME_310,
+ python_name = PYTHON_NAME_310,
)
-use_repo(interpreter, "interpreter_python3_9")
+use_repo(interpreter, INTERPRETER_NAME_39, INTERPRETER_NAME_310)
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
name = "pip",
- # Intentionally set it false because the "true" case is already covered by examples/bzlmod_build_file_generation
+ # Intentionally set it false because the "true" case is already
+ # covered by examples/bzlmod_build_file_generation
incompatible_generate_aliases = False,
- python_interpreter_target = "@interpreter_python3_9//:python",
+ python_interpreter_target = "@{}//:python".format(INTERPRETER_NAME_39),
requirements_lock = "//:requirements_lock.txt",
requirements_windows = "//:requirements_windows.txt",
)
commit 693a1587baf055979493565933f8f40225c00c6d
Author: Ignas Anikevicius <[email protected]>
Date: Fri May 19 00:12:13 2023 +0900
feat(bzlmod): support entry_point macro (#1220)
Add `entry_point` macro to the repo generated by the `pip.parse`
extension.
This works by using the canonical label literal, so should work without
users
needing to add the spoke repos to the `use_repo` statement. We test this
by
having an extra folder in the `bzlmod` example.
Fixes #958.
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index d2d7d63..145cebd 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -24,11 +24,19 @@ register_toolchains(
"@python3_9_toolchains//:all",
)
+interpreter = use_extension("@rules_python//python/extensions:interpreter.bzl", "interpreter")
+interpreter.install(
+ name = "interpreter_python3_9",
+ python_name = "python3_9",
+)
+use_repo(interpreter, "interpreter_python3_9")
+
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
name = "pip",
# Intentionally set it false because the "true" case is already covered by examples/bzlmod_build_file_generation
incompatible_generate_aliases = False,
+ python_interpreter_target = "@interpreter_python3_9//:python",
requirements_lock = "//:requirements_lock.txt",
requirements_windows = "//:requirements_windows.txt",
)
commit 1383bd4fa06c7c449156913f123c452533cdd724
Author: Zhongpeng Lin <[email protected]>
Date: Wed May 17 10:08:33 2023 -0700
fix: Using canonical name in requirements.bzl (#1176)
When `incompatible_generate_aliases = False`, `pip.parse` doesn't work
in bzlmod, because requirements.bzl has:
```
all_requirements = ["@rules_python~0.21.0~pip~pip_yapf//:pkg"]
all_whl_requirements = ["@rules_python~0.21.0~pip~pip_yapf//:whl"]
```
Starting Bazel 6, canonical names should be referred with double "@".
The reason why `incompatible_generate_aliases = True` worked is because
it uses apparent name by parsing the canonical label with `repo_name =
rctx.attr.name.split("~")[-1]`. This is dangerous because Bazel may
change its canonical name pattern in future versions.
This PR adds a new attribute "repo_name" to `pip_repository_bzlmod`, so
we have access to the human-readable name in the implementation.
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index 61d7967..d2d7d63 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -4,6 +4,7 @@ module(
compatibility_level = 1,
)
+bazel_dep(name = "bazel_skylib", version = "1.4.1")
bazel_dep(name = "rules_python", version = "0.0.0")
local_path_override(
module_name = "rules_python",
@@ -26,6 +27,8 @@ register_toolchains(
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
name = "pip",
+ # Intentionally set it false because the "true" case is already covered by examples/bzlmod_build_file_generation
+ incompatible_generate_aliases = False,
requirements_lock = "//:requirements_lock.txt",
requirements_windows = "//:requirements_windows.txt",
)
commit 46537cf32f8bf722a6be805821cb2ee17d7b936a
Author: Chris Love <[email protected]>
Date: Mon May 15 16:04:40 2023 -0600
feat(bzlmod)!: Move each bzlmod extension into its own file (#1226)
This commit refactors the files that contain the bzlmod
extensions.
- All extensions are moved under the new extensions folder
- Private extensions are moved under extensions/private
- All extension files are renamed to remove the _extension suffix
- pip and internal_deps extensions are moved to their own file
This commit organizes the extensions better and also follows the
best practice of having a single extension per file. Having each
extension in its own file allows them to use some additional features
while helping avoid backwards incompatible changes.
## BREAKING CHANGES
This splits `//python:extensions.bzl`, which previously held the
`python`
and `pip` extensions, into separate files (`python.bzl` and `pip.bzl`,
respectively). Unfortunately, moving the location of the extensions is a
breaking change due to how bzlmod extension identity works (see
https://bazel.build/external/extension#extension_identity). Fortunately,
by moving to one extension per file, we shouldn't have to ever do this
again.
Users must update the file path in their `use_repo()` statements as
follows:
* `use_extension("@rules_python//python:extensions.bzl", "python")` ->
`use_extension("@rules_python//python/extensions:python.bzl", "python")`
* `use_extension("@rules_python//python:extensions.bzl", "pip")` ->
`use_extension("@rules_python//python/extensions:pip.bzl", "pip")`
The following `sed` commands should approximate the necessary changes:
```
sed 'sXuse_extension("@rules_python//python:extensions.bzl", "python")Xuse_extension("@rules_python//python/extensions:python.bzl", "python")X'`
sed 'sXuse_extension("@rules_python//python:extensions.bzl", "pip")Xuse_extension("@rules_python//python/extensions:pip.bzl", "pip")X'`
```
See `examples/bzlmod_build_file_generation/MODULE.bazel` for an example
of the new paths.
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index ce91228..61d7967 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -10,7 +10,7 @@ local_path_override(
path = "../..",
)
-python = use_extension("@rules_python//python:extensions.bzl", "python")
+python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
name = "python3_9",
configure_coverage_tool = True,
@@ -23,7 +23,7 @@ register_toolchains(
"@python3_9_toolchains//:all",
)
-pip = use_extension("@rules_python//python:extensions.bzl", "pip")
+pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
name = "pip",
requirements_lock = "//:requirements_lock.txt",
commit 5ff514ab84c0486b432340e6a04de5beb9f2fd1f
Author: Michael Krasnyk <[email protected]>
Date: Wed Mar 8 20:16:02 2023 +0100
fix: generation of toolchain aliases //:defs.bzl file. (#1088)
## PR Checklist
Please check if your PR fulfills the following requirements:
- [ ] Tests for the changes have been added (for bug fixes / features)
- [ ] Docs have been added / updated (for bug fixes / features)
## PR Type
What kind of change does this PR introduce?
<!-- Please check the one that applies to this PR using "x". -->
- [X] Bugfix
- [ ] Feature (please, look at the "Scope of the project" section in the
README.md file)
- [ ] Code style update (formatting, local variables)
- [ ] Refactoring (no functional changes, no api changes)
- [ ] Build related changes
- [ ] CI related changes
- [ ] Documentation content changes
- [ ] Other... Please describe:
## What is the current behavior?
<!-- Please describe the current behavior that you are modifying, or
link to a relevant issue. -->
With `common --experimental_enable_bzlmod` option defs.bzl is generated
as
```
load("@rules_python~override//python/config_settings:transition.bzl", _py_binary = "py_binary", _py_test = "py_test")
load("@rules_python~override//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements")
```
and these lines cause a problem at
```
load("@python3_9//:defs.bzl", "interpreter")
```
as
```
ERROR: .../BUILD:25:11: error loading package 'src': at .../external/rules_python~override~python~python3_9/defs.bzl:4:6: Unable to find package for @[unknown repo 'rules_python~override' requested from @rules_python~override~python~python3_9]//python:pip.bzl: The repository '@[unknown repo 'rules_python~override' requested from @rules_python~override~python~python3_9]' could not be resolved: No repository visible as '@rules_python~override' from repository '@rules_python~override~python~python3_9'. and referenced by '...'
```
Issue Number: N/A
## What is the new behavior?
Generated load statements
```
load("@@rules_python~override//python/config_settings:transition.bzl", _py_binary = "py_binary", _py_test = "py_test")
load("@@rules_python~override//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements")
```
## Does this PR introduce a breaking change?
- [ ] Yes
- [X] No
<!-- If this PR contains a breaking change, please describe the impact
and migration path for existing applications below. -->
## Other information
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index 5f984c3..ce91228 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -16,6 +16,7 @@ python.toolchain(
configure_coverage_tool = True,
python_version = "3.9",
)
+use_repo(python, "python3_9")
use_repo(python, "python3_9_toolchains")
register_toolchains(
commit 488a0372a4242140cb2e4bfa28e4a8e652866670
Author: Ignas Anikevicius <[email protected]>
Date: Tue Jan 31 10:20:24 2023 +0900
feat(coverage): Register coverage.py to hermetic toolchains (#977)
This allows including the coverage package as part of the toolchain dependencies, which is mixed into a test's dependencies when `bazel coverage` is run (if coverage is not enabled, no extra dependency is added)
For now, it's disabled by default because enabling it poses the risk of having two versions of coverage installed (one from the toolchain, one from the user's dependencies).
The user can turn the coverage_tool setting by passing
`register_coverage_tool=(True|False)` to `python_register_toolchains` or
`python_register_multi_toolchains` call or specifying the
`coverage_tool` label as described in the `versions.bzl` file.
Use coverage.py v6.5.0 because the latest has `types.py` in the package
directory, which imports from Python's stdlib `types` [1]. Somehow the
Python interpreter is thinking that the `from types import FrameType` is
referring to the currently interpreted file and everything breaks. I
would have expected the package to use absolute imports and only attempt
to import from `coverage.types` if we use `coverage.types` and not just
a plain `types` import.
NOTE: Coverage is only for non-windows platforms.
Update tests to:
- ensure that we can still use the toolchain as previously.
- ensure that we are not downloading extra deps if they are not needed.
* Also changes the projects bazelrc to use a remotejdk, which makes it easier for contributors because they don't have to locally install a jdk to get going.
[1]: https://github.com/nedbat/coveragepy/blob/master/coverage/types.py
[3]: https://github.com/bazelbuild/bazel/issues/15835
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index 48fb4cb..5f984c3 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -13,6 +13,7 @@ local_path_override(
python = use_extension("@rules_python//python:extensions.bzl", "python")
python.toolchain(
name = "python3_9",
+ configure_coverage_tool = True,
python_version = "3.9",
)
use_repo(python, "python3_9_toolchains")
commit 96621391e7e8861eb448fb94b49aa1fcb9f45ba7
Author: Fabian Meumertzheim <[email protected]>
Date: Fri Jan 20 15:24:04 2023 +0100
runfiles: Apply repo mapping to Rlocation path (#998)
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index 89e5356..48fb4cb 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -29,7 +29,7 @@ pip.parse(
)
use_repo(pip, "pip")
-bazel_dep(name = "other_module", version = "")
+bazel_dep(name = "other_module", version = "", repo_name = "our_other_module")
local_path_override(
module_name = "other_module",
path = "other_module",
commit a475144054d328b540be46812604fb3806f36ee5
Author: Fabian Meumertzheim <[email protected]>
Date: Thu Jan 19 21:02:43 2023 +0100
runfiles: Import `CurrentRepository` from Bazel (#988)
Also adds an integration test to the BCR test module that replaces the
tests maintained in Bazel itself.
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index bbace86..89e5356 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -28,3 +28,9 @@ pip.parse(
requirements_windows = "//:requirements_windows.txt",
)
use_repo(pip, "pip")
+
+bazel_dep(name = "other_module", version = "")
+local_path_override(
+ module_name = "other_module",
+ path = "other_module",
+)
commit 70cce26432187a60b4e950118791385e6fb3c26f
Author: Greg Roodt <[email protected]>
Date: Wed Jan 11 09:00:51 2023 +1100
Refactor and separate concerns of external python package handling code (#953)
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index 14a338c..bbace86 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -1,23 +1,20 @@
module(
name = "example_bzlmod",
- compatibility_level = 1,
version = "0.0.0",
+ compatibility_level = 1,
)
bazel_dep(name = "rules_python", version = "0.0.0")
-
local_path_override(
module_name = "rules_python",
path = "../..",
)
python = use_extension("@rules_python//python:extensions.bzl", "python")
-
python.toolchain(
name = "python3_9",
python_version = "3.9",
)
-
use_repo(python, "python3_9_toolchains")
register_toolchains(
@@ -25,11 +22,9 @@ register_toolchains(
)
pip = use_extension("@rules_python//python:extensions.bzl", "pip")
-
pip.parse(
name = "pip",
requirements_lock = "//:requirements_lock.txt",
requirements_windows = "//:requirements_windows.txt",
)
-
use_repo(pip, "pip")
commit 00545742ad2450863aeb82353d4275a1e5ed3f24
Author: Derek Cormier <[email protected]>
Date: Mon Nov 21 13:19:37 2022 -0800
fix: fix a bug where some transitive deps could not be resolved via bzlmod
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index 275be3b..14a338c 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -29,6 +29,7 @@ pip = use_extension("@rules_python//python:extensions.bzl", "pip")
pip.parse(
name = "pip",
requirements_lock = "//:requirements_lock.txt",
+ requirements_windows = "//:requirements_windows.txt",
)
use_repo(pip, "pip")
commit af354c2e493e55abe5640a1c40043452cb858cdc
Author: Derek Cormier <[email protected]>
Date: Mon Oct 31 20:18:22 2022 -0700
Create a pip_parse bzlmod extension
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
index e3fc51a..275be3b 100644
--- a/examples/bzlmod/MODULE.bazel
+++ b/examples/bzlmod/MODULE.bazel
@@ -10,3 +10,25 @@ local_path_override(
module_name = "rules_python",
path = "../..",
)
+
+python = use_extension("@rules_python//python:extensions.bzl", "python")
+
+python.toolchain(
+ name = "python3_9",
+ python_version = "3.9",
+)
+
+use_repo(python, "python3_9_toolchains")
+
+register_toolchains(
+ "@python3_9_toolchains//:all",
+)
+
+pip = use_extension("@rules_python//python:extensions.bzl", "pip")
+
+pip.parse(
+ name = "pip",
+ requirements_lock = "//:requirements_lock.txt",
+)
+
+use_repo(pip, "pip")
commit 35391d933fe8f8a63cfbe7e785f106bb349e4141
Author: Alex Eagle <[email protected]>
Date: Mon Jul 11 22:14:44 2022 -0600
chore: support bzlmod (#744)
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
new file mode 100644
index 0000000..e3fc51a
--- /dev/null
+++ b/examples/bzlmod/MODULE.bazel
@@ -0,0 +1,12 @@
+module(
+ name = "example_bzlmod",
+ compatibility_level = 1,
+ version = "0.0.0",
+)
+
+bazel_dep(name = "rules_python", version = "0.0.0")
+
+local_path_override(
+ module_name = "rules_python",
+ path = "../..",
+)
commit 36082079b514529d4009a2b104475fde1cdd5b30
Author: Kevin Park <[email protected]>
Date: Tue Jul 11 06:48:58 2023 -0700
feat: Create `all_data_requirements` alias (#1292)
Minor change.
I am creating a Python virtualenv with Bazel, and having
`all_data_requirements` for data dependencies just like
`all_requirements` would be very helpful.
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
index e08a062..3db7751 100644
--- a/examples/bzlmod/BUILD.bazel
+++ b/examples/bzlmod/BUILD.bazel
@@ -6,7 +6,7 @@
# The names @pip and @python_39 are values that are repository
# names. Those names are defined in the MODULES.bazel file.
load("@bazel_skylib//rules:build_test.bzl", "build_test")
-load("@pip//:requirements.bzl", "all_requirements", "all_whl_requirements", "requirement")
+load("@pip//:requirements.bzl", "all_data_requirements", "all_requirements", "all_whl_requirements", "requirement")
load("@python_3_9//:defs.bzl", py_test_with_transition = "py_test")
load("@python_versions//3.10:defs.bzl", compile_pip_requirements_3_10 = "compile_pip_requirements")
load("@python_versions//3.9:defs.bzl", compile_pip_requirements_3_9 = "compile_pip_requirements")
@@ -83,6 +83,11 @@ build_test(
targets = all_whl_requirements,
)
+build_test(
+ name = "all_data_requirements",
+ targets = all_data_requirements,
+)
+
build_test(
name = "all_requirements",
targets = all_requirements,
commit 523b9de1e9e8b1fc8cbfcb530ee8287bef13a736
Author: Chris Love <[email protected]>
Date: Mon Jul 10 16:11:35 2023 -0600
fix(bzlmod)!: Changing repository name "python_aliases" to "python_versions" (#1304)
I think this name is more informative for a public API. The
functionality it exposes are rules/macros that use a specific Python
version to be used. These aren't really aliases.
This commit renames "python_aliases" to "python_versions". This isn't
technically a breaking
change because bzlmod support is still beta, but we'll flag it as such
just in case.
BREAKING CHANGE:
* The `python_aliases` repo is renamed to `python_versions`. You will
need to either
update references from `@python_aliases` to `@python_versions`, or use
repo-remapping
to alias the old name (`use_repo(python,
python_aliases="python_versions")`)
Closes #1273
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
index e788470..e08a062 100644
--- a/examples/bzlmod/BUILD.bazel
+++ b/examples/bzlmod/BUILD.bazel
@@ -8,8 +8,8 @@
load("@bazel_skylib//rules:build_test.bzl", "build_test")
load("@pip//:requirements.bzl", "all_requirements", "all_whl_requirements", "requirement")
load("@python_3_9//:defs.bzl", py_test_with_transition = "py_test")
-load("@python_aliases//3.10:defs.bzl", compile_pip_requirements_3_10 = "compile_pip_requirements")
-load("@python_aliases//3.9:defs.bzl", compile_pip_requirements_3_9 = "compile_pip_requirements")
+load("@python_versions//3.10:defs.bzl", compile_pip_requirements_3_10 = "compile_pip_requirements")
+load("@python_versions//3.9:defs.bzl", compile_pip_requirements_3_9 = "compile_pip_requirements")
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
# This stanza calls a rule that generates targets for managing pip dependencies
commit 1c581242c25925a1bd6764608877ec170724f11d
Author: Chris Love <[email protected]>
Date: Thu Jun 15 10:58:54 2023 -0600
feat(bzlmod)!: Calling pip multiple times allowing for multiple Python versions (#1254)
Implementing the capability to call pip bzlmod extension multiple times.
We can now call the pip extension with the same hub name multiple times.
This allows a user to have multiple different requirement files based on
the Python version.
With workspace, we need incompatible aliases because you get
@pip//tabulate or @pip_tabulate//:pkg.
The requirement() macro hides this, but changing the flag becomes a
breaking change if you've manually referenced things. With bzlmod,
though, the @pip_tabulate style isn't a realistic option (you'd have to
use_repo everything, which is a huge pain).
So we have chosen to have `@pip//tabulate`.
This commit implements that capability for pip.parse to determine the
Python version from programmatically
the provided interpreter.
See
https://github.com/bazelbuild/rules_python/blob/76aab0f91bd614ee1b2ade310baf28c38695c522/python/extensions/pip.bzl#L88
For more in-depth implementation details.
INTERFACE CHANGE::
- pip.parse argument python_version is introduced but not required. When
possible, it is
inferred.
BREAKING CHANGE:
* `pip.parse()` extension: the `name` attribute is renamed `hub_name`.
This is to reflect
that the user-provided name isn't unique to each `pip.parse()` call. We
now have a hub that is created, and each
pip.parse call creates a spoke.
* `pip.parse()` extension: the `incompatible_generate_aliases` arg is
removed; behavior
has changed to assume it is always True.
* `pip.parse()` calls are collected under the same `hub_name` to support
multiple Python versions under the same resulting repo name (the hub
name0.
Close: #1229
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
index 3bff208..e788470 100644
--- a/examples/bzlmod/BUILD.bazel
+++ b/examples/bzlmod/BUILD.bazel
@@ -8,17 +8,28 @@
load("@bazel_skylib//rules:build_test.bzl", "build_test")
load("@pip//:requirements.bzl", "all_requirements", "all_whl_requirements", "requirement")
load("@python_3_9//:defs.bzl", py_test_with_transition = "py_test")
+load("@python_aliases//3.10:defs.bzl", compile_pip_requirements_3_10 = "compile_pip_requirements")
+load("@python_aliases//3.9:defs.bzl", compile_pip_requirements_3_9 = "compile_pip_requirements")
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
-load("@rules_python//python:pip.bzl", "compile_pip_requirements")
# This stanza calls a rule that generates targets for managing pip dependencies
# with pip-compile.
-compile_pip_requirements(
- name = "requirements",
+compile_pip_requirements_3_9(
+ name = "requirements_3_9",
extra_args = ["--allow-unsafe"],
requirements_in = "requirements.in",
- requirements_txt = "requirements_lock.txt",
- requirements_windows = "requirements_windows.txt",
+ requirements_txt = "requirements_lock_3_9.txt",
+ requirements_windows = "requirements_windows_3_9.txt",
+)
+
+# This stanza calls a rule that generates targets for managing pip dependencies
+# with pip-compile.
+compile_pip_requirements_3_10(
+ name = "requirements_3_10",
+ extra_args = ["--allow-unsafe"],
+ requirements_in = "requirements.in",
+ requirements_txt = "requirements_lock_3_10.txt",
+ requirements_windows = "requirements_windows_3_10.txt",
)
# The rules below are language specific rules defined in
commit b228f6047671abcf4c78ea7318916218fb098831
Author: Richard Levasseur <[email protected]>
Date: Thu Jun 8 07:50:20 2023 -0700
fix(bzlmod)!: Remove ability to specify toolchain repo name. (#1258)
The main reasons this is removed is because if modules choose different
names for the same toolchain, only one of the two toolchains (which are,
hopefully, identical) will be used. Which toolchain is used depends on
the module graph dependency ordering.
Furthermore, as of #1238, only one repo per version is created; others
are ignored. This means a module doesn't know if the name it chooses
will result in a repo being created with that name.
Instead, the toolchain repos are named by rules_python:
`python_{major}_{minor}`. These repo names are currently part of the
public API, since they end up referenced in MODULE config (to wire the
toolchain interpreter to pip).
BREAKING CHANGES
This removes the `name` arg from `python.toolchain()`. Users will need
to remove such usages from their `MODULE.bazel` and update their
`use_repo()` statements. If keeping the custom repo name is necessary,
then repo mappings can be used. See #1232 for additional migration
steps, commands, and information.
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
index 0a068ce..3bff208 100644
--- a/examples/bzlmod/BUILD.bazel
+++ b/examples/bzlmod/BUILD.bazel
@@ -7,7 +7,7 @@
# names. Those names are defined in the MODULES.bazel file.
load("@bazel_skylib//rules:build_test.bzl", "build_test")
load("@pip//:requirements.bzl", "all_requirements", "all_whl_requirements", "requirement")
-load("@python_39//:defs.bzl", py_test_with_transition = "py_test")
+load("@python_3_9//:defs.bzl", py_test_with_transition = "py_test")
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
commit 148622aa92fdd5afcaf9f153d7bb6afce713e553
Author: Chris Love <[email protected]>
Date: Wed May 31 19:48:28 2023 -0600
feat(bzlmod): Moving register.toolchains internal (#1238)
This commit moves the register.toolchains bzlmod call to inside
of rules_python. Instead of a user having to call register.toolchains
in their MODULE.bazel, rules_python/MODULE.bazel calls it
on the internal hub.
This is a breaking change if you are using register.toolchains inside
of submodules. Using register.toolchains inside of submodules is
not recommended anyways. This is now broken because we are not
creating a repo for every Python version toolchain. All of the
toochain calls exist now in the hub's repo BUILD.bazel file.
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
index 8649822..0a068ce 100644
--- a/examples/bzlmod/BUILD.bazel
+++ b/examples/bzlmod/BUILD.bazel
@@ -8,9 +8,6 @@
load("@bazel_skylib//rules:build_test.bzl", "build_test")
load("@pip//:requirements.bzl", "all_requirements", "all_whl_requirements", "requirement")
load("@python_39//:defs.bzl", py_test_with_transition = "py_test")
-
-# This is not working yet till the toolchain hub registration is working
-# load("@python_310//:defs.bzl", py_binary_310 = "py_binary")
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
@@ -50,22 +47,6 @@ py_binary(
],
)
-# This is still WIP. Not working till we have the toolchain
-# registration functioning.
-
-# This is used for testing mulitple versions of Python. This is
-# used only when you need to support multiple versions of Python
-# in the same project.
-# py_binary_310(
-# name = "main_310",
-# srcs = ["__main__.py"],
-# main = "__main__.py",
-# visibility = ["//:__subpackages__"],
-# deps = [
-# ":lib",
-# ],
-# )
-
# see https://bazel.build/reference/be/python#py_test
py_test(
name = "test",
commit 60c61e51646f610e7886d6bc1eaddb154f80ad44
Author: Chris Love <[email protected]>
Date: Fri May 26 14:20:35 2023 -0600
feat(bzlmod): Allowing multiple python.toolchain extension calls (#1230)
We do this work for two reasons.
First, we must support Module dependencies and sub-modules using
`python.toolchain`. There are already two known instances of sub-modules
setting up a Python toolchain and colliding with another module
(nanobind and rules_testing both run into this).
Second, the upcoming multi-version support is going to work by having
each `python.toolchain()` call register its particular version with the
extra toolchain constraint. This also helps unify the version-aware and
non-version-aware code paths (the non-version aware paths are just
version-aware with a single version registered as the default)
This commit implements various business logic in the toolchain class.
Toolchains in Sub Modules
It will create a toolchain in a sub-module if the toolchain
of the same name does not exist in the root module. The extension stops
name
clashing between toolchains in the root module and sub-modules.
You cannot configure more than one toolchain as the default toolchain.
Toolchain set as the default version.
This extension will not create a toolchain in a sub-module
if the sub-module toolchain is marked as the default version. If you
have
more than one toolchain in your root module, you need to set one of the
toolchains as the default version. If there is only one toolchain, it
is set as the default toolchain.
See #1229 for more information
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
index e1f5790..8649822 100644
--- a/examples/bzlmod/BUILD.bazel
+++ b/examples/bzlmod/BUILD.bazel
@@ -1,9 +1,21 @@
+# Load various rules so that we can have bazel download
+# various rulesets and dependencies.
+# The `load` statement imports the symbol for the rule, in the defined
+# ruleset. When the symbol is loaded you can use the rule.
+
+# The names @pip and @python_39 are values that are repository
+# names. Those names are defined in the MODULES.bazel file.
load("@bazel_skylib//rules:build_test.bzl", "build_test")
load("@pip//:requirements.bzl", "all_requirements", "all_whl_requirements", "requirement")
-load("@python3_9//:defs.bzl", py_test_with_transition = "py_test")
+load("@python_39//:defs.bzl", py_test_with_transition = "py_test")
+
+# This is not working yet till the toolchain hub registration is working
+# load("@python_310//:defs.bzl", py_binary_310 = "py_binary")
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
+# This stanza calls a rule that generates targets for managing pip dependencies
+# with pip-compile.
compile_pip_requirements(
name = "requirements",
extra_args = ["--allow-unsafe"],
@@ -12,6 +24,11 @@ compile_pip_requirements(
requirements_windows = "requirements_windows.txt",
)
+# The rules below are language specific rules defined in
+# rules_python. See
+# https://bazel.build/reference/be/python
+
+# see https://bazel.build/reference/be/python#py_library
py_library(
name = "lib",
srcs = ["lib.py"],
@@ -22,6 +39,7 @@ py_library(
],
)
+# see https://bazel.build/reference/be/python#py_binary
py_binary(
name = "bzlmod",
srcs = ["__main__.py"],
@@ -32,6 +50,23 @@ py_binary(
],
)
+# This is still WIP. Not working till we have the toolchain
+# registration functioning.
+
+# This is used for testing mulitple versions of Python. This is
+# used only when you need to support multiple versions of Python
+# in the same project.
+# py_binary_310(
+# name = "main_310",
+# srcs = ["__main__.py"],
+# main = "__main__.py",
+# visibility = ["//:__subpackages__"],
+# deps = [
+# ":lib",
+# ],
+# )
+
+# see https://bazel.build/reference/be/python#py_test
py_test(
name = "test",
srcs = ["test.py"],
@@ -46,6 +81,11 @@ py_test_with_transition(
deps = [":lib"],
)
+# This example is also used for integration tests within
+# rules_python. We are using
+# https://github.com/bazelbuild/bazel-skylib
+# to run some of the tests.
+# See: https://github.com/bazelbuild/bazel-skylib/blob/main/docs/build_test_doc.md
build_test(
name = "all_wheels",
targets = all_whl_requirements,
commit 693a1587baf055979493565933f8f40225c00c6d
Author: Ignas Anikevicius <[email protected]>
Date: Fri May 19 00:12:13 2023 +0900
feat(bzlmod): support entry_point macro (#1220)
Add `entry_point` macro to the repo generated by the `pip.parse`
extension.
This works by using the canonical label literal, so should work without
users
needing to add the spoke repos to the `use_repo` statement. We test this
by
having an extra folder in the `bzlmod` example.
Fixes #958.
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
index 3183608..e1f5790 100644
--- a/examples/bzlmod/BUILD.bazel
+++ b/examples/bzlmod/BUILD.bazel
@@ -35,6 +35,7 @@ py_binary(
py_test(
name = "test",
srcs = ["test.py"],
+ main = "test.py",
deps = [":lib"],
)
commit 1383bd4fa06c7c449156913f123c452533cdd724
Author: Zhongpeng Lin <[email protected]>
Date: Wed May 17 10:08:33 2023 -0700
fix: Using canonical name in requirements.bzl (#1176)
When `incompatible_generate_aliases = False`, `pip.parse` doesn't work
in bzlmod, because requirements.bzl has:
```
all_requirements = ["@rules_python~0.21.0~pip~pip_yapf//:pkg"]
all_whl_requirements = ["@rules_python~0.21.0~pip~pip_yapf//:whl"]
```
Starting Bazel 6, canonical names should be referred with double "@".
The reason why `incompatible_generate_aliases = True` worked is because
it uses apparent name by parsing the canonical label with `repo_name =
rctx.attr.name.split("~")[-1]`. This is dangerous because Bazel may
change its canonical name pattern in future versions.
This PR adds a new attribute "repo_name" to `pip_repository_bzlmod`, so
we have access to the human-readable name in the implementation.
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
index 7ecc035..3183608 100644
--- a/examples/bzlmod/BUILD.bazel
+++ b/examples/bzlmod/BUILD.bazel
@@ -1,4 +1,5 @@
-load("@pip//:requirements.bzl", "requirement")
+load("@bazel_skylib//rules:build_test.bzl", "build_test")
+load("@pip//:requirements.bzl", "all_requirements", "all_whl_requirements", "requirement")
load("@python3_9//:defs.bzl", py_test_with_transition = "py_test")
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
@@ -43,3 +44,13 @@ py_test_with_transition(
main = "test.py",
deps = [":lib"],
)
+
+build_test(
+ name = "all_wheels",
+ targets = all_whl_requirements,
+)
+
+build_test(
+ name = "all_requirements",
+ targets = all_requirements,
+)
commit 5ff514ab84c0486b432340e6a04de5beb9f2fd1f
Author: Michael Krasnyk <[email protected]>
Date: Wed Mar 8 20:16:02 2023 +0100
fix: generation of toolchain aliases //:defs.bzl file. (#1088)
## PR Checklist
Please check if your PR fulfills the following requirements:
- [ ] Tests for the changes have been added (for bug fixes / features)
- [ ] Docs have been added / updated (for bug fixes / features)
## PR Type
What kind of change does this PR introduce?
<!-- Please check the one that applies to this PR using "x". -->
- [X] Bugfix
- [ ] Feature (please, look at the "Scope of the project" section in the
README.md file)
- [ ] Code style update (formatting, local variables)
- [ ] Refactoring (no functional changes, no api changes)
- [ ] Build related changes
- [ ] CI related changes
- [ ] Documentation content changes
- [ ] Other... Please describe:
## What is the current behavior?
<!-- Please describe the current behavior that you are modifying, or
link to a relevant issue. -->
With `common --experimental_enable_bzlmod` option defs.bzl is generated
as
```
load("@rules_python~override//python/config_settings:transition.bzl", _py_binary = "py_binary", _py_test = "py_test")
load("@rules_python~override//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements")
```
and these lines cause a problem at
```
load("@python3_9//:defs.bzl", "interpreter")
```
as
```
ERROR: .../BUILD:25:11: error loading package 'src': at .../external/rules_python~override~python~python3_9/defs.bzl:4:6: Unable to find package for @[unknown repo 'rules_python~override' requested from @rules_python~override~python~python3_9]//python:pip.bzl: The repository '@[unknown repo 'rules_python~override' requested from @rules_python~override~python~python3_9]' could not be resolved: No repository visible as '@rules_python~override' from repository '@rules_python~override~python~python3_9'. and referenced by '...'
```
Issue Number: N/A
## What is the new behavior?
Generated load statements
```
load("@@rules_python~override//python/config_settings:transition.bzl", _py_binary = "py_binary", _py_test = "py_test")
load("@@rules_python~override//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements")
```
## Does this PR introduce a breaking change?
- [ ] Yes
- [X] No
<!-- If this PR contains a breaking change, please describe the impact
and migration path for existing applications below. -->
## Other information
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
index 7b7566b..7ecc035 100644
--- a/examples/bzlmod/BUILD.bazel
+++ b/examples/bzlmod/BUILD.bazel
@@ -1,4 +1,5 @@
load("@pip//:requirements.bzl", "requirement")
+load("@python3_9//:defs.bzl", py_test_with_transition = "py_test")
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
@@ -35,3 +36,10 @@ py_test(
srcs = ["test.py"],
deps = [":lib"],
)
+
+py_test_with_transition(
+ name = "test_with_transition",
+ srcs = ["test.py"],
+ main = "test.py",
+ deps = [":lib"],
+)
commit 488a0372a4242140cb2e4bfa28e4a8e652866670
Author: Ignas Anikevicius <[email protected]>
Date: Tue Jan 31 10:20:24 2023 +0900
feat(coverage): Register coverage.py to hermetic toolchains (#977)
This allows including the coverage package as part of the toolchain dependencies, which is mixed into a test's dependencies when `bazel coverage` is run (if coverage is not enabled, no extra dependency is added)
For now, it's disabled by default because enabling it poses the risk of having two versions of coverage installed (one from the toolchain, one from the user's dependencies).
The user can turn the coverage_tool setting by passing
`register_coverage_tool=(True|False)` to `python_register_toolchains` or
`python_register_multi_toolchains` call or specifying the
`coverage_tool` label as described in the `versions.bzl` file.
Use coverage.py v6.5.0 because the latest has `types.py` in the package
directory, which imports from Python's stdlib `types` [1]. Somehow the
Python interpreter is thinking that the `from types import FrameType` is
referring to the currently interpreted file and everything breaks. I
would have expected the package to use absolute imports and only attempt
to import from `coverage.types` if we use `coverage.types` and not just
a plain `types` import.
NOTE: Coverage is only for non-windows platforms.
Update tests to:
- ensure that we can still use the toolchain as previously.
- ensure that we are not downloading extra deps if they are not needed.
* Also changes the projects bazelrc to use a remotejdk, which makes it easier for contributors because they don't have to locally install a jdk to get going.
[1]: https://github.com/nedbat/coveragepy/blob/master/coverage/types.py
[3]: https://github.com/bazelbuild/bazel/issues/15835
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
index 2094567..7b7566b 100644
--- a/examples/bzlmod/BUILD.bazel
+++ b/examples/bzlmod/BUILD.bazel
@@ -12,7 +12,7 @@ compile_pip_requirements(
py_library(
name = "lib",
- srcs = ["__init__.py"],
+ srcs = ["lib.py"],
deps = [
requirement("pylint"),
requirement("tabulate"),
commit 2c29f401dcab6e6b84b56d9b4e9849fe0cf0a550
Author: Arne-Christian Rundereim <[email protected]>
Date: Fri Dec 30 21:26:14 2022 +0100
Fix hyphen in requirement didn't work with bzlmod (#952) (#957)
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
index a00c96d..2094567 100644
--- a/examples/bzlmod/BUILD.bazel
+++ b/examples/bzlmod/BUILD.bazel
@@ -16,6 +16,7 @@ py_library(
deps = [
requirement("pylint"),
requirement("tabulate"),
+ requirement("python-dateutil"),
],
)
commit 00545742ad2450863aeb82353d4275a1e5ed3f24
Author: Derek Cormier <[email protected]>
Date: Mon Nov 21 13:19:37 2022 -0800
fix: fix a bug where some transitive deps could not be resolved via bzlmod
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
index 5693fa6..a00c96d 100644
--- a/examples/bzlmod/BUILD.bazel
+++ b/examples/bzlmod/BUILD.bazel
@@ -7,12 +7,14 @@ compile_pip_requirements(
extra_args = ["--allow-unsafe"],
requirements_in = "requirements.in",
requirements_txt = "requirements_lock.txt",
+ requirements_windows = "requirements_windows.txt",
)
py_library(
name = "lib",
srcs = ["__init__.py"],
deps = [
+ requirement("pylint"),
requirement("tabulate"),
],
)
commit af354c2e493e55abe5640a1c40043452cb858cdc
Author: Derek Cormier <[email protected]>
Date: Mon Oct 31 20:18:22 2022 -0700
Create a pip_parse bzlmod extension
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
index 3e0349b..5693fa6 100644
--- a/examples/bzlmod/BUILD.bazel
+++ b/examples/bzlmod/BUILD.bazel
@@ -1,8 +1,20 @@
+load("@pip//:requirements.bzl", "requirement")
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
+load("@rules_python//python:pip.bzl", "compile_pip_requirements")
+
+compile_pip_requirements(
+ name = "requirements",
+ extra_args = ["--allow-unsafe"],
+ requirements_in = "requirements.in",
+ requirements_txt = "requirements_lock.txt",
+)
py_library(
name = "lib",
srcs = ["__init__.py"],
+ deps = [
+ requirement("tabulate"),
+ ],
)
py_binary(
@@ -10,7 +22,9 @@ py_binary(
srcs = ["__main__.py"],
main = "__main__.py",
visibility = ["//:__subpackages__"],
- deps = [":lib"],
+ deps = [
+ ":lib",
+ ],
)
py_test(
commit 35391d933fe8f8a63cfbe7e785f106bb349e4141
Author: Alex Eagle <[email protected]>
Date: Mon Jul 11 22:14:44 2022 -0600
chore: support bzlmod (#744)
diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel
new file mode 100644
index 0000000..3e0349b
--- /dev/null
+++ b/examples/bzlmod/BUILD.bazel
@@ -0,0 +1,20 @@
+load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
+
+py_library(
+ name = "lib",
+ srcs = ["__init__.py"],
+)
+
+py_binary(
+ name = "bzlmod",
+ srcs = ["__main__.py"],
+ main = "__main__.py",
+ visibility = ["//:__subpackages__"],
+ deps = [":lib"],
+)
+
+py_test(
+ name = "test",
+ srcs = ["test.py"],
+ deps = [":lib"],
+)
Thanks, Chris.
Summarizing (WIP still)...
(Also note I'm just thinking of the move from our initial bzlmod api to what we have today)
Definitely (or almost certainly) breaking changes:
- python.toolchain extension:
- extension path changed: //python/extensions.bzl -> //python/extensions:python.bzl
- "name" arg removed; the user cannot select the name anymore; the repo names are "python_X_Y" now.
- pip.parse extension:
- Extension path changed: //python/extensions.bzl -> //python/extensions/pip.bzl
- "name" arg renamed to "hub_name"
- "incompatible_generate_aliases" arg removed; it is always True
Potentially breaking changes:
- When there are multiple python.toolchain calls, one must be explictly marked
is_default=True
Other notable changes, but ones that aren't necessarily breaking changes:
- User modules no longer need to call register_toolchains() manually. Manually calling register toolchains will still work, it just changes the toolchain precedence, as expected.
- pip.parse detects and rejects duplicate pip hub names; previously this would fail later, when the underlying repos were declared.
(note to self: left off at commit 9dd944e963807b02b21489ad286715b60aec8c84)
(WIP still; need to finish going through the change log)
Potential Backwards compatibility concessions
Re-add name arg to pip.parse; mark it deprecated. This is pretty easy and might go a long way.
A backwards compatibility path for the extensions path change is tough. We can re-introduce extensions.bzl, but:
- We have to pick to put either pip or python in it. The two have to be separated because the pip extension loads code generated by the python extension.
- Even if e.g.
extension.bzl#piploads and just re-exportsextensions/pip.bzl#pip, the two are distinct extension as far as Bazel is concerned. i.e. the module_extension implementation function runs once for each, and they can't see each other's tag_class calls.
I'm thinking that re-introducing extensions.bzl#pip is probably the better because the pip.parse stuff is specific to a module, while toolchains are cross-module. The upgrade path for the toolchain API changes is also probably more straight-forward than pip? i.e., users can just delete the config entirely, or just need to remove the "name" arg and change a use_repo() call. Pip, in comparison, changes the arg name, plus you'll want to have a different arg value entirely since those names are global.
Seems like this is a stale issue, feel free to create a new one if I have missed anything.
This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days. Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue was automatically closed because it went 30 days without a reply since it was labeled "Can Close?"