MODULE.bazel.lock file contains user specific paths

[jsharpe]

Description of the bug:

If you use a workspace local registry then the attributes recorded in the lockfile contain the expanded value for %workspace% instead of using the %workspace% placeholder.

Which category does this issue belong to?

External Dependency

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Use --registry=file://%workspace%/registry and then depend on a module in that registry.

The applied patches from the registry are referenced using the expanded path of %workspace% instead of using a placeholder.

Which operating system are you running Bazel on?

Ubuntu linux

What is the output of bazel info release?

6.4.0rc1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[jsharpe]

@bazel-io flag

[iancha1992]

@bazel-io fork 6.4.0

[iancha1992]

A fix for this issue has been included in Bazel 6.4.0 RC2. Please test out the release candidate and report any issues as soon as possible. Thanks!

[BoleynSu]

@iancha1992 @SalmaSamy The issue is still not fixed for extensions. See https://github.com/bazelbuild/bazel/blob/8e8ddaba0e90c280bfd85644d6ccd91df5b6d353/src/main/java/com/google/devtools/build/lib/bazel/bzlmod/ModuleFileGlobals.java#L520

[meteorcloudy]

@BoleynSu Can you please give more details of what's still broken? Ideally with a minimal reproducible case.

[BoleynSu]

      "extensionUsages": [
        { 
          "extensionBzlFile": "@rules_java//java:extensions.bzl",
          "extensionName": "toolchains",
          "usingModule": "[email protected]",
          "location": {
            "file": "file://PATH_TO_LOCAL_REGISTRY/modules/rules_java/7.3.2/MODULE.bazel",         
            "line": 19,
            "column": 27
          },
          "imports": {
            "remote_java_tools": "remote_java_tools",
            "remote_java_tools_linux": "remote_java_tools_linux",
            "remote_java_tools_windows": "remote_java_tools_windows",
            "remote_java_tools_darwin_x86_64": "remote_java_tools_darwin_x86_64",

It should be repro-able by building any repo with --registry=file://some_local_checkout_of_bcr.

[Wyverald]

This is working as intended. If you're using a local path registry, we have to write the path to the lockfile to ensure reproducibility (unless the path is under the workspace root). If you're just doing some local testing and don't want to commit things into the lockfile, please use --lockfile_mode=off.

[BoleynSu]

Sorry that I did not make it claer. The local path is within workspace, i.e. %workspace%/reg.

[Wyverald]

@SalmaSamy could you try to repro and investigate? And close if unreproducible.

[BoleynSu]

Friendly ping. The issue is not resolved yet in Bazel 7.0.1. To repro, can bazel build --config=bzlmod --lockfile_mode=update using https://github.com/BoleynSu-Org/monorepo/

[SalmaSamy]

@BoleynSu Thanks, I was able to reproduce it and working on fixing it.

[fmeum]

Should this be cherry-picked into 7.0.1?

[Wyverald]

@bazel-io fork 7.1.0

[iancha1992]

A fix for this issue has been included in Bazel 7.1.0 RC1. Please test out the release candidate and report any issues as soon as possible. Thanks!

[lalten]

Hi @iancha1992, we are hitting this on 7.1.1 still.

Below is a snippet of the MODULE.bazel.lock. You can see it uses the %workspace%/registry registry for rules_qnx, which is an internal ruleset that lives inside the monorepo under %workspace%/modules/rules_qnx.

    "[email protected]": {
      "name": "rules_qnx",
      "version": "0.0.1",
      "key": "[email protected]",
      "repoName": "rules_qnx",
      "executionPlatformsToRegister": [],
      "toolchainsToRegister": [],
      "extensionUsages": [
        {
          "extensionBzlFile": "@rules_qnx//qnx:extensions.bzl",
          "extensionName": "patched_repo",
          "usingModule": "[email protected]",
          "location": {
            "file": "file://%workspace%/registry/modules/rules_qnx/0.0.1/MODULE.bazel",
            "line": 10,
            "column": 29
          },
          "imports": {
            "qnx_cc_toolchain_config": "qnx_cc_toolchain_config"
          },
          "devImports": [],
          "tags": [
            {
              "tagName": "create",
              "attributeValues": {
                "name": "qnx_cc_toolchain_config",
                "out": "qnx_cc_toolchain_config.bzl",
                "patch": "@rules_qnx//toolchain:cc_toolchain_config_bzl.patch",
                "upstream": "@bazel_tools//tools/cpp:unix_cc_toolchain_config.bzl"
              },
              "devDependency": false,
              "location": {
                "file": "file://%workspace%/registry/modules/rules_qnx/0.0.1/MODULE.bazel",
                "line": 14,
                "column": 20
              }
            }
          ],
          "hasDevUseExtension": false,
          "hasNonDevUseExtension": true
        }
      ],
      "deps": {
        "bazel_skylib": "[email protected]",
        "rules_cc": "[email protected]",
        "platforms": "[email protected]",
        "bazel_tools": "bazel_tools@_",
        "local_config_platform": "local_config_platform@_"
      },
      "repoSpec": {
        "ruleClassName": "local_repository",
        "attributes": {
          "path": "/home/username/monorepo/modules/rules_qnx"
        }
      }
    },

Here is the content MODULE.bazel of rules_qnx:

module(
    name = "rules_qnx",
    version = "0.0.1",
)

bazel_dep(name = "bazel_skylib", version = "1.5.0")
bazel_dep(name = "rules_cc", version = "0.0.9")
bazel_dep(name = "platforms", version = "0.0.7")

patched_repo = use_extension("//qnx:extensions.bzl", "patched_repo")

# This creates a tiny repo containing a patched version of the `unix_cc_toolchain_config.bzl` file.
# The patch adds a "requires-network" tag on all compile+link actions, which is necessary for the QNX toolchain to talk to the license server without being blocked by the sandbox.
patched_repo.create(
    name = "qnx_cc_toolchain_config",
    out = "qnx_cc_toolchain_config.bzl",
    patch = "@rules_qnx//toolchain:cc_toolchain_config_bzl.patch",
    upstream = "@bazel_tools//tools/cpp:unix_cc_toolchain_config.bzl",
)
use_repo(patched_repo, "qnx_cc_toolchain_config")

I'm hoping this is enough information to help you reproduce/fix the issue?

[meteorcloudy]

@lalten What is your expected behaviour?

[lalten]

The "path": "/home/username/monorepo/modules/rules_qnx" part is not portable. Probably something like "path": "%workspace%/modules/rules_qnx" would be better. For now we cannot use the Lockfile because of the absolute path that's not present on most machines.

[meteorcloudy]

Oh, sorry, I missed that.

      "repoSpec": {
        "ruleClassName": "local_repository",
        "attributes": {
          "path": "/home/username/monorepo/modules/rules_qnx"
        }
      }

But why is the root module a local_repository? /cc @SalmaSamy @Wyverald

@lalten Can you create a full reproducible case for this?

[appthumb]

I'm having the same issue, when I use my local registry relative to my %workspace%, and the MODULE.bazel.lock ends up storing the full absolute path in the "attribute".

The directory structure:

root |--- bazel_registry / modules / repo_A |--- repo_A |--- repo_B

And repo_B depends on repo_A through bazel_registry.

In repo_B/.bazelrc, I have:

common --registry=file://%workspace%/../bazel_registry
common --registry=https://bcr.bazel.build

In bazel_registry/bazel_registry.json, I have:

{
  "module_base_path": "../"
}

In bazel_registry/modules/repo_A/1.0.0/source.json, I have:

{
  "type": "local_path",
  "path": "repo_A"
}

This works correctly locally, and Bazel correctly finds the /root/repo_A and uses it when I build repo_B.

But the MODULE.bazel.lock file contains the absolute full path of repo_A in the "attribute":

      "repoSpec": {
        "ruleClassName": "local_repository",
        "attributes": {
          "path": "/Users/<my-local-directory>/repo_A"
        }
      }

This makes it not portable on a different machine, because the local path is different. Ideally the resolved attributes would be relative to %workspace%. According to bazel document, the source path would be resolved as:

If path and module_base_path are both relative paths, it resolves to <registry_path>/<module_base_path>/. Registry must be hosted locally and used by --registry=file://<registry_path>

So something like "%workspace%/../bazel_registry/../repo_A" would be a direct translation of the rules. Maybe supporting relative path so we will end up with "../repo_A" in the "attribute" is even better.

[appthumb]

Also, I'm using bazel 7.1.1.

$ bazelisk --version
bazel 7.1.1

[meteorcloudy]

@SalmaSamy Can you take a look?

This seems to happen when source.json is using local_path

{
  "type": "local_path",
  "path": "repo_A"
}

[fmeum]

@SalmaSamy I just noticed that this would go away entirely with the new lockfile format as neither local registry contents nor any repo specs for Bazel modules are tracked in it.

[iancha1992]

A fix for this issue has been included in Bazel 7.2.0 RC1. Please test out the release candidate and report any issues as soon as possible. If you're using Bazelisk, you can point to the latest RC by setting USE_BAZEL_VERSION=7.2.0rc1. Thanks!

[appthumb]

Thanks for the fix! It works smoothly with my local bazel registry, and the MODULE.bazel.lock doesn't include any local path and works fine across different workspaces. Interestingly, when I look at the lock file, the dependency on my local repo is not mentioned at all, which seems to be intended according to https://github.com/bazelbuild/bazel/issues/19621#issuecomment-2079621790.

Overall this is very nice, thanks!