-Wconf:src filter resolves symlinks, breaking `external/` filters in Bazel 9

[mbland]

Reproduction steps

The following requires Bazelisk for selecting the Bazel version.

$ mkdir wconfsrc-repro
$ cd wconfsrc-repro
$ cat >MODULE.bazel
module(name = "wconfsrc_repro")

bazel_dep(name = "rules_scala", version = "7.1.5")

scala_config = use_extension(
    "@rules_scala//scala/extensions:config.bzl",
    "scala_config",
)
scala_config.settings(scala_version = "2.13.17")

scala_deps = use_extension(
    "@rules_scala//scala/extensions:deps.bzl",
    "scala_deps",
)
scala_deps.scala()

register_toolchains("//...:all")
^D

$ cat >BUILD
load("@rules_scala//scala:scala_toolchain.bzl", "scala_toolchain")

scala_toolchain(
    name = "toolchain_impl",
    scalacopts = [
        "-Xfatal-warnings",
        "-deprecation",
        "-Wconf:src=bazel-out/.*:s",
        "-Wconf:src=external/.*:s",
    ],
)

toolchain(
    name = "toolchain",
    toolchain = ":toolchain_impl",
    toolchain_type = "@rules_scala//scala:toolchain_type",
    visibility = ["//visibility:public"],
)
^D

# Output edited for readability
$ USE_BAZEL_VERSION=9.0.0rc1 bazel build \
  @rules_scala//third_party/dependency_analyzer/src/main:dependency_analyzer

INFO: Analyzed target
  @@rules_scala+//third_party/dependency_analyzer/src/main:dependency_analyzer
  (1 packages loaded, 15 targets and 16 aspects configured).
  [ ...snip... ]
ERROR: /private/var/tmp/_bazel_mbland/.../external/rules_scala+/third_party/dependency_analyzer/src/main/BUILD:4:39:
  scala @@rules_scala+//third_party/dependency_analyzer/src/main:dependency_analyzer failed:
  [ ...snip... ]
external/rules_scala+/third_party/dependency_analyzer/src/main/io/bazel/rulesscala/dependencyanalyzer/Reporter213.scala:21:
  warning: object JavaConverters in package collection is deprecated (since 2.13.0): Use `scala.jdk.CollectionConverters` instead
      case r: DepsTrackingReporter => r.registerAstUsedJars(usedJarPathToPositions.keys.toSet.asJava)
                                                                                        ^
external/rules_scala+/third_party/dependency_analyzer/src/main/io/bazel/rulesscala/dependencyanalyzer/Reporter213.scala:28:
  warning: method doReport in class FilteringReporter is deprecated (since 2.13.12): use the `doReport` overload instead
    global.reporter.doReport(pos, message, global.reporter.ERROR)
                    ^
external/rules_scala+/third_party/dependency_analyzer/src/main/io/bazel/rulesscala/dependencyanalyzer/Reporter213.scala:33:
  warning: method doReport in class FilteringReporter is deprecated (since 2.13.12): use the `doReport` overload instead
    global.reporter.doReport(pos, message, global.reporter.WARNING)
                    ^
error: No warnings can be incurred under -Werror.
3 warnings
1 error
Build failure with errors.
Target @@rules_scala+//third_party/dependency_analyzer/src/main:dependency_analyzer failed to build
[ ...snip... ]

Problem

The -Wconf:src filter in 2.12 and 2.13 resolves each source path to its canonical form before matching it. If the original path is a symlink, the actual path may not match the filter. This renders -Wconf:src=external/.*:s filters to silence warnings in external source repositories under Bazel 9 ineffective.

• The -Wconf:src parser creates a SourcePattern object from its regex.
• SourcePattern.check() ultimately applies its pattern to java.io.File.getCanonicalPath().
• java.io.File.getCanonicalPath() resolves all symlinks.

Scala3 does not have this issue, since Scala 3.5.0 and later uses toAbsolutePath in its implementation instead. Updating the MODULE.bazel file to use Scala 3.5.2 and adding/removing the -Wconf:src=external/.*:s flag demonstrates this.

Users may currently work around this problem by applying -Wconf:src=cache/repos/v1/contents/.*:s when building with Bazel 9. However, the behavior is still surprising, and the workaround isn't particularly discoverable or stable.

I'm happy to send a pull request to update SourcePattern.check() to use absolutePath instead.

(Granted, I/we should update the rules_scala implementations to address the deprecation warnings. But for now, they effectively illustrate the problem.)

---

More detail on the Bazel side: The Bazel output directory layout stores unpacked external source archives, such as rules_scala, in an external/ directory. This directory contains symlinks to the actual archives elsewhere on the file system.

• Under Bazel 8, the actual path contains an external/ segment.
• Under Bazel 9, the actual path no longer contains an external/ segment. It now reflects the changes from bazelbuild/bazel@ad74aa5d9e29e7d4ab3043328ee25901be9e14f6 and bazelbuild/bazel@e66fe55aa187e8eaacbbd949d7e02e2a8cc56ce3.

# Change to the execroot directory revealed by building with
# `bazel build -s`
$ pushd /private/var/tmp/_bazel_mbland/.../execroot/_main

# Same `external/` symlink for both versions, pointing into the `external/`
# sibling of `execroot/`
$ ls -l external/rules_scala+
lrwxr-xr-x  1 mbland  wheel  85 Dec 11 14:08 external/rules_scala+@ -> /private/var/tmp/_bazel_mbland/.../external/rules_scala+

# The actual `external/rules_scala+` directory exists under 8.4.2
$ ls -l ../../external/rules_scala+
[ ...shows contents of actual external/rules_scala+ directory... ]

# The `external/rules_scala+` symlink under 9.0.0rc1,
# where the `...` segments are hash values
$ ls -l ../../external/rules_scala+
lrwxr-xr-x  1 mbland  wheel  148 Dec 11 16:45 ../../external/rules_scala+@ -> /var/tmp/_bazel_mbland/cache/repos/v1/contents/.../...

cc: @fmeum @Wyverald @rafikk

Also cc: @lrytz, author of scala/scala@39d3b3a93cd3cf01303ba9aba487b742943d8e57 and scala/scala#8373, in case there was a rationale for using canonicalPath over absolutePath.

[lrytz]

I'm happy to send a pull request to update SourcePattern.check() to use absolutePath instead.

Sounds good to me. I guess the reason I picked canonical is to remove redundant . / ... So let's also call https://docs.oracle.com/javase/8/docs/api/java/nio/file/Path.html#normalize--.

Or https://docs.oracle.com/javase/8/docs/api/java/net/URI.html#normalize-- like Scala 3, I assume they picked that to also normalize windows \ to / at the same time.

[fmeum]

From the perspective of Bazel and other build systems that use sandboxing to increase hermeticity, it would probably be even better if this only used normalize and not absolutePath. Bazel invokes tools with fully deterministic relative paths, while the absolute paths often contain non-reproducible temp dir segments.

Is there a good reason to convert to absolute paths?

[lrytz]

There can be many tools invoking the compiler, even on the same codebase, using an absolute path potentially makes behavior more deterministic

[mbland]

Filed scala/scala#11192.