bazelbuild
`external/gazelle~~go_deps~com_github_dougthor42_go_tree_sitter/python/scanner.c:1:10: fatal error: '../array.h' file not found` with gazelle 0.38+
🐞 bug report
Affected Rule
The issue is caused by the rule: rules_python_gazelle_plugin (version 1.1.0) when running with gazelle 0.38+Is this a regression?
Maybe, the issue starts with gazelle 0.38Description
A clear and concise description of the problem...When running gazelle bazel run //python:gazelle update go_tree_sitter fails to build
This is using the config from https://github.com/bazelbuild/rules_python/tree/main/examples/bzlmod_build_file_generation
When downgrading gazelle to 0.37 it runs (at the expense of features in go, namely the build_file_generation = "clean" which was introduced in 0.38)
The issue seems linked to https://github.com/smacker/go-tree-sitter/issues/175
🔬 Minimal Reproduction
https://github.com/bazelbuild/rules_python/tree/main/examples/bzlmod_build_file_generation with gazelle 0.38+
🔥 Exception or Error
INFO: Analyzed target //python:gazelle (51 packages loaded, 282 targets configured).
ERROR: /private/var/tmp/_bazel_ripper/5d3e93b0f5b6e19252cd0dca794d49db/external/gazelle~~go_deps~com_github_dougthor42_go_tree_sitter/python/BUILD.bazel:3:11: GoCompilePkg external/gazelle~~go_deps~com_github_dougthor42_go_tree_sitter/python/python.a failed: (Exit 1): builder failed: error executing GoCompilePkg command (from target @@gazelle~~go_deps~com_github_dougthor42_go_tree_sitter//python:python) bazel-out/darwin_x86_64-opt-exec-ST-d57f47055a04/bin/external/rules_go~~go_sdk~myproject__download_0/builder_reset/builder compilepkg -sdk external/rules_go~~go_sdk~myproject__download_0 -goroot ... (remaining 43 arguments skipped)
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
external/gazelle~~go_deps~com_github_dougthor42_go_tree_sitter/python/scanner.c:1:10: fatal error: '../array.h' file not found
#include "../array.h"
^~~~~~~~~~~~
1 error generated.
compilepkg: error running subcommand external/toolchains_llvm~~llvm~llvm_toolchain/bin/cc_wrapper.sh: exit status 1
Target //python:gazelle failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 29.799s, Critical Path: 5.75s
INFO: 33 processes: 9 internal, 24 darwin-sandbox.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target
🌍 Your Environment
Operating System:
Mac OS 15.1.1 (24B91) Intel
Output of bazel version:
Build label: 7.4.1-homebrew
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Tue Jan 1 00:00:00 1980 (315532800)
Build timestamp: 315532800
Build timestamp as int: 315532800
Rules_python version:
bazel_dep(name = "rules_python", version = "1.1.0")
bazel_dep(name = "rules_python_gazelle_plugin", version = "1.1.0")
Anything else relevant?
I think this may be duplicate of #1913. Does the error go away when using https://github.com/bazel-contrib/musl-toolchain or similar?
here is a repro branch https://github.com/remiphilippe/bazel-rules_python_array_c/tree/not-working I added musl to it (already had it on my repo) same issue.
ok I think I just found, the issue is not linked to gazelle version, but the default build_file_generation we use. If I add go_deps.gazelle_override( path = "github.com/dougthor42/go-tree-sitter", build_file_generation = "auto", )
to my MODULES.bazel it works (example branch https://github.com/remiphilippe/bazel-rules_python_array_c/tree/working)
We also see a similar issue. The change of build_file_generation didn't fix it.
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
ld.lld: error: undefined symbol: ts_current_free
>>> referenced by array.h:164 (./external/gazelle++go_deps+com_github_dougthor42_go_tree_sitter/python/../array.h:164)
>>> /tmp/rules_go_work-3424267487/cgo/github.com/dougthor42/go-tree-sitter/python/_x3.o:(_array__delete)
ld.lld: error: undefined symbol: ts_current_realloc
>>> referenced by array.h:185 (./external/gazelle++go_deps+com_github_dougthor42_go_tree_sitter/python/../array.h:185)
>>> /tmp/rules_go_work-3424267487/cgo/github.com/dougthor42/go-tree-sitter/python/_x3.o:(_array__reserve)
ld.lld: error: undefined symbol: ts_current_malloc
>>> referenced by array.h:187 (./external/gazelle++go_deps+com_github_dougthor42_go_tree_sitter/python/../array.h:187)
>>> /tmp/rules_go_work-3424267487/cgo/github.com/dougthor42/go-tree-sitter/python/_x3.o:(_array__reserve)
compilepkg: error running subcommand external/hermetic_cc_toolchain++toolchains+zig_sdk/tools/x86_64-linux-gnu.2.19/c++: exit status 1
FWIW I patched this when using tree-sitter: https://github.com/aspect-build/aspect-cli/blob/main/third_party/github.com/smacker/go-tree-sitter/cc_library.patch
Is this at all related to the reasons for using a fork of go-tree-sitter?
@jbedard thanks for the patch, it works!
however, not if you are using hermetic_cc_toolchain on linux/amd64, like @tokongs is. I've removed the dependency and removed the toolchain resolution and the build was successful. Do you are an inkling, why that'd make a difference?
an additional data point: hermetic_cc_toolchain on darwin/aarch64 works
More data points:
build_file_generation = "auto"doesn't work with hermetic_cc_toolchain, with error like "ld.lld: error: undefined symbol: ts_current_free"- this patch works with hermetic_cc_toolchain on linux/amd64
