buck2
buck2 copied to clipboard
facebook
cxx_zig_toolchain fails to build binaries
Currently I'm getting this error:
buck2 build :main
Starting new buck2 daemon...
Connected to new buck2 daemon.
Action failed: root//:main (c_compile main.c)
Local command returned non-zero exit code 1
Reproduce locally: `env -- 'BUCK_SCRATCH_PATH=buck-out/v2/tmp/root/8989b40cb9216c61/c_compile/main.c' buck-out/v2/gen/to ...<omitted>... jects__/main.c.o @buck-out/v2/gen/root/904931f735703749/__main__/.c.cxx_compile_argsfile -c ./main.c (run `buck2 log what-failed` to get the full command)`
stdout:
stderr:
error: unable to parse command line parameters: NestedResponseFile
Build ID: 9fe5020c-bba6-48a0-86fc-230305ee96f6
Network: Up: 0B Down: 47MiB
Loading targets. Remaining 0/11 243 dirs read, 100 targets declared
Analyzing targets. Remaining 0/45 58 actions, 58 artifacts declared
Executing actions. Remaining 0/15 1.8s exec time total
Command: build. Finished 3 local
Time elapsed: 4.4s
BUILD FAILED
Failed to build 'root//:main (prelude//platforms:default#904931f735703749)'
buck2 log what-failed
Showing commands from: buck2 build :main
build root//:main (prelude//platforms:default#904931f735703749) (c_compile main.c) local env -- 'TMPDIR=/app/buck-out/v2/tmp/root/8989b40cb9216c61/c_compile/main.c' 'BUCK_SCRATCH_PATH=buck-out/v2/tmp/root/8989b40cb9216c61/c_compile/main.c' 'BUCK2_DAEMON_UUID=ec19ee45-75a3-4c2f-9adc-befe6fa86667' 'BUCK_BUILD_ID=9fe5020c-bba6-48a0-86fc-230305ee96f6' buck-out/v2/gen/toolchains/904931f735703749-904931f735703749/__cxx__/zig_cc.sh -o buck-out/v2/gen/root/904931f735703749/__main__/__objects__/main.c.o @buck-out/v2/gen/root/904931f735703749/__main__/.c.cxx_compile_argsfile -c ./main.c
Using these toolchains:
load("@prelude//toolchains/cxx/zig:defs.bzl", "download_zig_distribution", "cxx_zig_toolchain")
load("@prelude//toolchains:genrule.bzl", "system_genrule_toolchain")
load("@prelude//toolchains:python.bzl", "system_python_bootstrap_toolchain")
load("@prelude//toolchains:rust.bzl", "system_rust_toolchain")
# load("//rust:rules.bzl", "rust_toolchain")
download_zig_distribution(
name = "zig-x86_64-linux",
version = "0.14.0",
arch = "x86_64",
os = "linux",
)
download_zig_distribution(
name = "zig-x86_64-macos",
version = "0.14.0",
arch = "x86_64",
os = "macos",
)
download_zig_distribution(
name = "zig-x86_64-windows",
version = "0.14.0",
arch = "x86_64",
os = "windows",
)
alias(
name = "zig",
actual = select({
"@prelude//os:linux": ":zig-x86_64-linux",
"@prelude//os:macos": ":zig-x86_64-macos",
"@prelude//os:windows": ":zig-x86_64-windows",
}),
)
cxx_zig_toolchain(
name = "cxx",
distribution = ":zig",
visibility = ["PUBLIC"],
)
system_genrule_toolchain(
name = "genrule",
visibility = ["PUBLIC"],
)
system_python_bootstrap_toolchain(
name = "python_bootstrap",
visibility = ["PUBLIC"],
)
system_rust_toolchain(
name = "rust",
default_edition = "2024",
visibility = ["PUBLIC"],
)
And this root BUCK file:
cxx_binary(
name = "main",
srcs = ["main.c"],
)
With main.c:
int main(void) {
return 0;
}
The problem seems to be that the at argsfile itself contains at files, which zig does not seem to support.
You can prove this by deleting the contents of buck-out/v2/gen/root/904931f735703749/__main__/.c.cxx_compile_argsfile (you shouldn't normally mess with buck-out manually): the next build will succeed.
This should probably be reported to https://github.com/ziglang/zig instead.
For what it's worth, I ran into this problem myself, and made it work by writing a small wrapper script to re-compact the args files. The basic script is simple enough, although I feel bad about re-writing the args file. Probably going to get in trouble for that at some point:
"""A script to flatten argument lists for the zig compiler.
The bug is that, as of 0.14.1, zig doesn't handle nested argument files. That's
really easy to work around: just re-flatten the argument files into a single
argument file. No problem, and we can use the rest of buck2's machinery!
"""
import subprocess
def flatten_args(args: list[str]):
for arg in args:
arg = arg.strip()
if arg.startswith("@"):
with open(arg[1:], "r", encoding="utf-8") as f:
lines = f.readlines()
for arg in flatten_args(lines):
yield arg
else:
yield arg
def rewrite_arg_file(file_name):
with open(file_name, "r", encoding="utf-8") as f:
lines = f.readlines()
with open(file_name, "w", encoding="utf-8") as f:
for arg in flatten_args(lines):
# print(arg)
f.write(arg)
f.write("\n")
def main(argv: list[str]) -> int:
args = argv[1:]
flat_args = []
for arg in args:
if arg.startswith("@"):
rewrite_arg_file(arg[1:])
flat_args.append(arg)
proc = subprocess.run(flat_args)
return proc.returncode
if __name__ == "__main__":
import sys
sys.exit(main(sys.argv))
The script is made available as toolchains//zig:zc:
python_bootstrap_binary(
name="zc",
main="zc.py",
visibility = ["PUBLIC"],
)
Then I made a little shim rule (in my toolchains//zig:defs.bzl) that wraps zig_distribution to wrap the compiler binary:
load("@prelude//toolchains/cxx/zig:defs.bzl", _zig_distribution="zig_distribution", "ZigDistributionInfo")
def _zig_wrapper_impl(ctx):
compiler_wrapper = ctx.attrs._wrapper[RunInfo]
original_compiler = ctx.attrs.dist[RunInfo]
return [
ctx.attrs.dist[DefaultInfo],
RunInfo(args = [compiler_wrapper, original_compiler]),
ctx.attrs.dist[ZigDistributionInfo],
]
_zig_wrapper=rule(
impl=_zig_wrapper_impl,
attrs = {
"dist": attrs.exec_dep(providers = [RunInfo, ZigDistributionInfo]),
"_wrapper": attrs.exec_dep(providers = [RunInfo], default="//zig:zc"),
},
)
def zig_distribution(
name: str,
dist: str,
prefix: str,
suffix: str,
version: str,
arch: str,
os: str,
visibility: list[str],
):
od_name = "{}-original".format(name)
_zig_distribution(
name=od_name,
dist=dist,
prefix=prefix,
suffix=suffix,
version=version,
arch=arch,
os=os,
visibility=visibility,
)
_zig_wrapper(
name=name,
dist=":{}".format(od_name),
visibility=visibility,
)
