Build a proper portable Zig `build` (zig upstream issue!?)

[plajjan]

Acton uses the Zig build system to do the low level build. The acton compiler first does its compilation passes and produces .c / .h output, which is then fed into the Zig build system. There are currently two ways in which zig build is run by the acton compiler:

• for "persistent" projects, where we do essentially a vanilla zig build after generating a build.zig file
• for compilation of standalone .act files, where we have our own hacked up build builder executable and just run that This issue is about trying to reduce the hackery for the second case. We prefer to align on the idiomatic Zig ways.

All Acton projects (how Acton source code is organized) look quite similar. They have a src/ directory with Acton source code and a out/types directory where the generated .c / .h files are stored. out/bin is for outputted binaries (where zig is told to put them). Projects that have dependencies are a tad more complex and we generate the build.zig and build.zig.zon customized per such project. The initial compilation of build_runner.zig + build.zig -> build executable takes time, which is acceptable, since it is then cached so subsequent runs is very fast.

Acton supports compiling standalone .act files, like you have ~/foo.act in your home dir and you can run acton foo.act to compile this into an executable binary. Under the hood, this works by creating a temporary directory like /tmp/actonc-123456 and creating a project structure in there, so we have /tmp/actonc-123456/src and compiled output goes into /tmp/actonc-123456/out/types etc. The challenge here is that if we generate a /tmp/actonc-123456/build.zig, it will first be compiled which can take upwards of 10 seconds. Unlike normal projects that are relatively persistent, these temporary projects used for standalone .act file compilation are completely ephemeral. Each compilation run will thus take at least 10 seconds and this is just not a very nice experience at all. Since compilation of standalone .act files is completely uniform (no dependencies or other things that make the build.zig unique), it really should be possible to cache, but I think Zig doesn't realize it is since the paths are always unique. Can we hint / lie to the caching system somehow, like "hey, look at the checksum of this file, you really do have it cached already". This is still problematic due to dependencies absolute paths though, see below...

(AFAIK) The build executable is produced by taking build_runner.zig and adding in the local build.zig file, so the build_runner can reach the build function in the build.zig module. A dependencies.zig file is generated and injected that contains Zig structs with all the dependencies. This file is normally generated by the zig compiler from build.zig.zon. Instead, we have our own hard-coded dependencies.zig. The big difference is that the zig compiler turns relative paths in build.zig.zon into absolute ones when it produces dependencies.zig and later the build executable, i.e. the build executable is hard-coded for some absolute paths, which makes it impossible to reuse for our temporary projects. By injecting our own dependencies.zig, we can use relative paths. Since we actually do run in various different directories and need to reach things outside of the local directory, we use symlinks to take us to the right place. The acton compiler creates these symlinks as necessary to guarantee that the zig builder runs in a "known environment".

Solutions I can think of

• zig build to be so fast that compiling the build executable is instant and thus this whole problem goes away
• can paths be relative in dependencies.zig to the build root (where build.zig is)?
  • I think overall Zig appears to be doing a great job with respect to openat, b.path() and using relative vs absolute paths in different places
  • however, isn't the point really to do normalization of paths, so for relative input paths, we need to know about what they are relative to, so we can translate that into something else!?
  • today it is translated into an absolute path, but can't it be a path relative to the build root???
• just continue like today, it does work, so unless there are big changes to the Zig build system, I suppose we can continue

Pointers:

• https://github.com/actonlang/acton/blob/main/builder/dependencies.zig
• https://github.com/actonlang/acton/blob/main/Makefile#L367 for how we build our own build executable

[plajjan]

I'm going to close this. I think it's pretty clear that we are violating the intention for how Zig was to be used and it's just a world of pain down that route. For project builds, it's not too bad having to wait a tiny bit for the builder to be built since later builds have it cached. For standalone files, we make use of temporary directories so that the build.zig is still cached (since we rotate between a few pre-defined scratch folders, not entirely random temporary dir names). Also, in newer versions of Zig, build.zig is built by Zig codegen, not LLVM, which should bring a nice speedup too.