introduce std.Build.Module and extract some logic into it

[andrewrk]

Part 1: Per-Module Settings in the Build System

This moves many settings from std.Build.Step.Compile and into std.Build.Module, and then makes them transitive.

In other words, it adds support for exposing Zig modules in packages, which are configured in various ways, such as depending on other link objects, include paths, or even a different optimization mode.

Now, transitive dependencies will be included in the compilation, so you can, for example, make a Zig module depend on some C source code, and expose that Zig module in a package.

closes #14719

Part 2: Per-Module Settings in the Compiler

This branch also migrates many settings from being per-Compilation to being per-Module in the compiler itself. This required extensive changes to the interaction between the CLI as well as Compilation.

Before, the compiler frontend autogenerated only one @import("builtin") module for the entire compilation, however, this branch makes it honor the differences in modules, so that modules can be compiled with different optimization modes, code model, valgrind integration, or even target CPU feature set.

Part 2A: Yak Shave: Linker Options

This ballooned into an enormous changeset because I decided that the link.Options struct should not be stored in link.File, and I wanted to make those fields be resolved and set directly in link.File (the base struct) or the ObjectFormat-specific structs (such as MachO, Elf, etc.).

This change has been needed for a long time, so this pays off some tech debt. I hope it does not cause too many conflicts.

Merge Checklist

• [x] Rename std.zig.CrossTarget to std.zig.Target.Query
• [x] Make the dynamic linker part of std.Target
• [x] Introduce std.Build.ResolvedTarget to avoid redundantly resolving target queries
• [x] test on a project with a big dependency tree (groovebasin)
• [ ] Make sure there's a check and error message for other modules depending on the main module, since that's a scenario the frontend can't currently encode
• [x] Give each module its own @import("builtin") that can be different from the root module one.
• [ ] Make each function respect the optimization mode, even in the LLVM backend. Add a test for mixing modules with different optimization modes into the same compilation.
• [ ] write the upgrade guide

Upgrade Guide

todo

[ikskuh]

With that structure, how can i add a file that i can import with @embedFile? With the previous system, i added the binary file as a module and could just import it. Will that feature stay?

[xdBronch]

how can i add a file that i can import with @embedFile?

if im not wrong you would do

exe.root_module.addAnonymousImport("name", .{ 
    .target = target,
    .optimize = optimize, 
    .root_source_file = some_lazy_path,
});

[mlugg]

Nice - I had tried a few times to make this change in the frontend, but it does make sense to first make the corresponding build system changes, since they're the biggest practical issue. I can't look through this in much detail currently (got a lot going on), but I'm glad it's been worked on. Make sure there's a check and error message for other modules depending on the main module, since that's a scenario the frontend can't currently encode!

[andrewrk]

With that structure, how can i add a file that i can import with @embedFile? With the previous system, i added the binary file as a module and could just import it. Will that feature stay?

Still works. Perhaps a convenience API could be added to make it more clear that it's allowed.

[kubkon]

@andrewrk I believe https://github.com/ziglang/zig/pull/18207 should fix the CI for you.

[andrewrk]

@mlugg

Make sure there's a check and error message for other modules depending on the main module, since that's a scenario the frontend can't currently encode!

Better yet, the frontend can now encode this scenario, and I added test coverage for it!

[mlugg]

Nice! I'm excited to see this get in - it's the last puzzle piece needed to make the package manager generally usable, but it also unlocks a lot of interesting possibilities wrt modules in general. I'm looking forward to exploring some of these ideas :)