Build Oak using Bazel

[tiziano88]

Tracking / discussion issue about considering switching to Bazel to build the Rust code. There is no concrete plan yet, but we can reference this issue over time with thoughts and potential blockers. Feel free to comment with thoughts or opinions, this is not a plan of record.

Pros:

• easier to postprocess binaries (e.g. package, strip, merge) via bazel rules
• can remove most of the unwieldy logic in xtask
• easier to sync code into google3
• more precise dependency tracking / caching across languages

Cons:

• unclear IDE support -- would VSCode autocomplete / xrefs work?
  • maybe useful: https://bazelbuild.github.io/rules_rust/rust_analyzer.html
• generally less used by the Rust community

[tiziano88]

One thing that we should determine before we go too deep in this direction, is whether it is possible to have a rust library, on which three rust binaries depend on (one wasm, one x86-linux, one x86-none), and also if another binary (e.g. an integration test) is able to depend on all those binaries (as data) as well as the original library (as code).

[tiziano88]

BTW @bmclarnon was just telling me about Bazel transitions, which may be the solution to the problem I mentioned above. (thanks Brett!)

[bmclarnon]

BTW @bmclarnon was just telling me about Bazel transitions, which may be the solution to the problem I mentioned above. (thanks Brett!)

https://bazel.build/rules/lib/builtins/transition is what I was referring to. We use them in the federated-compute repo to build C++ code with a different sdlib and CXX and link flags based on the context: https://github.com/google/federated-compute/blob/main/fcp/tensorflow/system_provided_tf/templates/system_provided_tf.bzl.tpl. I think you can use the same approach to change the target platform so that bazel will select a different rust toolchain.

I'm happy to chat more if/when you have any questions.

[tiziano88]

When we start looking into this, I wonder if we should start with bazelifying something else like the system image for Oak Containers, which currently takes a long time to build. We could then use RBE to cache it

[tiziano88]

@andrisaar @conradgrobler @ernoc WDYT?

Also the system image is quite separate from the rest, so it's easier to migrate in isolation.

It would also require building the linux kernel via bazel though, which may not be trivial (but definitely doable).

[conradgrobler]

That sounds like a useful starting point. Another option that might be slightly smaller is to build the stage 1 initial RAM disk using Bazel first, as that won't involve figuring out how to build the kernel and only requires building one Rust binary. It does require generating a CPIO archive thought, so would still not be trivial.

[tiziano88]

cc @jblebrun

[tiziano88]

@ernoc should we "move" this issue to buganizer (just copy some of its content there and close this)? IIRC you mentioned you prefer that

[ernoc]

Hm.. let's leave it here to preserve context and comment history - for future issues I'll use b/ . For this one issue I'll manage it through GH. Unless there's a more compelling reason ofc.

[tiziano88]

@bmclarnon IIRC you also mentioned that you managed to get mostly working versions of our Rust crates in bazel using the automated conversion tool. Did you use https://bazelbuild.github.io/rules_rust/crate_universe.html for that, or the older https://github.com/google/cargo-raze (or something else)?

[bmclarnon]

@bmclarnon IIRC you also mentioned that you managed to get mostly working versions of our Rust crates in bazel using the automated conversion tool. Did you use https://bazelbuild.github.io/rules_rust/crate_universe.html for that, or the older https://github.com/google/cargo-raze (or something else)?

I used https://bazelbuild.github.io/rules_rust/crate_universe.html. It worked great for std builds, but I ran into issues with no_std builds when the same crate was used with both std and no_std features (e.g., std during tests or build steps). I think the solution is to have two separate crates_repository calls: one for dependencies used in a std environment, and one for dependencies used in a no_std environment.

I didn't experiment with the build.rs support since I anticipated using https://bazelbuild.github.io/rules_rust/rust_proto.html for protos and a new starlark rule for micro_rpc.

[ernoc]

@bmclarnon did you need to deal with crates_repository complaining about not finding info for rust-hypervisor-firmware-virtio, as it's an internal path dependency (IIUC path dependencies are not supported yet).

[bmclarnon]

@ernoc no, I used bazel dependencies for anything within the workspace. I suspect you might need to build rust-hypervisor-firmware-virtio with bazel as well, not Cargo + a path dependency.