Aaron Siddhartha Mondal

# Description Please include a summary of the changes and the related issue. Please also include relevant motivation and context. Fixes # (issue) ## Type of change Please delete options...

Sometimes our GHA workflows fail due to issues that seem like they stem from some sort of cache corruption. Figure out why and harden the pipelines.

At the moment it's still necessary to have an OS-specific CA installed. Apart from making https resolution less reproducible this complicates packaging and increases container size. Adding external CAs should...

Spin-off from #749. On linux, the `nix build .#` target uses a cross-toolchain targeting a static musl system. Builds invoked on darwin (MacOS) hosts target darwin natively by default. Those...

Add AWS-native k8s Deployment

12

I'd like to run turbo-cache in a k8s cluster deployed with Pulumi so that we can automatically set it up for users as part of rules_ll. Simple yaml-manifests would be...

Panic during `running_actions_manager_tests::kill_all_waits_for_all_tasks_to_finish`

2

Reproducible by uncommenting https://github.com/TraceMachina/nativelink/blob/35daf433f01150cdf3b5da4e9a97e561be03cbdf/flake.nix#L142-L149 and running `nix flake check -L`. ``` nativelink-nextest> SIGABRT [ 0.093s] nativelink-worker::running_actions_manager_test running_actions_manager_tests::kill_all_waits_for_all_tasks_to_finish nativelink-nextest> --- STDOUT: nativelink-worker::running_actions_manager_test running_actions_manager_tests::kill_all_waits_for_all_tasks_to_finish --- nativelink-nextest> running 1 test nativelink-nextest> nativelink-nextest> ---...

Potential improvements to fastcdc

4

The code at https://github.com/TraceMachina/nativelink/blob/main/nativelink-util/src/fastcdc.rs seems unnecessary. Potential alternatives: - https://github.com/nlfiedler/fastcdc-rs - https://github.com/florian-g2/fastcdc-rs-alt There is also an interesting algorithm called `UltraCDC` from some of the `FastCDC` authors that seems worth investigating:...

Since Rust 1.75.0 the language has native support for `async fn` in Traits for some usecases (https://blog.rust-lang.org/2023/12/21/async-fn-rpit-in-traits.html). Dynamic dispatch isn't supported yet (https://blog.rust-lang.org/2023/12/21/async-fn-rpit-in-traits.html#dynamic-dispatch), meaning that a full removal of the...

Worker connection failures on startup shouldn't raise errors

1

At the moment we raise errors when the worker can't find the scheduler: https://github.com/TraceMachina/nativelink/actions/runs/7558646440/job/20580653217?pr=614#step:9:197 This might lead to issues when running health and readiness checks and readiness checks in K8s....

Autogenerate docs from existing READMEs

2

Syncing documentation manually is a tedious, error prone task. Since we have Docusaurus with per-commit deployments it should be fairly straightforward to auto-sync all information from the READMEs. Related: #593