buck2 icon indicating copy to clipboard operation
buck2 copied to clipboard

buck2_client: download traces via `buck2.log_url` config key

Open thoughtpolice opened this issue 5 months ago • 2 comments

When doing a command like buck2 log replay --trace-id ... it helps to be able to download previously created logs from users, automated CI, etc. This functionality exists within Meta, as logs are available for download through the "Manifest" system, but it isn't usable in OSS, despite being useful for diagnostics and getting help.

The only missing thing to really get things working is a download mechanism for log files, and a way to know where they should come from.

With this patch, if a user configures the buck2.log_url key, which is expected to look something like:

[buck2]
log_url = https://example.com

Then, upon executing a log command with a --trace-id flag, this server will be queried with a:

GET /v1/get/{uuid}

request, which is expected to return the raw zst-encoded protobuf file. So, buck2 will do that and download the trace, and then use it like normal.

The request is done with curl, though in principle it could be done within Buck itself.

This shares as much code as possible with the existing infrastructure and tries to only insert a small key set of #[cfg(fbcode_build)] directives. Notably, the path Meta uses for their "Manifold" client is still fully available in the OSS version; fbcode_build is only used to gate what the default choice is.

How the server gets access to these files and how they are uploaded is another question. But for now, this can be done several ways outside of buck2 core, so this is good enough.

GitHub Issue: https://github.com/facebook/buck2/issues/441

thoughtpolice avatar Sep 05 '24 00:09 thoughtpolice