Idea for a local execution and cache

[danmx]

Have you considered extending buckd to act as a simple remote cache + remote worker (maybe backed by a container runtime) and e.g. expose REAPI via a socket?

This would allow the client to use the same API and it only puts extra logic on a daemon.

[ndmitchell]

Great idea! Yep, that's something we've considered. The other alternative is to make a generic "local remote execution" client separate to Buck2, which can be reused by both Buck2 and other RE compatible projects (e.g. Bazel). In an ideal world the local RE client would be a Rust library, so other people can use it stand alone, and we can also easily import it into buck2d and offer it with a zero-install option for Buck2.

I don't think anyone is working on it yet.

[danmx]

So what's the direction? Do you look for contributions?

[ndmitchell]

I guess a contributor would be most welcome. Ideally doing it in the generic sense as a library on crates.io that we then pull in, but open to other alternatives.

[thoughtpolice]

I've begun to sketch out an implementation of the 3 remote execution APIs (logstream, asset, execution) here, using tonic:

https://github.com/thoughtpolice/reapi-server

It's extremely primitive, and every RPC endpoints cancels as unimplemented!(), but you can query its schema with grpcurl to see its structure. While I'm personally experimenting with BuildBarn for a clustered solution, a local only solution will still be needed IMO, and it might be useful for other things later on.

The goal would be inevitably for buck2d to use this, I suppose.

I'll be implementing each API as I can slowly, and some of it might be reusable, but please, I welcome anyone helping. :)

[danmx]

Check out turbo-cache. Maybe some parts could be reused