thought-machine
Remove dependency on build.bazel.remote.asset.v1.Fetch to make it compatible with BuildBarn
As in title. Default implementation of BuildBarn do not provide build.bazel.remote.asset.v1.Fetch . You probably can emulate the functionality with other API endpoints, given this one is marked as experimental.
Can we make Please not rely on experimental features?
It's not easy to replicate the functionality using other API endpoints. You could try to convert the remote_file definition into some command that gets executed, but it's not trivial to match all the semantics in Bash, plus doing SRI checks etc.
Doesn't Buildbarn support it via bb-remote-asset?
given this one is marked as experimental.
Can we make Please not rely on experimental features?
What is experimental here, sorry? The remote asset proto has been v1 for some time, and there's no mention in it that I can see of any of the RPCs being experimental.
You are right, it's experimental in bazel-remote, not Bazel. My bad.
I enabled it manually and ended up diagnosing that it's unable to handle requests like "remote-asset-1 | 2025/12/08 23:04:00 Fetching Blob [https://github.com/please-build/arcat/releases/download/v1.3.1/arcat-1.3.1-linux_amd64] with qualifiers [name:"checksum.sri" value:"sha256-avLPEIWSU1cBqpOV86Xe60il376BdKjr49VruT3iwlU= sha256-UHDvBdFMZqhdQ49ADG/3NKI4M5KXddaCS2kge3BANL8= sha256-Ba1qxFvjpMoSOLsb0JIHpZb4/1+IVBX430/y3ISfoE4= sha256-rshUJTVSkeUVzRCsCt3sOlvJ4FydB68BrKjDSq8PEiI= sha256-gmbLlcyEsjZCvKZWf4tL0Y3jmciHy1hFq2qQHQ26VNI="]
Though SRI standard supports "multiple SHAs per asset" (which they don't, but I am working on it ) , it assumes these are different SHAs algorithms for the same binary, not same algorithm and different SHAs for different binaries.
So I guess I can lower my ask: "can we get Please to work with bb-remote-asset"?
If no, what is the backend you are testing with?
"can we get Please to work with bb-remote-asset"?
Yup, that seems a very reasonable goal. I think I have tested it in the past and it did work, but that was some time ago; I certainly can't have tried multiple hashes which must never have worked. This should absolutely be the goal though, and should be very achievable.
Your PR seems like the ideal approach to me. There was a bit of discussion about multiple hashes in the REAPI working group some time back, we ended up agreeing on this approach but I don't think everyone realised that was actually supported by the spec initially.
In the meantime I suppose we could also reconfigure the internal package to switch on the host os/arch with only one hash emitted. It's not super nice but would work.
If no, what is the backend you are testing with?
We had an internal one at Thought Machine. For context, I left a little while back, so while I'm highly familiar with how it used to work, these days I just moonlight on some relevant issues in this repo :-)
So I just spent an hour or two and got BuildFarm working. And it's saying the same thing: you cannot have multiple values for the same SHA function. So even if bb-remote-asset merges my change, it won't fix the problem. We need a fix in PLZ too.
OK, I figured it out. Just removing SHA values for different platforms of arcat fixes buildbarn. Of course starting it with v1.Fetch requires some modification to "official docker compose" files, but I confirm it is compatible when only one SHA is sent.