bazel-buildfarm
bazel-buildfarm copied to clipboard
FetchService.fetchBlob giving NumberFormatException for a valid SHA256
We are trying to call the Fetch Blob method of the FetchService (buildfarm server running in one of our eks cluster) using below request params :
grpcurl -d ' {
"uris": [
"https://github.com/bazelbuild/remote-apis/archive/v2.0.0.tar.gz"
],
"qualifiers": [
{
"name": "checksum.sri",
"value": "sha256-79204ed1fa385c03b5235f65b25ced6ac51cf4b00e45e1157beca6a28bdb8043"
}
]
}
' ramjet.dev.aws.jpmchase.net:443 build.bazel.remote.asset.v1.Fetch.FetchBlob
ERROR: Code: Unknown Message:
This is a valid sha and we are able to locally download the artifacts, using the same, but getting below error on buildfarm server :
[SEVERE ] io.grpc.internal.SerializingExecutor run - Exception while executing runnable io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed@377418e7 java.lang.NumberFormatException: [efddb4e1e7757dadfce5cd376f9db7e5feb96f6e5c79de9a739d5c7f86f4d1ee397b5d79edb79c6ba6b6f1b75bf34e37] is not a valid SHA256 hash. at build.buildfarm.common.DigestUtil.build(DigestUtil.java:192) at build.buildfarm.common.DigestUtil.build(DigestUtil.java:198) at build.buildfarm.server.FetchService.parseChecksumSRI(FetchService.java:105) at build.buildfarm.server.FetchService.fetchBlob(FetchService.java:52) at build.buildfarm.server.FetchService.fetchBlob(FetchService.java:37) at build.bazel.remote.asset.v1.FetchGrpc$MethodHandlers.invoke(FetchGrpc.java:274) at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182) at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35) at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23) at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40) at io.grpc.util.TransmitStatusRuntimeExceptionInterceptor$1.onHalfClose(TransmitStatusRuntimeExceptionInterceptor.java:74)
Seems it is converting the sha256 to sha384 format and validating against that. Upon debugging, we found that the parseChecksumSRI method is generating incorrect hexDigest -
private Digest parseChecksumSRI(String checksum) {
String[] components = checksum.split("-");
if (components.length != 2) {
throw Status.INVALID_ARGUMENT
.withDescription(format("Invalid checksum format '%s'", checksum))
.asRuntimeException();
}
String hashFunction = components[0];
String encodedDigest = components[1];
DigestUtil digestUtil = DigestUtil.forHash(hashFunction.toUpperCase());
return digestUtil.build(BaseEncoding.base64().decode(encodedDigest), -1);
}
This is causing the build method to fail and throw error -
public Digest build(String hexHash, long size) { if (!hashFn.isValidHexDigest(hexHash)) { throw new NumberFormatException( String.format("[%s] is not a valid %s hash.", hexHash, hashFn.name())); } return buildDigest(hexHash, size); }
Kindly assist.
Per the sri docs, the digest specified must be a base64 encoding of the bytes (actual bytes, not hex) of the digest, not hex plaintext, as yours is. The appropriate replacement for your supplied digest would be sha256-eSBO0fo4XAO1I19lslztasUc9LAOReEVe+ymoovbgEM=
(edited: forgot the requirement that the value be base64 for the binary digest)