bazel-buildfarm icon indicating copy to clipboard operation
bazel-buildfarm copied to clipboard

FetchService.fetchBlob giving NumberFormatException for a valid SHA256

Open KhushbooJ opened this issue 3 years ago • 1 comments

We are trying to call the Fetch Blob method of the FetchService (buildfarm server running in one of our eks cluster) using below request params :

grpcurl -d '  {                                                                                         
   "uris": [
     "https://github.com/bazelbuild/remote-apis/archive/v2.0.0.tar.gz"                                                         
   ],
   "qualifiers": [
     {
       "name": "checksum.sri",
       "value": "sha256-79204ed1fa385c03b5235f65b25ced6ac51cf4b00e45e1157beca6a28bdb8043" 
     }
   ]
 }
' ramjet.dev.aws.jpmchase.net:443 build.bazel.remote.asset.v1.Fetch.FetchBlob

ERROR: Code: Unknown Message:

This is a valid sha and we are able to locally download the artifacts, using the same, but getting below error on buildfarm server :

[SEVERE ] io.grpc.internal.SerializingExecutor run - Exception while executing runnable io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed@377418e7 java.lang.NumberFormatException: [efddb4e1e7757dadfce5cd376f9db7e5feb96f6e5c79de9a739d5c7f86f4d1ee397b5d79edb79c6ba6b6f1b75bf34e37] is not a valid SHA256 hash. at build.buildfarm.common.DigestUtil.build(DigestUtil.java:192) at build.buildfarm.common.DigestUtil.build(DigestUtil.java:198) at build.buildfarm.server.FetchService.parseChecksumSRI(FetchService.java:105) at build.buildfarm.server.FetchService.fetchBlob(FetchService.java:52) at build.buildfarm.server.FetchService.fetchBlob(FetchService.java:37) at build.bazel.remote.asset.v1.FetchGrpc$MethodHandlers.invoke(FetchGrpc.java:274) at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182) at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35) at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23) at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40) at io.grpc.util.TransmitStatusRuntimeExceptionInterceptor$1.onHalfClose(TransmitStatusRuntimeExceptionInterceptor.java:74)

Seems it is converting the sha256 to sha384 format and validating against that. Upon debugging, we found that the parseChecksumSRI method is generating incorrect hexDigest -

private Digest parseChecksumSRI(String checksum) {
    String[] components = checksum.split("-");
    if (components.length != 2) {
      throw Status.INVALID_ARGUMENT
          .withDescription(format("Invalid checksum format '%s'", checksum))
          .asRuntimeException();
    }
    String hashFunction = components[0];
    String encodedDigest = components[1];
    DigestUtil digestUtil = DigestUtil.forHash(hashFunction.toUpperCase());
    return digestUtil.build(BaseEncoding.base64().decode(encodedDigest), -1);
  }

This is causing the build method to fail and throw error - public Digest build(String hexHash, long size) { if (!hashFn.isValidHexDigest(hexHash)) { throw new NumberFormatException( String.format("[%s] is not a valid %s hash.", hexHash, hashFn.name())); } return buildDigest(hexHash, size); }

Kindly assist.

KhushbooJ avatar Sep 09 '22 10:09 KhushbooJ

Per the sri docs, the digest specified must be a base64 encoding of the bytes (actual bytes, not hex) of the digest, not hex plaintext, as yours is. The appropriate replacement for your supplied digest would be sha256-eSBO0fo4XAO1I19lslztasUc9LAOReEVe+ymoovbgEM=

(edited: forgot the requirement that the value be base64 for the binary digest)

werkt avatar Sep 14 '22 00:09 werkt