Clarification: Ordering of entries in BatchReadBlobsResponse

[santigl]

The API allows a client to request a list of blobs using BatchReadBlobs(). That sends a BatchReadBlobsRequest with a list of digests to the server, which in turn replies with a BatchReadBlobsResponse.

That response will contain, in the best-case scenario, a list of blobs that the client asked for. Since the specification states that the blobs should be allowed to fail separately, it could happen that not all of the blobs requested are contained in the response, although there will be an entry for them in the reply with status set to an error code.

The current specification, however, does not specify what should happen with the ordering of the blobs contained in the BatchReadBlobsResponse.

Should there be any guarantees or expectations regarding the relation between the ordering of the blobs in the response and the order of their corresponding requests?

[bergsieker]

You are correct that the current API does not specify an ordering. Note that there's no ambiguity here: each result includes a digest, so the client can understand which bytes/status belongs to each requested blob. I don't believe that the lack of ordering was by design, so we could potentially specify an ordering, although I'm not sure how much it would simplify clients in reality. @buchgr WDYT?

[ulfjack]

If the ordering is required, then the digests don't have to be set, reducing the response size. What about eliding successful replies? The client could implicitly assume that anything not mentioned was OK.

[jmillikin]

How about using a uint32 request_index in each response item? That would provide the same functionality as the current protocol, without requiring the server to send back the original digests.

edit: ah, this has already been suggested at https://github.com/bazelbuild/remote-apis/issues/160

Something like this:

message FindMissingBlobsRequest {
  // [...]
  repeated Digest blob_digests = 2;
  bool respond_digest_indexes = 3;
}

message FindMissingBlobsResponse {
  // not populated if request's `respond_digest_indexes` was true
  repeated Digest missing_blob_digests = 2;

  // request::blob_digests[missing_blob_digest_indexes[N]] == missing_blob_digests[N]
  repeated uint32 missing_blob_digest_indexes = 3;
}

message BatchUpdateBlobsRequest {
  // [...]
  repeated Request requests = 2;
  bool respond_digest_indexes = 3;
}

message BatchUpdateBlobsResponse {
  message Response {
    // [...]
    uint32 request_index = 3;
  }
}

// same for BatchReadBlobsRequest / BatchReadBlobsResponse