google-cloud-cpp icon indicating copy to clipboard operation
google-cloud-cpp copied to clipboard

Support idempotency token for Storage client

Open coryan opened this issue 2 years ago • 4 comments

The GCS API supports (or soon will support) an idempotency token header. The header allows GCS to detect if a request is a duplicate, and returns the same value for duplicate requests.

To implement this we need to:

  • Create the unique id in the RetryClient
  • The decorators below RetryClient need to accept a new "context" field on each function
  • The gRPC and REST-based implementations need to consume this field and send it to the service as a header.

It is time to split the RawClient stack in two, like we do for all the other libraries. The *Connection stack will be used for mocks and to implement the retry loop. The typical function will look like so:

  virtual StatusOr<BucketMetadata> GetBucketMetadata(
      GetBucketMetadataRequest const& request) = 0;

The *Stub will implement logging, tracing, and actually sending the data to the service, the typical function will look like so:

  virtual StatusOr<BucketMetadata> GetBucketMetadata(
      rest_internal::RestContext& context,
      Options const& options,
      GetBucketMetadataRequest const& request) = 0;

Note how the options are passed explicitly and not tunneled via CurrentOptions. Also note that the "context" object is REST-based. We can change both over time, as the classes in the *Stub hierarchy are not needed for mocking.


Java implemented this in:

https://github.com/googleapis/java-storage/pull/2027

Internally, the design doc is go/gcs-client-idem-token

coryan avatar Aug 02 '23 11:08 coryan

I also need to stop using the idempotency policy, as these RPCs are now always idempotent.

coryan avatar Aug 09 '23 15:08 coryan

This is blocked on service-side work.

coryan avatar Aug 11 '23 20:08 coryan

Still blocked on service-side work.

coryan avatar Feb 14 '24 20:02 coryan