kafka-connect-storage-common icon indicating copy to clipboard operation
kafka-connect-storage-common copied to clipboard

google-cloud-storage Library Requires Update Due to Issue With Lifecycles

Open archy-bold opened this issue 2 years ago • 0 comments

I received the below email from Google Cloud regarding the GCS connector needing to be upgraded to version 1.114.0 in order to work with certain lifecycle conditions. I checked and version 5.5.8 of the connector uses version 1.112.0 of the gooogle-cloud-storage library.

Update your Java Libraries for them to continue working with certain Google Cloud Storage Object Lifecycle Management conditions.

Dear Google Cloud Storage Customer,

We're writing to let you know that as of August 23, 2021, when completing a read, list, update or patch, Cloud Storage buckets with the customTimeBefore, daysSinceCustomTime, daysSinceNoncurrentTime and noncurrentTimeBefore lifecycle conditions will require an update of your Java client library to version 1.114.0 or higher.

In addition, if you consume the library through the Google Cloud Java libraries-bom, you will need to upgrade these Java client libraries to 20.5.0 or higher as soon as possible.

What do I need to know?

We have identified an issue with older versions of the Java Client Libraries and Object Lifecycle Management. Versions 1.50.0-1.111.2 of the Cloud Storage client library and 1.0.0-10.0.0 of the Google Cloud Java libraries-bom do not support the customTimeBefore, daysSinceCustomTime, daysSinceNoncurrentTime and noncurrentTimeBefore lifecycle conditions.

For Java client version < 1.114.0 and Google Cloud Java libraries-bom version < 20.5.0, the Cloud Storage API has the following client-side behaviors:

  • If a user is attempting to update bucket metadata that contains new lifecycle conditions, this will result in a failure with a 400 HTTP error code.
  • For read-only operations (e.g. get or list), new lifecycle conditions are removed from the response.

What do I need to do?

To be able to use Cloud Storage with those lifecycle conditions, update your Java client library to version 1.114.0 or higher as soon as possible. Maven Central Repository details and an Overview of the latest Storage Parent will assist you in integrating your Java client library to work with the Cloud Storage Object Lifecycle Management conditions mentioned above.

What if I’m unable to upgrade my Java version?

If it's not possible to upgrade your Java client library version, you can alternatively select specific bucket fields that are needed. Note, however, that you will not be able to modify lifecycle rules with the old version of the library this way. Instead, you will first need to upgrade the library to do so.

The following example gets a bucket by selecting all bucket fields except the lifecycle rules:

Bucket bucket = storage.get("bucket-name",
    Storage.BucketGetOption.fields(Storage.BucketField.ID,
    Storage.BucketField.SELF_LINK,
    Storage.BucketField.NAME,
    Storage.BucketField.TIME_CREATED,
    Storage.BucketField.METAGENERATION,
    Storage.BucketField.ACL,
    Storage.BucketField.DEFAULT_OBJECT_ACL,
    Storage.BucketField.OWNER,
    Storage.BucketField.LABELS,
    Storage.BucketField.LOCATION,
    Storage.BucketField.LOCATION_TYPE,
    Storage.BucketField.WEBSITE,
    Storage.BucketField.VERSIONING,
    Storage.BucketField.CORS,
    Storage.BucketField.STORAGE_CLASS,
    Storage.BucketField.ETAG,
    Storage.BucketField.ENCRYPTION,
    Storage.BucketField.BILLING,
    Storage.BucketField.DEFAULT_EVENT_BASED_HOLD,
    Storage.BucketField.RETENTION_POLICY,
    Storage.BucketField.IAMCONFIGURATION,
    Storage.BucketField.LOGGING,
    Storage.BucketField.UPDATED));

The following example lists buckets by selecting all bucket fields except the lifecycle rules:

storage.list(Storage.BucketListOption.fields(Storage.BucketField.ID,
    Storage.BucketField.SELF_LINK,
    Storage.BucketField.NAME,
    Storage.BucketField.TIME_CREATED,
    Storage.BucketField.METAGENERATION,
    Storage.BucketField.ACL,
    Storage.BucketField.DEFAULT_OBJECT_ACL,
    Storage.BucketField.OWNER,
    Storage.BucketField.LABELS,
    Storage.BucketField.LOCATION,
    Storage.BucketField.LOCATION_TYPE,
    Storage.BucketField.WEBSITE,
    Storage.BucketField.VERSIONING,
    Storage.BucketField.CORS,
    Storage.BucketField.STORAGE_CLASS,
    Storage.BucketField.ETAG,
    Storage.BucketField.ENCRYPTION,
    Storage.BucketField.BILLING,
    Storage.BucketField.DEFAULT_EVENT_BASED_HOLD,
    Storage.BucketField.RETENTION_POLICY,
    Storage.BucketField.IAMCONFIGURATION,
    Storage.BucketField.LOGGING,
    Storage.BucketField.UPDATED));

archy-bold avatar Sep 16 '21 08:09 archy-bold