Potential data corruption issue for OCI HDFS Connector with RefreshableOnNotAuthenticatedProvider

[mricken]

Update 2020-11-03: We believe we have fixed this issue in version 3.2.1.3. If you are using a RefreshableOnNotAuthenticatedProvider (Instance Principals or Resource Principals, for example), we urge you to update to version 3.2.1.3 or later.

Original issue If you are using any of the OCI HDFS Connector and you use a RefreshableOnNotAuthenticatedProvider (e.g. InstancePrincipalsCustomAuthenticator, or generally for Resource Principals or Instance Principals) you may be affected by silent data corruption.

Description

When using a RefreshableOnNotAuthenticatedProvider, the OCI Java SDK will attempt to acquire a new security token when receiving a 401 response, which indicates authentication problems, and then automatically retry the request. In this situation, the clients fail to reset the stream position for requests that upload streams. As a result, an empty stream is successfully uploaded, and success may be returned, even though the actual stream content was not uploaded.

Affected and Not Affected Code

It only happens when an authentication details provider that implements RefreshableOnNotAuthenticatedProvider is used, e.g.:

• Affected: InstancePrincipalsCustomAuthenticator
• Affected: InstancePrincipalsAuthenticationDetailsProvider
• Affected: ResourcePrincipalAuthenticationDetailsProvider

It does not happen for other authentication details provider that do not implement said interface, e.g.:

• Not affected: SimpleAuthenticationDetailsProvider
• Not affected: ConfigFileAuthenticationDetailsProvider

Workarounds

We believe we have fixed this issue in OCI Java SDK version 1.25.2. If you are using a RefreshableOnNotAuthenticatedProvider (InstancePrincipalsCustomAuthenticator in particular, or generally Instance Principals or Resource Principals, for example) with the OCI HDFS Connector, we urge you to update to version 1.25.2 of the OCI Java SDK by overriding the version of the OCI Java SDK that the OCI HDFS Connector uses.

We apologize for any inconvenience this issue may cause you. We will release an updated version of the OCI HDFS Connector, which uses the fixed version 1.25.2 of the OCI Java SDK, shortly. Please watch this GitHub issue for updates.