AWS SDK 1.x EOL - Migrate AWS SDK for Java from 1.x to 2.x

[yurmix]

Description

Druid uses a deprecated version of the AWS SDK Java, 1.x. AWS is requesting developers to move to the new version, 2.x.

Those are two different packages: 1.x is com.amazonaws and 2.x is software.amazon.awssdk.

This issue isn't a detailed design, rather a placeholder to raise awareness. There will need to be additional work to come up with the required design and code changes.

Motivation

The AWS SDK for Java 1.x is in maintenance mode since July 31, 2024, and will enter End-of-Support on December 31, 2025. Moreover, new AWS SDK features are no longer implemented for 1.x. One example is Kinesis dedicated throughput (enhanced fan-out).

[kachida]

Hi @yurmix @abhishekagarwal87 , I'm interested to work on this issue, can you please assign it to me

[cryptoe]

@kachida you can start working on this issue. Before doing the actual code change I suggest you first

• Figure out all the AWS API's druid is using.
• Make a list of things which are deprecated and the corresponding newer apis in the v2 jdk
• List of improvements in the new SDK version which would be helpful for us.

[abhishekagarwal87]

Feel free to pick it up. Let us know when you need help with review

[kachida]

sure @cryptoe @abhishekagarwal87 , will work on getting details needed before the actual code change

[abhishekagarwal87]

@kachida - are you still planning to work on this?

[yurmix]

I started exploring the use of AWS's migration tool based on OpenRewrite.

[a2l007]

@kachida @yurmix Please let us know if you still have plans to work on this. If not, I can take a stab at it. In addition to moving away from a deprecated version, this would allow us to provide REST catalog signer support for the Iceberg connector 😸