[SPARK-39814] Use AmazonKinesisClientBuilder.withCredentials instead of new AmazonKinesisClient(credentials)

[panbingkun]

What changes were proposed in this pull request?

Use AmazonKinesisClientBuilder.withCredentials instead of new AmazonKinesisClient(credentials) to cleanup following compilation warnings:

[warn] /home/runner/work/spark/spark/connector/kinesis-asl/src/main/scala/org/apache/spark/examples/streaming/KinesisWordCountASL.scala:108:25: [deprecation @ org.apache.spark.examples.streaming.KinesisWordCountASL.main.kinesisClient | origin=com.amazonaws.services.kinesis.AmazonKinesisClient. | version=] constructor AmazonKinesisClient in class AmazonKinesisClient is deprecated

[warn] /home/runner/work/spark/spark/connector/kinesis-asl/src/main/scala/org/apache/spark/examples/streaming/KinesisWordCountASL.scala:224:25: [deprecation @ org.apache.spark.examples.streaming.KinesisWordProducerASL.generate.kinesisClient | origin=com.amazonaws.services.kinesis.AmazonKinesisClient. | version=] constructor AmazonKinesisClient in class AmazonKinesisClient is deprecated

[warn] /home/runner/work/spark/spark/connector/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala:142:24: [deprecation @ org.apache.spark.streaming.kinesis.KinesisSequenceRangeIterator.client | origin=com.amazonaws.services.kinesis.AmazonKinesisClient. | version=] constructor AmazonKinesisClient in class AmazonKinesisClient is deprecated

[warn] /home/runner/work/spark/spark/connector/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisTestUtils.scala:58:18: [deprecation @ org.apache.spark.streaming.kinesis.KinesisTestUtils.kinesisClient.client | origin=com.amazonaws.services.kinesis.AmazonKinesisClient. | version=] constructor AmazonKinesisClient in class AmazonKinesisClient is deprecated

Why are the changes needed?

Cleanup deprecation api usage related to kinesis, include all case.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass GA.

[panbingkun]

https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-kinesis/src/main/java/com/amazonaws/services/kinesis/AmazonKinesisClient.java#L174~L189

[AmplabJenkins]

Can one of the admins verify this patch?

[LuciferYang]

I wonder if we have enough UTs to check the change of kinesis related code

[panbingkun]

Maybe I need an Amazon account, but I really don't have one at present. Who can help me?

[steveloughran]

I just want to highlight here that the move to using the client builder apis in the s3a client (HADOOP-13551) was one of the most traumatic regressions of recent years because all test systems met one or more of the conditions under which things actually worked (in EC2, explicit region set in client config or with AWS CLI installed)

hence: https://issues.apache.org/jira/browse/HADOOP-17771

review the code very carefully here to make sure that there are no new options which are now mandatory except in those same deployment environments where are the S3A code actually worked.

[github-actions[bot]]

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!