seatunnel [Feature][Connector-V2][Iceberg]Modify the scope of flink-shaded-hadoop-2 to provided to be compatible with hadoop3.x

[Feature][Connector-V2][Iceberg]Modify the scope of flink-shaded-hadoop-2 to provided to be compatible with hadoop3.x

Open s7monk opened this issue 2 years ago • 7 comments

@https://github.com/apache/incubator-seatunnel/issues/3044 …ovided

Modify the scope of flink-shaded-hadoop-2 of the Connector-V2-Iceberg module to provided to be compatible with hadoop3.x

[x] Code changed are covered with tests, or it does not need tests for reason:
[x] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
[x] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs

Oct 09 '22 17:10 s7monk

Please add e2e test case in hadoop3.x if you want to support hadoop3.x.
You need update https://github.com/apache/incubator-seatunnel/blob/dev/docs/en/connector-v2/source/Iceberg.md and let users know all of hadoop2.x and hadoop3.x are supported.

Oct 10 '22 02:10 EricJoy2048

Please add e2e test case in hadoop3.x

Oct 10 '22 02:10 s7monk

I have tested spark with no problem

Oct 12 '22 07:10 s7monk

I have tested the hadoop catalog on hadoop2.7.1 and hadoop3.1.3, and also tested the hive catalog on hive3.1.2, and it runs without problems.

Oct 12 '22 07:10 s7monk

If this pr is ok, please @ us to review.

Oct 13 '22 13:10 EricJoy2048

this pr is ok , please review it @CalvinKirs @EricJoy2048 @hailin0

Oct 13 '22 13:10 s7monk

PTAL @CalvinKirs @hailin0

Oct 15 '22 08:10 s7monk

PTAL @EricJoy2048 @hailin0

Nov 26 '22 11:11 s7monk