[Improvement]: Improve distribution dependencies

[link3280]

Search before asking

• [X] I have searched in the issues and found no similar issues.

What would you like to be improved?

Currently, the distribution lib folder has more than 400 jars as a result of copying all modules' runtime dependencies. It causes redundancy and is hard to manage.

The situation could be improved in the following ways:

1. filter out the unused dependencies (like tephra)
2. fix different versions of the same artifacts (like jersey)

How should we improve?

1. filter out the unused dependencies (like tephra)
2. fix different versions of the same artifacts (like jackson, jersey)

Are you willing to submit PR?

• [X] Yes I am willing to submit a PR!

Subtasks

• [x] https://github.com/apache/amoro/issues/2763
• [x] https://github.com/apache/amoro/issues/2765
• [x] https://github.com/apache/amoro/issues/2752
• [ ] fix duplicate antlr
• [ ] remove spark unused
• [x] https://github.com/apache/amoro/issues/2761
• [ ] remove unnecessary flink dependencies

Code of Conduct

• [X] I agree to follow this project's Code of Conduct