seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Improve][Core] Optimize package logic

Open VolodymyrDuke opened this issue 1 year ago • 4 comments

Move jdbc driver jars into plugins

Purpose of this pull request

Check list

  • [ ] Code changed are covered with tests, or it does not need tests for reason:
  • [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
  • [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
  • [ ] If you are contributing the connector code, please check that the following files are updated:
    1. Update change log that in connector document. For more details you can refer to connector-v2
    2. Update plugin-mapping.properties and add new connector information in it
    3. Update the pom file of seatunnel-dist
  • [ ] Update the release-note.

VolodymyrDuke avatar Aug 04 '23 07:08 VolodymyrDuke

@Hisoka-X PTAL

EricJoy2048 avatar Aug 04 '23 08:08 EricJoy2048

Hi, can you explain why want to change this?

Hisoka-X avatar Aug 05 '23 02:08 Hisoka-X

Hi, can you explain why want to change this?

When we pakcage manually, all jdbc driver jars will be put in lib/, every seatunnel job before start it will load all jars in lib, so if any jobs doesn't need these dependencies, it maybe casue jar conflict. This change only infect the package manually. e.g. : mvn clean package -Pci. But we have another concern about it because all ci steps do not be executed. This change whether infect the jdbc connectors we don't a clearly result.

VolodymyrDuke avatar Aug 07 '23 06:08 VolodymyrDuke

so if any jobs doesn't need these dependencies, it maybe casue jar conflict

Seem this PR doesn't fix this error. All driver also would be loaded when submit a job if put it in plugin/lib.

Hisoka-X avatar Aug 22 '23 10:08 Hisoka-X