seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Improve][seatunnel-engine-common] move serializer to common

Open liugddx opened this issue 2 years ago • 10 comments

Purpose of this pull request

Check list

  • [ ] Code changed are covered with tests, or it does not need tests for reason:
  • [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
  • [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
  • [ ] If you are contributing the connector code, please check that the following files are updated:
    1. Update change log that in connector document. For more details you can refer to connector-v2
    2. Update plugin-mapping.properties and add new connector information in it
    3. Update the pom file of seatunnel-dist

liugddx avatar Dec 28 '22 09:12 liugddx

Other modules need this function? A lot of modules depend on common module, add extra dependency in common module can cause the jar so fat. If only st-engine using the proto serializer I think it put in engine module might be better.

cc @hailin0 @Hisoka-X @ic4y @EricJoy2048 @CalvinKirs @ashulin

Hey guys, what do you think about it?

TyrantLucifer avatar Dec 28 '22 10:12 TyrantLucifer

Other modules need this function? A lot of modules depend on common module, add extra dependency in common module can cause the jar so fat. If only st-engine using the proto serializer I think it put in engine module might be better.

cc @hailin0 @Hisoka-X @ic4y @EricJoy2048 @CalvinKirs @ashulin

Hey guys, what do you think about it?

+1

Hisoka-X avatar Dec 28 '22 10:12 Hisoka-X

Other modules need this function? A lot of modules depend on common module, add extra dependency in common module can cause the jar so fat. If only st-engine using the proto serializer I think it put in engine module might be better.↳

cc @hailin0 @Hisoka-X @ic4y @EricJoy2048 @CalvinKirs @ashulin

Hey guys, what do you think about it?↳

yes, it belongs to the engine module.

CalvinKirs avatar Dec 29 '22 00:12 CalvinKirs

Other modules need this function? A lot of modules depend on common module, add extra dependency in common module can cause the jar so fat. If only st-engine using the proto serializer I think it put in engine module might be better.↳ cc @hailin0 @Hisoka-X @ic4y @EricJoy2048 @CalvinKirs @ashulin Hey guys, what do you think about it?↳

yes, it belongs to the engine module.

Ok, I will migrate to the engine module.

liugddx avatar Dec 29 '22 02:12 liugddx

#3792

liugddx avatar Dec 29 '22 02:12 liugddx

the checkpoint-storage should also be replaced.

Done.

liugddx avatar Jan 03 '23 03:01 liugddx

@CalvinKirs @EricJoy2048 PTAL thanks.

liugddx avatar Jan 04 '23 10:01 liugddx

image should delete this comment

CalvinKirs avatar Jan 05 '23 10:01 CalvinKirs

Btw, A better way is to serialize as a separate module, and we may support other ways in the future, such as protobuf, json and more

CalvinKirs avatar Jan 05 '23 10:01 CalvinKirs

Btw, A better way is to serialize as a separate module, and we may support other ways in the future, such as protobuf, json and more

Done. PTAL thanks.

liugddx avatar Jan 06 '23 02:01 liugddx