incubator-streampark icon indicating copy to clipboard operation
incubator-streampark copied to clipboard

Added DataSource module

Open wolfboys opened this issue 3 years ago • 3 comments

wolfboys avatar Jan 12 '22 05:01 wolfboys

I am willing contribute this module.

Kyofin avatar Apr 21 '22 05:04 Kyofin

I am willing contribute this module.

great !!! 👍👍👍

wolfboys avatar Apr 21 '22 07:04 wolfboys

@wolfboys

I can finish kafka datasource which inlclude : json auto infer and so on.

  • The feature may include:
  1. (promise) flink kafka catalog .
  2. (promise) json -> flink sql logicalType infer.
  3. (non promise) user privilege

User privilege depends on StreamGraph to analysis which need some trick which will change the sql commit way.(Difference with streamxpark done .)

  • Detail

Catalog

  1. Meta center control physic column define.
  2. Create / hints to support : (Schema# All schema type) . Such as : primary key define, watermark ,timestamp define and logical column.

Json infer

  1. Auto infer json type to Flink sql logicalType.

Whojohn avatar Aug 24 '22 09:08 Whojohn

Expired proposal

wolfboys avatar Dec 07 '23 04:12 wolfboys