seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Improve][Connector-V2] Improve the paimon source

Open dailai opened this issue 1 year ago • 3 comments

Purpose of this pull request

  1. Support filter read for paimon Source. Link #6343, We can also use the sql of this feature to realize the column projection feature later.
  2. Support factory for paimon source. Link #6695.
  3. Support hadoop ha for paimon source
  4. Support hivecatalog for paimon source
  5. Support read timestamp(n) for paimon source

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

  • [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
  • [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
  • [ ] If you are contributing the connector code, please check that the following files are updated:
    1. Update change log that in connector document. For more details you can refer to connector-v2
    2. Update plugin-mapping.properties and add new connector information in it
    3. Update the pom file of seatunnel-dist
  • [ ] Update the release-note.

dailai avatar May 23 '24 01:05 dailai

Read from hdfs with hadoop ha:

截屏2024-05-22 13 59 36

dailai avatar May 23 '24 01:05 dailai

Modify the paimon document to refer to the mysql.md format

zhilinli123 avatar May 23 '24 06:05 zhilinli123

Modify the paimon document to refer to the mysql.md format

Ok

dailai avatar May 23 '24 08:05 dailai

link #6343

Carl-Zhou-CN avatar Jun 06 '24 03:06 Carl-Zhou-CN

Could you take a look? @zhilinli123

Hisoka-X avatar Jun 07 '24 12:06 Hisoka-X