seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Feature][Connector] Clickhouse-File performance boost

Open Hisoka-X opened this issue 2 years ago • 2 comments

Code of Conduct

Search before asking

  • [X] I had searched in the issues and found no similar issues.

Describe the proposal

In order to improve the code quality and performance of ClickhouseFile Sink, the following work needs to be done. First version is #1501 . Design in #1382

Task list

  • [ ] Support Rsync to transfer clickhouse data file
  • [ ] Support zero copy to boost file transfer performance
  • [x] Change data stream from echo command to file stream #1566
  • [ ] Exactly-Once support

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Hisoka-X avatar Mar 19 '22 04:03 Hisoka-X

@Hisoka-X Can i import the project rsync4j to support Rsync to transfer clickhouse data file?

Emor-nj avatar Jun 21 '22 02:06 Emor-nj

@Hisoka-X Can i import the project rsync4j to support Rsync to transfer clickhouse data file?

Hi, of course. Welcome to contribute this feature!

Hisoka-X avatar Jun 21 '22 02:06 Hisoka-X