paimon icon indicating copy to clipboard operation
paimon copied to clipboard

[Feature] Improve the parallelism of Flink database compression commit operators

Open zhourui999 opened this issue 1 year ago • 0 comments

Search before asking

  • [X] I searched in the issues and found nothing similar.

Motivation

The parallelism of the commit operator in the current Flink database compression is 1, which is relatively inefficient image image

Solution

I think we can improve the parallelism of the commit operator by shuffling by table

Anything else?

No response

Are you willing to submit a PR?

  • [ ] I'm willing to submit a PR!

zhourui999 avatar Apr 18 '24 06:04 zhourui999