[Feature] Append table support distribution mode before write to reduce small files

[eric666666]

Search before asking

• [X] I searched in the issues and found nothing similar.

Motivation

Currently Paimon's append table is written directly by Flink writer operator subtask which will cause too many small files. We can set some shuffle conditions before writing to reduce the generation of small files, especially stream processing.Such as key by partition.

Solution

No response

Anything else?

No response

Are you willing to submit a PR?

• [X] I'm willing to submit a PR!