paimon
paimon copied to clipboard
apache
Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
### Purpose - The result of expire_partitions is out of order, sort it. - Add max_expires param to limit the expired partitions number at one time. before:  after: ###...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version paimon-flink-action-1.0-20240923.002611-17.jar ### Compute Engine flink1.18 ### Minimal reproduce step when Dedicated Compaction...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version 1.0 ### Compute Engine flink 1.18 ### Minimal reproduce step 1. ```...
### Purpose Currently, changelog files are not compacted. If Flink's checkpoint interval is short (for example, 30 seconds) and the number of buckets is large, each snapshot may produce lots...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation We need sourceIdleTime metric to get the right currentFetchEventTimeLag metric. User also need...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version master(8ac121a) ### Compute Engine Flink ### Minimal reproduce step Check the Flink...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation Gravitino is a unified metadata management system which supports Paimon catalog, we are...
### Purpose Linked issue: close #xxx Fix the loadTable of SparkSource can't recognize correct catalog as below case `dataset.format("paimon").options(...).save();` ### Tests ### API and Format ### Documentation
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version paimon-flink-1.18-1.0-20240923.002611-17.jar ### Compute Engine flink1.18 ### Minimal reproduce step set 'table.exec.sink.upsert-materialize'='NONE'; set...
### Purpose Linked issue: close #xxx ### Tests ### API and Format ### Documentation
