paimon
paimon copied to clipboard
apache
Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation SnapshotInfo add totalFileSize & totalDataFiles ### Solution _No response_ ### Anything else? _No...
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version 1.20 ### Compute Engine Flink ### Minimal reproduce step I am having...
### Purpose Linked issue: close #5293 ### Tests ### API and Format ### Documentation
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version 1.1-snapshot https://github.com/apache/paimon/actions/runs/13919826892/job/38950250367 ### Anything else? _No response_ ### Are you willing to...
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation After Spark3.3+, which support field with meatdata, currently paimon schema don't support the...
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation In spark SPJ, it supports to report the partition key as the output...
### Purpose There are several operations during data writing such as `partition-expire`,`snapshot-expire`,`create-tag`... Currently, these actions are either executed together or not executed at all in the framework. The purpose of...
### Purpose Linked issue: close #5214 ### Tests MergeTreeCompactManagerTest#testRefresh() KeyValueFileStoreWriteTest#testRefreshL0FilesEnabled() KeyValueFileStoreWriteTest#testRefreshL0FilesDisabled() ### API and Format ### Documentation
### Purpose Linked issue: close #5215 ### Tests BucketedAppendCompactManagerTest#testToCompactRefresh AppendOnlyFixedBucketFileStoreWriteTest#testRefreshCompactFilesEnabled AppendOnlyFixedBucketFileStoreWriteTest#testRefreshCompactFilesDisabled ### API and Format ### Documentation
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation Currently, in paimon profiles, spark3 is bound to scala2.12 version, spark4 is bound...