paimon
paimon copied to clipboard
apache
Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
### Purpose Linked issue: close #4140 ### Tests no ### API and Format no ### Documentation no
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version 0.9.0 ### Compute Engine Flink 1.18.1 ### Minimal reproduce step You can...
### Purpose @umeshdangat has an outstanding PR which was updated after PR by @zhuangchong was merged. That PR: https://github.com/apache/paimon/pull/3323 The above patch allows consuming data from avro data from kafka...
### Purpose partial-update merge engine support batch deletion using flink sql ### Tests paimon-flink/paimon-flink-common/src/test/java/org/apache/paimon/flink/PartialUpdateITCase#testRemoveRecordOnDelete ### API and Format ### Documentation
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation Branch supports expiration time ### Solution _No response_ ### Anything else? _No response_...
### Purpose Linked issue: close #xxx add more test cases for `ManifestFileMerger#tryFullCompaction` ### Tests org.apache.paimon.manifest.ManifestFileMetaTest ### API and Format ### Documentation
### Purpose If the aggregate all group by keys are from partition columns, we can support push down it. It makes user to get every partition (or with partition filter)...
### Purpose to #4209, Support nested col pruning, e.g. ```sql CREATE TABLE students ( name STRING, age INT, course STRUCT ) USING paimon; ``` ```sql SELECT course.grade FROM students; ```...
Open orc switch useSelected,allowSARGToFilter to make sure pushdown works， To reduce query response time. ### Purpose Linked issue: close #xxx ### Tests ### API and Format ### Documentation
### Purpose Scenario：user query statistic table by snapshot-id and analyzed table multiple times. Such as user can do the analyzed at regular time and collect the statistic metric to monitor....
