paimon
paimon copied to clipboard
Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation Get metadata from kafka header, such as database, tablename, and other keys/values flink...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version 0.8 ### Compute Engine spark ### Minimal reproduce step CALL sys.create_tag(table =>...
[Bug] ArrayIndexOutOfBoundsException when insert data into partitioned table by hive engine on mr.
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version Paimon 0.7 Hive 3.1.3 (On MR) ### Compute Engine Hive ### Minimal...
### Purpose 1. reusing the logic of scan table. 2. add scan unaware bucket table logic. related issue: #2670 ### Tests ### API and Format ### Documentation
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version paimon 0.7.0 ### Compute Engine Flink 1.17.2 ### Minimal reproduce step /bin/flink...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation CDC Action requires a large number of parameters, which is difficult to reuse...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation Add Procedure in Spark for compact_database ### Solution _No response_ ### Anything else?...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation Snapshot.dataManifests() will read all the ManifestFileMeta in memory . If there are too...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version 0.7.0 ### Compute Engine flink 1.14 ### Minimal reproduce step @humengyu2012 use...
### Purpose Linked issue: related https://github.com/apache/paimon/issues/2670 ### Tests ### API and Format ### Documentation