paimon
paimon copied to clipboard
apache
Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation When creating an Append for Message Queue table, as depicted in the screenshot...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation HIVE has now released 4.0.0. It solves many of the problems that HIVE...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation subtask of (https://github.com/apache/paimon/issues/2861) ### Solution _No response_ ### Anything else? _No response_ ###...
### Purpose Introduce new parameters to control whether the committer operator and writer operator are chained together Linked issue: close #3232 ### Tests ### API and Format ### Documentation
…e FileFormat. ### Purpose update FileFormatType from Enum to String, to support userdefine fileformat， besides [orc, avro, parquet]. Linked issue: close #xxx ### Tests ### API and Format ### Documentation
### Purpose FLINK-32001 has been resolved, so partial columns can now be used for updates. Different merge engines have different requirements(aggregation has also been supported for updates): 1. deduplicate: only...
### Purpose Linked issue: close #3295 This PR aims to add async refresh of lookup table to avoid lag in job when the lookup operator is blocked by refresh data...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation Currently, the lookup operator checks the time when getting data from lookup table,...
### Purpose In the previous behavior, no bucket-key would take the whole line and make it bucket-key, which is a poor performance behavior and we should disable it. ### Tests...
### Purpose Restarting the flink-paimon-mysql-cdc job schemaCompatible failed when the data type length was modified between the stop and restart of the flink-paimon-mysql-cdc job ### Tests ### API and Format...
