paimon
paimon copied to clipboard
apache
Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
### Search before asking - [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar. ### Motivation Currently `Paimon` will create a global commit operator for flink job which makes...
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation https://paimon.apache.org/docs/0.8/program-api/java-api/ comes with a warning at the top > We do not recommend...
### Purpose Linked issue: close #3386 Support async refresh for PrimaryKeyPartialLookupTable. Async refresh files generated by new snapshots which are intervened with expired cached files. The total size of new...
### Purpose Linked issue: close #xxx Currently spark side not support migrate_database procedure which can improve user to migrate their tables, this pr is aimed to support it. ### Tests...
### Purpose This PR adds a more detailed error message to the snapshot expiration process, showing users a possible cause of the FileNotFoundException they may face and the recommended solution...
### Purpose Read manifest to find used files will cost time, if the candidates is empty, we don't need to do that ### Tests ### API and Format ### Documentation
Compatible with Spark4 （upgrade antlr4 to version 4.13.1 Compatible with jdk17 ）
### Purpose This PR provides default parallelism configuration for sink operators, in cases where undefined/adaptive parallelism has not been supported. ### Tests Existing UTs are modified to verify that the...
### Purpose Introduce expire_tags procedure, support expire tags by time. ```sql -- expire tags by tagCreateTime and tagTimeRetained CALL sys.expire_tags(table => 'default.T') -- expire tags that tagCreateTime less than older_than...
### Purpose public API ```java public interface ReadBuilder extends Serializable { ReadBuilder withRowTypeProjection(RowTypeProjection rowTypeProjection); } public class RowTypeProjection { public static RowTypeProjection from(RowType rowType); } ``` inner api ```java public...
