paimon

paimon copied to clipboard

[core] FileEntry supports file creation time

1

### Purpose Linked issue: close #xxx ### Tests ### API and Format ### Documentation

yuzelin

[doc] add bladepipe cdc ingestion

2

### Purpose Introduce a new way to ingest data from other sources to Paimon. ### Tests ### API and Format ### Documentation

ChocZoe

### Purpose Reduce the number of io calls for object storage. ### Tests N/A ### API and Format N/A

sundapeng

### Purpose This PR fixes an issue where the compaction logic in `BucketedAppendCompactManager` only filtered the first large file from the queue, leaving subsequent large files in the compaction set...

WenDing-Y

## Purpose from #5955 ### What is the purpose of the change Optimize SyncDatabaseAction performance by removing expensive listTables operations during initialization, improving scalability for databases with many tables. ###...

huyuanfeng2018

[core] Fix scan metric report for extra file-index files

5

### Purpose Reading table files contains scan phase and partition read phase. When we use bloom-filter or other file indexs, we found the scan metrics are the total data files...

Askwang

[spark] Spark 3.3 throws write exceptions during partial-update engine operations

2

### Purpose Linked issue: https://github.com/apache/paimon/issues/5932 fix bug with Spark 3.3 throws write exceptions during partial-update engine operations ``` CREATE TABLE T( f1 int, f2 string, f3 string, f4 string) TBLPROPERTIES...

Aiden-Dong

add a judgment of high level befor records on whether it has been deleted or not

2

### Purpose Linked issue: close #5724 When LookupChangelogMergeFunctionWrapper is used for changelog and config: 'changelog-producer.row-deduplicate' = 'true' , add a judgment of 'highLevel' record on whether it has been deleted...

yunlou11

### Purpose Linked issue: close #5709 fix branch : paimon-1.2-snapshot ### Tests ``` spark.sql( """ |create table my_table ( | k int, | v string |) tblproperties ( | 'primary-key'...

Pandas886

[metrics] Add io-metrics for paimon in Flink

6

### Purpose 1. Monitor the input/output (IO) of Flink during read and write operations. ### Tests ### API and Format ### Documentation

zhuanshenbsj1