hudi

hudi copied to clipboard

[HUDI-9332] Pluggable Table Format Support with native Integration

3

### Change Logs [HUDI-9332] Pluggable Table Format Support with native Integration 1. Includes base interface for pluggable table format 2. Includes native hudi integration for pluggable table format 3. Includes...

bvaradar

Use ScanOperation for Spark version less than or equal to 3.3

1

### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance impact._...

suryaprasanna

[HUDI-9395] Validation schema evolution

1

### Change Logs Test three cases: 1. Different base file schema; 2. Different log file schema; 3. Different base and log file schema. With extra primitive column and extra nested...

linliu-code

[HUDI-8233] Remove unused codepaths after HoodieFileGroupReader migration

1

### Change Logs - Removes unused codepaths ### Impact - Clean up the code ### Risk level (write none, low medium or high below) None ### Documentation Update _Describe any...

the-other-tim-brown

[HUDI-9529] Bloom Index can fail with async compaction

1

### Change Logs - The bloom index uses the `HoodieKeyLookupHandle` which looks up the latest base file for the provided file group ID and partition path. This runs the risk...

the-other-tim-brown

[HUDI-8553] Support writing record positions to log blocks from Spark SQL UPDATE and DELETE statements

1

### Change Logs Spark SQL UPDATE and DELETE do not write record positions to the log files. The PR aims to add the metadata in log files so that it...

lokeshj1703

[HUDI-9504] support in-memory buffer sort in append write

3

### Change Logs Add in memory buffer sort in append write function to improve the parquet compression ratio. From our experiment and testing, It can improve 300% compression ratio with...

HuangZhenQiu

[HUDI-9340] Adding streaming support to secondary index writes w/ metadata table

3

### Change Logs To be filled in. ### Impact _Describe any public API or user-facing feature change or any performance impact._ ### Risk level (write none, low medium or high...

nsivabalan

[SUPPORT]HUDI ParquetDecodingException caused by gzip stream CRC failure

4

> [!NOTE] > I am experiencing an issue related to parquet gzip decoding. Table is in cow format, the issue occurs after hundreds of commits, which is repeatable. compress codec...

ligou525

[HUDI-9512] Avoid creating empty files when there is content to write

2

### Change Logs - Updates code to avoid creating empty files when there is content to write ### Impact - Avoid risk of creating an empty file due to a...

the-other-tim-brown