iceberg-rust
iceberg-rust copied to clipboard
feat: Don't drop additional statistics
This is a behavioral change.
In Iceberg-Rust we require upper/lower bounds to be part of the schema. But in some cases, this in't the case, most obvious schema evolution.
In PyIceberg we expect these values in some tests:
FAILED tests/integration/test_inspect_table.py::test_inspect_files[2] - AssertionError: Difference in column lower_bounds: {} != {2147483546: b's3://warehouse/default/table_metadata_files/data/00000-0-8d621c18-079b-4217-afd8-559ce216e875.parquet', 2147483545: b'\x00\x00\x00\x00\x00\x00\x00\x00'}
assert {} == {2147483545: ...e875.parquet'}
Right contains 2 more items:
{2147483545: b'\x00\x00\x00\x00\x00\x00\x00\x00',
2147483546: b's3://warehouse/default/table_metadata_files/data/00000-0-8d621c1'
b'8-079b-4217-afd8-559ce216e875.parquet'}
Full diff:
{
+ ,
- 2147483545: b'\x00\x00\x00\x00\x00\x00\x00\x00',
- 2147483546: b's3://warehouse/default/table_metadata_files/data/00000-0-8d621c1'
- b'8-079b-4217-afd8-559ce216e875.parquet',
}
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
==== 1 failed, 238 passed, 32 skipped, 3123 deselected in 61.56s (0:01:01) =====
This is a positional delete where the field-IDs are constant, but never part of a schema (they are reserved).
Which issue does this PR close?
- Closes #.
What changes are included in this PR?
Are these changes tested?
manually retriggering ci runs, due to #1838 😞