iceberg-rust icon indicating copy to clipboard operation
iceberg-rust copied to clipboard

feat: Don't drop additional statistics

Open Fokko opened this issue 1 month ago • 1 comments

This is a behavioral change.

In Iceberg-Rust we require upper/lower bounds to be part of the schema. But in some cases, this in't the case, most obvious schema evolution.

In PyIceberg we expect these values in some tests:

FAILED tests/integration/test_inspect_table.py::test_inspect_files[2] - AssertionError: Difference in column lower_bounds: {} != {2147483546: b's3://warehouse/default/table_metadata_files/data/00000-0-8d621c18-079b-4217-afd8-559ce216e875.parquet', 2147483545: b'\x00\x00\x00\x00\x00\x00\x00\x00'}
assert {} == {2147483545: ...e875.parquet'}
  Right contains 2 more items:
  {2147483545: b'\x00\x00\x00\x00\x00\x00\x00\x00',
   2147483546: b's3://warehouse/default/table_metadata_files/data/00000-0-8d621c1'
               b'8-079b-4217-afd8-559ce216e875.parquet'}
  Full diff:
    {
  +  ,
  -  2147483545: b'\x00\x00\x00\x00\x00\x00\x00\x00',
  -  2147483546: b's3://warehouse/default/table_metadata_files/data/00000-0-8d621c1'
  -              b'8-079b-4217-afd8-559ce216e875.parquet',
    }
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
==== 1 failed, 238 passed, 32 skipped, 3123 deselected in 61.56s (0:01:01) =====

This is a positional delete where the field-IDs are constant, but never part of a schema (they are reserved).

Which issue does this PR close?

  • Closes #.

What changes are included in this PR?

Are these changes tested?

Fokko avatar Nov 12 '25 18:11 Fokko

manually retriggering ci runs, due to #1838 😞

kevinjqliu avatar Nov 12 '25 19:11 kevinjqliu