vortex icon indicating copy to clipboard operation
vortex copied to clipboard

Published 20 hours ago • verified publisher icon spiraldb

chore(deps): update arrow-rs to v57 (major)

Open renovate[bot] opened this issue 2 weeks ago • 2 comments

This PR contains the following updates:

Package Type Update Change
arrow-arith workspace.dependencies major 56 -> 57
arrow-array workspace.dependencies major 56 -> 57
arrow-buffer workspace.dependencies major 56 -> 57
arrow-cast workspace.dependencies major 56 -> 57
arrow-data workspace.dependencies major 56 -> 57
arrow-ipc workspace.dependencies major 56 -> 57
arrow-ord workspace.dependencies major 56 -> 57
arrow-schema workspace.dependencies major 56 -> 57
arrow-select workspace.dependencies major 56 -> 57
arrow-string workspace.dependencies major 56 -> 57
parquet workspace.dependencies major 56 -> 57

Release Notes

apache/arrow-rs (arrow-arith)

v57.1.0

Compare Source

Full Changelog

Implemented enhancements:

  • Eliminate bound checks in filter kernels #​8865 [arrow]
  • Respect page index policy option for ParquetObjectReader when it's not skip #​8856 [parquet]
  • Speed up collect_bool and remove unsafe #​8848 [arrow]
  • Error reading parquet FileMetaData with empty lists encoded as element-type=0 #​8826 [parquet]
  • ValueStatistics methods can't be used from generic context in external crate #​8823 [parquet]
  • Custom Pretty-Printing Implementation for Column when Formatting Record Batches #​8821 [arrow]
  • Parquet-concat: supports bloom filter and page index #​8804 [parquet]
  • [Parquet] virtual row number support #​7299
  • [Variant] Enforce shredded-type validation in shred_variant #​8795 [arrow]
  • Simplify decision logic to call FilterBuilder::optimize or not #​8781 [arrow]
  • [Variant] Add variant to arrow for DataType::{Binary, LargeBinary, BinaryView} #​8767 [arrow]
  • Provide algorithm that allows zipping arrays whose values are not prealigned #​8752 [arrow]
  • [Parquet] ParquetMetadataReader decodes too much metadata under point-get scenerio #​8751 [parquet]
  • arrow-json supports encoding binary arrays, but not decoding #​8736 [arrow]
  • Allow FilterPredicate instances to be reused for RecordBatches #​8692 [arrow]
  • ArrowJsonBatch::from_batch is incomplete #​8684 [arrow]
  • parquet-layout: More info about layout including footer size, page index, bloom filter? #​8682 [parquet]
  • Rewrite ParquetRecordBatchStream (async API) in terms of the PushDecoder #​8677 [parquet]
  • [JSON] Add encoding for binary view #​8674 [arrow]
  • Refactor arrow-cast decimal casting to unify the rescale logic used in Parquet variant casts #​8670 [arrow]
  • [Variant] Support Uuid/FixedSizeBinary(16) shredding #​8665
  • [Parquet]There should be an encoding counter to know how many encodings the repo supports in total #​8662 [parquet]
  • Improve parse_data_type for List, ListView, LargeList, LargeListView, FixedSizeList, Union, Map, RunEndCoded. #​8648 [arrow]
  • [Variant] Support variant to arrow primitive support null/time/decimal_* #​8637
  • Return error from RleDecoder::reset rather than panic #​8632 [parquet]
  • Add bitwise ops on BooleanBufferBuilder and MutableBuffer that mutate directly the buffer #​8618 [arrow]
  • [Variant] Add variant_to_arrow Utf-8, LargeUtf8, Utf8View types support #​8567 [arrow]

Fixed bugs:

  • Regression: Parsing List(Int64) results in nullable list in 57.0.0 and a non-nullable list in 57.1.0 #​8883
  • Regression: FixedSlizeList data type parsing fails on 57.1.0 #​8880
  • (dyn ArrayFormatterFactory + 'static) can't be safely shared between threads #​8875
  • RowNumber reader has wrong row group ordering #​8864 [parquet]
  • ThriftMetadataWriter::write_column_indexes cannot handle a ColumnIndexMetaData::NONE #​8815 [parquet]
  • "Archery test With other arrows" Integration test failing on main: #​8813 [arrow]
  • [Parquet] Writing in 57.0.0 seems 10% slower than 56.0.0 #​8783 [parquet]
  • Parquet reader cannot handle files with unknown logical types #​8776 [parquet]
  • zip now treats nulls as false in provided mask regardless of the underlying bit value #​8721 [arrow]
  • [avro] Incorrect version in crate.io landing page #​8691 [arrow]
  • Array: ViewType gc() has bug when array sum length exceed i32::MAX #​8681 [arrow]
  • Parquet 56: encounter error: item_reader def levels are None when reading nested field with row filter #​8657 [parquet]
  • Degnerate and non-nullable FixedSizeListArrays are not handled #​8623 [arrow]
  • [Parquet]Performance Degradation with RowFilter on Unsorted Columns due to Fragmented ReadPlan #​8565 [parquet]

Documentation updates:

Performance improvements:

Closed issues:

  • Variant to NullType conversion ignores strict casting #​8810
  • Unify display representation for Field #​8784
  • Misleading configuration name: skip_arrow_metadata #​8780
  • Inconsistent display for types with Metadata #​8761 [arrow]
  • Internal arrow-integration-test crate is linked from arrow docs #​8739 [arrow]
  • Add benchmark for RunEndEncoded casting #​8709 [arrow]
  • [Varaint] Support VariantArray::value to return a Result<Variant> #​8672

Merged pull requests:

* This Changelog was automatically generated by github_changelog_generator

v57.0.0

Compare Source

Full Changelog

Implemented enhancements:

  • Eliminate bound checks in filter kernels #​8865 [arrow]
  • Respect page index policy option for ParquetObjectReader when it's not skip #​8856 [parquet]
  • Speed up collect_bool and remove unsafe #​8848 [arrow]
  • Error reading parquet FileMetaData with empty lists encoded as element-type=0 #​8826 [parquet]
  • ValueStatistics methods can't be used from generic context in external crate #​8823 [parquet]
  • Custom Pretty-Printing Implementation for Column when Formatting Record Batches #​8821 [arrow]
  • Parquet-concat: supports bloom filter and page index #​8804 [parquet]
  • [Parquet] virtual row number support #​7299
  • [Variant] Enforce shredded-type validation in shred_variant #​8795 [arrow]
  • Simplify decision logic to call FilterBuilder::optimize or not #​8781 [arrow]
  • [Variant] Add variant to arrow for DataType::{Binary, LargeBinary, BinaryView} #​8767 [arrow]
  • Provide algorithm that allows zipping arrays whose values are not prealigned #​8752 [arrow]
  • [Parquet] ParquetMetadataReader decodes too much metadata under point-get scenerio #​8751 [parquet]
  • arrow-json supports encoding binary arrays, but not decoding #​8736 [arrow]
  • Allow FilterPredicate instances to be reused for RecordBatches #​8692 [arrow]
  • ArrowJsonBatch::from_batch is incomplete #​8684 [arrow]
  • parquet-layout: More info about layout including footer size, page index, bloom filter? #​8682 [parquet]
  • Rewrite ParquetRecordBatchStream (async API) in terms of the PushDecoder #​8677 [parquet]
  • [JSON] Add encoding for binary view #​8674 [arrow]
  • Refactor arrow-cast decimal casting to unify the rescale logic used in Parquet variant casts #​8670 [arrow]
  • [Variant] Support Uuid/FixedSizeBinary(16) shredding #​8665
  • [Parquet]There should be an encoding counter to know how many encodings the repo supports in total #​8662 [parquet]
  • Improve parse_data_type for List, ListView, LargeList, LargeListView, FixedSizeList, Union, Map, RunEndCoded. #​8648 [arrow]
  • [Variant] Support variant to arrow primitive support null/time/decimal_* #​8637
  • Return error from RleDecoder::reset rather than panic #​8632 [parquet]
  • Add bitwise ops on BooleanBufferBuilder and MutableBuffer that mutate directly the buffer #​8618 [arrow]
  • [Variant] Add variant_to_arrow Utf-8, LargeUtf8, Utf8View types support #​8567 [arrow]

Fixed bugs:

  • Regression: Parsing List(Int64) results in nullable list in 57.0.0 and a non-nullable list in 57.1.0 #​8883
  • Regression: FixedSlizeList data type parsing fails on 57.1.0 #​8880
  • (dyn ArrayFormatterFactory + 'static) can't be safely shared between threads #​8875
  • RowNumber reader has wrong row group ordering #​8864 [parquet]
  • ThriftMetadataWriter::write_column_indexes cannot handle a ColumnIndexMetaData::NONE #​8815 [parquet]
  • "Archery test With other arrows" Integration test failing on main: #​8813 [arrow]
  • [Parquet] Writing in 57.0.0 seems 10% slower than 56.0.0 #​8783 [parquet]
  • Parquet reader cannot handle files with unknown logical types #​8776 [parquet]
  • zip now treats nulls as false in provided mask regardless of the underlying bit value #​8721 [arrow]
  • [avro] Incorrect version in crate.io landing page #​8691 [arrow]
  • Array: ViewType gc() has bug when array sum length exceed i32::MAX #​8681 [arrow]
  • Parquet 56: encounter error: item_reader def levels are None when reading nested field with row filter #​8657 [parquet]
  • Degnerate and non-nullable FixedSizeListArrays are not handled #​8623 [arrow]
  • [Parquet]Performance Degradation with RowFilter on Unsorted Columns due to Fragmented ReadPlan #​8565 [parquet]

Documentation updates:

Performance improvements:

Closed issues:

  • Variant to NullType conversion ignores strict casting #​8810
  • Unify display representation for Field #​8784
  • Misleading configuration name: skip_arrow_metadata #​8780
  • Inconsistent display for types with Metadata #​8761 [arrow]
  • Internal arrow-integration-test crate is linked from arrow docs #​8739 [arrow]
  • Add benchmark for RunEndEncoded casting #​8709 [arrow]
  • [Varaint] Support VariantArray::value to return a Result<Variant> #​8672

Merged pull requests:

Configuration

📅 Schedule: Branch creation - Between 12:00 AM and 03:59 AM, only on Monday ( * 0-3 * * 1 ) (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about these updates again.

  • [ ] If you want to rebase/retry this PR, check this box

This PR was generated by Mend Renovate. View the repository job log.

renovate[bot] avatar Nov 24 '25 01:11 renovate[bot]

⚠️ Artifact update problem

Renovate failed to update an artifact related to this branch. You probably do not want to merge this PR as-is.

♻ Renovate will retry this branch, including artifacts, only when one of the following happens:

  • any of the package files in this branch needs updating, or
  • the branch becomes conflicted, or
  • you click the rebase/retry checkbox if found above, or
  • you rename this PR's title to start with "rebase!" to trigger it manually

The artifact failure details are included below:

File name: Cargo.lock
Command failed: cargo update --config net.git-fetch-with-cli=true --manifest-path Cargo.toml --package [email protected] --precise 57.1.0
    Updating crates.io index
error: failed to select a version for the requirement `arrow-arith = "^56.1"`
candidate versions found which didn't match: 57.1.0
location searched: crates.io index
required by package `lance v0.39.0`
    ... which satisfies dependency `lance = "^0.39.0"` of package `bench-vortex v0.1.0 (/tmp/renovate/repos/github/vortex-data/vortex/bench-vortex)`

renovate[bot] avatar Nov 24 '25 01:11 renovate[bot]

CodSpeed Performance Report

Merging #5482 will degrade performances by 51.14%

Comparing renovate/major-57-arrow-rs (ab3e18f) with develop (5458ab2)

Summary

⚡ 4 improvements
❌ 111 regressions
✅ 1390 untouched
🆕 7 new
⏩ 157 skipped[^skipped]

:warning: Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
❌ new_alp_prim_test_between[f32, 32768] 376.2 µs 503.1 µs -25.21%
❌ old_alp_prim_test_between[f32, 16384] 192 µs 269.5 µs -28.78%
❌ old_alp_prim_test_between[f32, 32768] 366.6 µs 498 µs -26.39%
❌ old_alp_prim_test_between[f64, 16384] 232 µs 326.2 µs -28.88%
❌ old_alp_prim_test_between[f64, 32768] 356.7 µs 546.5 µs -34.73%
❌ old_bp_prim_test_between[i16, 16384] 149.1 µs 240.6 µs -38.02%
❌ old_bp_prim_test_between[i16, 2048] 96.6 µs 108.7 µs -11.06%
❌ old_bp_prim_test_between[i16, 32768] 207.6 µs 391.1 µs -46.91%
❌ old_bp_prim_test_between[i32, 16384] 168.1 µs 245.3 µs -31.46%
❌ old_bp_prim_test_between[i32, 2048] 96.9 µs 109.1 µs -11.17%
❌ old_bp_prim_test_between[i32, 32768] 258.3 µs 398.7 µs -35.21%
❌ old_bp_prim_test_between[i64, 16384] 206.7 µs 302.9 µs -31.74%
❌ old_bp_prim_test_between[i64, 2048] 105.8 µs 117.7 µs -10.08%
❌ old_bp_prim_test_between[i64, 32768] 320.2 µs 511.7 µs -37.42%
❌ old_raw_prim_test_between[f32, 16384] 149.1 µs 216.6 µs -31.17%
❌ old_raw_prim_test_between[f32, 32768] 219 µs 352.3 µs -37.83%
❌ old_raw_prim_test_between[f64, 16384] 178.4 µs 252.6 µs -29.35%
❌ old_raw_prim_test_between[f64, 2048] 88.7 µs 98.9 µs -10.3%
❌ old_raw_prim_test_between[f64, 32768] 277.3 µs 424 µs -34.61%
❌ old_raw_prim_test_between[i32, 16384] 139.5 µs 217.5 µs -35.88%
... ... ... ... ...

:information_source: Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks. [^skipped]: 157 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

codspeed-hq[bot] avatar Nov 24 '25 01:11 codspeed-hq[bot]