duckdb-deltatable-extension icon indicating copy to clipboard operation
duckdb-deltatable-extension copied to clipboard

Update Rust crate deltalake to 0.17.3

Open renovate[bot] opened this issue 1 year ago • 0 comments
trafficstars

Mend Renovate

This PR contains the following updates:

Package Type Update Change
deltalake dependencies patch 0.17.1 -> 0.17.3

Release Notes

delta-io/delta.rs (deltalake)

v0.17.3

Full Changelog

Implemented enhancements:

  • Limit concurrent ObjectStore access to avoid resource limitations in constrained environments #​2457
  • How to get a DataFrame in Rust? #​2404
  • Allow checkpoint creation when partion column is "timestampNtz " #​2381
  • is there a way to make writing timestamp_ntz optional #​2339
  • Update arrow dependency #​2328
  • Release GIL in deltalake.write_deltalake #​2234
  • Unable to retrieve custom metadata from tables in rust #​2153
  • Refactor commit interface to be a Builder #​2131

Fixed bugs:

  • Handle rate limiting during write contention #​2451
  • regression : delta.logRetentionDuration don't seems to be respected #​2447
  • Issue writing to mounted storage in AKS using delta-rs library #​2445
  • TableMerger - when_matched_delete() fails when Column names contain special characters #​2438
  • Generic DeltaTable error: External error: Arrow error: Invalid argument error: arguments need to have the same data type - while merge data in to delta table #​2423
  • Merge on predicate throw error on date colum: Unable to convert expression to string #​2420
  • Writing Tables with Append mode errors if the schema metadata is different #​2419
  • Logstore issues on AWS Lambda #​2410
  • Datafusion timestamp type doesn't respect delta lake schema #​2408
  • Compacting produces smaller row groups than expected #​2386
  • ValueError: Partition value cannot be parsed from string. #​2380
  • Very slow s3 connection after 0.16.1 #​2377
  • Merge update+insert truncates a delta table if the table is big enough #​2362
  • Do not add readerFeatures or writerFeatures keys under checkpoint files if minReaderVersion or minWriterVersion do not satisfy the requirements #​2360
  • Create empty table failed on rust engine #​2354
  • Getting error message when running in lambda: message: "Too many open files" #​2353
  • Temporary files filling up _delta_log folder - increasing table load time #​2351
  • compact fails with merged schemas #​2347
  • Cannot merge into table partitioned by date type column on 0.16.3 #​2344
  • Merge breaks using logical datatype decimal128 #​2343
  • Decimal types are not checked against max precision/scale at table creation #​2331
  • Merge update+insert truncates a delta table #​2320
  • Extract add.stats_parsed with wrong type #​2312
  • Process fails without error message when executing merge #​2310
  • delta_rs don't seems to respect the row group size #​2309
  • Auth error when running inside VS Code #​2306
  • Unable to read deltatables with binary columns: Binary is not supported by JSON #​2302
  • Schema evolution not coercing with Large arrow types #​2298
  • Panic in deltalake_core::kernel::snapshot::log_segment::list_log_files_with_checkpoint::{{closure}} #​2290
  • Checkpoint does not preserve reader and writer features for the table protocol. #​2288
  • Z-Order with larger dataset resulting in memory error #​2284
  • Successful writes return error when using concurrent writers #​2279
  • Rust writer should raise when decimal types are incompatible (currently writers and puts table in invalid state) #​2275
  • Generic DeltaTable error: Version mismatch with new schema merge functionality in AWS S3 #​2262
  • DeltaTable is not resilient to corrupted checkpoint state #​2258
  • Inconsistent units of time #​2256
  • Partition column comparison is an assertion rather than if block with raise exception #​2242
  • Unable to merge column names starting from numbers #​2230
  • Merging to a table with multiple distinct partitions in parallel fails #​2227
  • cleanup_metadata not respecting custom logRetentionDuration #​2180
  • Merge predicate fails with a field with a space #​2167
  • When_matched_update causes records to be lost with explicit predicate #​2158
  • Merge execution time grows exponetially with the number of column #​2107
  • _internal.DeltaError when merging #​2084

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.


  • [ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.

renovate[bot] avatar May 01 '24 20:05 renovate[bot]