QP Hou

WIP: parquet2 implementation backed by parquet2 feature gate

5

decouple core from arrow # Description WIP parquet2 implementation. The goal of this PR is to implement full read support leveraging parquet2. Write support is out of the scope and...

Support CDC

4

# Description See design doc at: https://docs.google.com/document/d/1FQSl0S0Irckim331YvcAKyxxGBhdLiJcJH6v3fEyiC0/edit#. This will make non-append only table usable as a streaming source!

# Description Allow users of delta-rs to build distributed applications that can recover from long pauses. Applications that prefer to hard crash on pauses can still do so by match...

Check for copy object response content

1

# Environment **Delta-rs version**: all **Binding**: rust *** # Bug Turns out S3 object copy call can fail and return 200 status at the same time: https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html#API_CopyObject_ResponseElements. We need to...

Refactor DeltaTable to generic over storage backend trait

2

# Description **Use Case** Right now DetalTable references storage backend using dynamic dispatch, this could turn into a potential performance bottleneck because our access to storage backend is now tied...

# Description See reference implementation at https://github.com/delta-io/delta/blob/master/core/src/main/scala/org/apache/spark/sql/delta/files/TahoeFileIndex.scala. tasks: - [ ] read file index - [ ] write file index

Support bloom filter table indexes

10

# Description **Use Case** This could help speed up table scans. This feature is not documented in the official spec yet, see more details in https://docs.databricks.com/delta/optimizations/bloom-filters.html.

Prevent invalid remove action during deserialization

6

# Environment **Delta-rs version**: all **Binding**: all *** # Bug **What happened**: Follow up for https://github.com/delta-io/delta-rs/pull/450/files#r723861467. It looks like we could end up with invalid remove action that has both...

Safe guard DeltaTableState using Rust's type system

5

# Description **Use Case** Follow up for https://github.com/delta-io/delta-rs/pull/456#issuecomment-941926489. When we call `process_action` on a particular DeltaTableState, we need to make sure the passed in actions do not cross commit boundaries....

API to return full table history

2

# Description **Use Case** See https://docs.delta.io/0.8.0/delta-utility.html#retrieve-delta-table-history