Nick
> I just wanted to know if there was a possibility of performing it in a single transaction. Strictly speaking, no, but it's an existing request if you'd like to...
There are many approaches... The least intrusive would be to roll your own custom transaction using `OptimisticTransaction` (take a look at the existing command implementations). This would require **no change**...
Hi @dohongdayi - no, Delta 2.1 does not yet natively support MOR. The good news is that one of MOR's dependencies, deletion vectors, is on the H2 roadmap here: https://github.com/delta-io/delta/issues/1307...
Thanks for creating this issue @geoHeil! Copying over a suggested approach from the Slack thread: - No out-of-the-box way to do this in a single atomic command today - You...
@amirmor1 awesome! Here are a few examples: [Trino/Presto Connector](https://docs.google.com/document/d/16S7xoAmXpSax7W1OWYYHo5nZ71t5NvrQ-F79pZF6yb8), [Delta Standalone Writer](https://docs.google.com/document/d/1MJhmW_H7doGWY2oty-I78vciziPzBy_nzuuB-Wv5XQ8), [Apache Flink Sink](https://docs.google.com/document/d/19CU4eJuBXOwW7FC58uSqyCbcLTsgvQ5P1zoPOPgUSpI), [S3 Multi-cluster writes](https://docs.google.com/document/d/1Gs4ZsTH19lMxth4BSdwlWjUNR-XhKHicDvBjd2RqNd8/edit#heading=h.mjjuxw9mcz9h)
Hi @pedrosmv - thank you for reporting this. Confirming we can reproduce this based on your steps. Since an error is still being thrown and no data corruption occurs, we'll...
Hi @dwsmith1983 - following up on this. It would be helpful to make this discussion explicit. Can you share a small example of today's behavior (data, z-order command, resulting file...
Is it possible to generate a candidate set that has the NULL distribution you want to address? The example below will be 50% NULLs in 1 file before the zorder...
I tried to get this working but was unable to do so, maybe this will help others: (using x86_64 linux) 1. Setup layer directory (https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html) ``` mkdir python pip3 install...
The pyarrow build we need just got merged into aws-sdk-pandas 🥳 🥳 https://github.com/aws/aws-sdk-pandas/pull/1977 Last step now is to create the builds for our layers in this repo. A few questions:...