Adrian Ehrsam

> I am keeping this open, until it's also part of the operations API writer because that's the one that's also used by the Python bindings I'm not too deep...

> @aersam the schema evolution logic got added into the RecordBatchWriter, which sits here: crates/core/src/writer/record_batch.rs > > However we want this logic to be added as well here: crates/core/src/operations/write.rs >...

PR #2289 would fix the error, I hope I can finish it soon

Oh, there are timeout issues? 🙂 which ones are you talking about?

Ok, would be interesting if those are even resolved by the PR as writing starts much earlier and way less RAM is required

Well I would also recommend implementing it in Rust: https://arrow.apache.org/rust/arrow/record_batch/trait.RecordBatchReader.html But the thing is that a RecordBatchReader can be constructed from anything, from either Rust or PyArrow. It's a very...

Don't know if this helps, just tried to read a deletion vector file, and this seems to be working with the roaring crate: ```rust fn get_deletion_vectors( filename: &str, ) ->...

Would you accept a PR that does add the required metadata as a first step?

Well if it's about weeks I can wait. I know that actually column mapping would be first, just thought that cannot be that hard ;) I did not know about...

Trying to get the metadata running here: https://github.com/bmsuisse/delta-rs/tree/deletion_vector_meta Once you have the metadata you could use them for example together with duckdb's `read_parquet([parquets...],file_row_number=True)` to read tables with deletion vectors