Delta Table already exists with `write_deltalake(..., mode='overwrite', overwrite_schema=True)`

[rspears74]

Environment

Delta-rs version: 0.15.1

Binding: Python

Environment:

• Cloud provider: AWS
• OS: MacOS
• Other: Github Action

---

Bug

What happened: Calling write_deltalake(path, data, mode='overwrite', overwrite_schema=True, storage_options=...) fails with Delta Table already exists at that location.

What you expected to happen: The existing Delta table to be overwritten.

How to reproduce it: Call the above function to write to a path with an existing Delta table.

More details:

[rspears74]

Changing the path seems to have fixed this. But it seems the writer cannot handle a non-existent AWS S3 prefix. So if you have the following bucket structure: Bucket: bucket

• path1
• path2 And you try to write to s3://bucket/path1/new-table, it works. But if you try to write to s3://bucket/path1/new-location/new-table, it doesn't.