delta-rs
delta-rs copied to clipboard
Opening an incorrect table path causes an empty directory to be made
Environment
Delta-rs version: 0.22.3
Binding: python
Environment:
- Cloud provider: :upside_down_face:
- OS: Linux/amd64
- Other:
Bug
What happened:
>>> from deltalake import DeltaTable
>>> dt = DeltaTable('i_just_made_this_up_lol')
[2024-12-16T14:15:26Z DEBUG deltalake_core::table::builder] creating table builder with file:///home/tyler/source/github/delta-io/delta-rs/python/i_just_made_this_up_lol
[2024-12-16T14:15:26Z DEBUG deltalake_core::table::builder] build_storage() with file:///home/tyler/source/github/delta-io/delta-rs/python/i_just_made_this_up_lol
[2024-12-16T14:15:26Z DEBUG deltalake_core::table::builder] Loading a logstore based off the location: Url { scheme: "file", cannot_be_a_base: false, username: "", password: None, host: None, port: None, path: "/home/tyler/source/github/delta-io/delta-rs/python/i_just_made_this_up_lol", query: None, fragment: None }
[2024-12-16T14:15:26Z DEBUG deltalake_core::logstore] Found a storage provider for file:/// (file:///home/tyler/source/github/delta-io/delta-rs/python/i_just_made_this_up_lol)
[2024-12-16T14:15:26Z DEBUG deltalake_core::logstore] Found a logstore provider for file:///
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/tyler/source/github/delta-io/delta-rs/python/deltalake/table.py", line 415, in __init__
self._table = RawDeltaTable(
_internal.TableNotFoundError: no log files
>>>
(venv) plantain% ls i_just_made_this_up_lol
(venv) plantain%
What you expected to happen:
No directory should be made :smile:
How to reproduce it:
More details:
Encountered this issue while using the deltalake Python package.
After a quick lookup into the Rust implementation of RawDeltaTable, it seems this is a deliberate design in DeltaTableBuilder::from_uri calling ensure_table_uri?
https://github.com/delta-io/delta-rs/blob/72311ed2fc0c459dd3d5c20358b04efe0ad8d001/crates/core/src/table/builder.rs#L384
/// The location could be: /// * A valid URL, which will be parsed and returned /// * A path to a directory, which will be created and then converted to a URL. /// /// If it is a local path, it will be created if it doesn't exist.
Why is there this requirement of creating an empty directory if the URI is local?
From a user perspective, directory creation should only be performed when actually creating a Delta Table (DeltaTable.create), not trying to open an existing one.
take