delta-rs icon indicating copy to clipboard operation
delta-rs copied to clipboard

Opening an incorrect table path causes an empty directory to be made

Open rtyler opened this issue 11 months ago • 2 comments

Environment

Delta-rs version: 0.22.3

Binding: python

Environment:

  • Cloud provider: :upside_down_face:
  • OS: Linux/amd64
  • Other:

Bug

What happened:

>>> from deltalake import DeltaTable
>>> dt = DeltaTable('i_just_made_this_up_lol')
[2024-12-16T14:15:26Z DEBUG deltalake_core::table::builder] creating table builder with file:///home/tyler/source/github/delta-io/delta-rs/python/i_just_made_this_up_lol
[2024-12-16T14:15:26Z DEBUG deltalake_core::table::builder] build_storage() with file:///home/tyler/source/github/delta-io/delta-rs/python/i_just_made_this_up_lol
[2024-12-16T14:15:26Z DEBUG deltalake_core::table::builder] Loading a logstore based off the location: Url { scheme: "file", cannot_be_a_base: false, username: "", password: None, host: None, port: None, path: "/home/tyler/source/github/delta-io/delta-rs/python/i_just_made_this_up_lol", query: None, fragment: None }
[2024-12-16T14:15:26Z DEBUG deltalake_core::logstore] Found a storage provider for file:/// (file:///home/tyler/source/github/delta-io/delta-rs/python/i_just_made_this_up_lol)
[2024-12-16T14:15:26Z DEBUG deltalake_core::logstore] Found a logstore provider for file:///
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/tyler/source/github/delta-io/delta-rs/python/deltalake/table.py", line 415, in __init__
    self._table = RawDeltaTable(
_internal.TableNotFoundError: no log files
>>>
(venv) plantain% ls i_just_made_this_up_lol
(venv) plantain%

What you expected to happen:

No directory should be made :smile:

How to reproduce it:

More details:

rtyler avatar Dec 16 '24 14:12 rtyler

Encountered this issue while using the deltalake Python package. After a quick lookup into the Rust implementation of RawDeltaTable, it seems this is a deliberate design in DeltaTableBuilder::from_uri calling ensure_table_uri?

https://github.com/delta-io/delta-rs/blob/72311ed2fc0c459dd3d5c20358b04efe0ad8d001/crates/core/src/table/builder.rs#L384

/// The location could be: /// * A valid URL, which will be parsed and returned /// * A path to a directory, which will be created and then converted to a URL. /// /// If it is a local path, it will be created if it doesn't exist.

Why is there this requirement of creating an empty directory if the URI is local?

From a user perspective, directory creation should only be performed when actually creating a Delta Table (DeltaTable.create), not trying to open an existing one.

rclement avatar Dec 26 '24 07:12 rclement

take

smeyerre avatar May 30 '25 19:05 smeyerre