Daft icon indicating copy to clipboard operation
Daft copied to clipboard

`read_deltalake` attempts to use S3 credentials for local files

Open apostolos-geyer opened this issue 5 months ago • 2 comments

Describe the bug A clear and concise description of what the bug is.

When attempting to read a local deltalake, daft will log multiple errors and attempt to retrieve S3 credentials, create a client for us-east-1, etc. Not sure if this is a bug or if there is some behaviour or configuration for daft to understand I'm working with local files and not to try to use S3, but I couldn't find anything about this in the docs. The file is still read successfully, but it would be nice to not have to wait for it to fail to get a session token, and attempt to create an S3 client.

To Reproduce Steps to reproduce the behavior:

import daft
daft.read_deltalake('path/to/a/local/file')

output:

failed to load region from IMDS err=failed to load IMDS session token: dispatch failure: timeout: error trying to connect: HTTP connect timeout occurred after 1s: HTTP connect timeout occurred after 1s: timed out (FailedToLoadToken(FailedToLoadToken { source: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Timeout, source: hyper::Error(Connect, HttpTimeoutError { kind: "HTTP connect", duration: 1s }), connection: Unknown } }) }))
failed to load region from IMDS err=failed to load IMDS session token: dispatch failure: timeout: error trying to connect: HTTP connect timeout occurred after 1s: HTTP connect timeout occurred after 1s: timed out (FailedToLoadToken(FailedToLoadToken { source: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Timeout, source: hyper::Error(Connect, HttpTimeoutError { kind: "HTTP connect", duration: 1s }), connection: Unknown } }) }))
S3 Credentials not provided or found when making client for us-east-1! Reverting to Anonymous mode. the credential provider was not enabled

Expected behavior A clear and concise description of what you expected to happen.

The local deltalake should be read without attempting to use S3 or any other network locations and without logging errors.

Screenshots If applicable, add screenshots to help explain your problem.

Screenshot 2024-09-21 at 2 03 36 PM

Desktop (please complete the following information): Screenshot 2024-09-21 at 2 07 41 PM

  • Daft Version: 0.3.2

If you guys are looking for contributors, I'd be happy to try and fix this myself. Never contributed to anything before so not sure if there's any procedures but if I can I'll give it a shot.

apostolos-geyer avatar Sep 21 '24 18:09 apostolos-geyer