lance icon indicating copy to clipboard operation
lance copied to clipboard

Add `storage_options` to `LanceCommitter`

Open tonyf opened this issue 1 year ago • 1 comments

Currently, LanceFragmentWriter supports a storage_options kwarg to support specifying s3 credentials/endpoints. However, LanceCommitter does not have the same option and raises TypeError: object.__init__() takes exactly one argument (the instance to initialize) when trying to initialize it with storage_options.

tonyf avatar Jul 18 '24 18:07 tonyf

This sounds good. Seems like it's just a matter of piping storage_options down to lance.LanceDataset.commit here:

https://github.com/lancedb/lance/blob/c59996dc671c80d03bb7b9f11ccf0fcff76f5c5b/python/python/lance/ray/sink.py#L142

wjones127 avatar Jul 18 '24 18:07 wjones127

Can we get a release with this fix? 🙏

BitPhinix avatar Jul 24 '24 00:07 BitPhinix