datachain icon indicating copy to clipboard operation
datachain copied to clipboard

Need a get_file() method to obtain storage content directly

Open volkfox opened this issue 1 year ago • 0 comments

Description

It is a common pattern to simply read the contents of a storage file by filename.

Currently this requires a lengthy construct like this:

next(DataChain.from_storage('gs://datachain-demo/dogs-and-cats/cat.1009.jpg', type='image').collect("file")).read()

A suggestion is to introduce a get_file() method returning a File object (or an iterator if several). This would simplify the syntax to:

DataChain.from_storage('gs://datachain-demo/dogs-and-cats/cat.1009.jpg', type='image').get_file().read()

The option signal should indicate File object location:

DataChain.from_storage('gs://datachain-demo/dogs-and-cats/cat.1009.jpg', type='image').get_file(signal="myfile")

volkfox avatar Aug 12 '24 21:08 volkfox