filesystem_spec icon indicating copy to clipboard operation
filesystem_spec copied to clipboard

Shortcut to allow sequential read?

Open martindurant opened this issue 2 years ago • 0 comments

A common access pattern is

with fsspec.open(...) as f:
    process(f.read())

For AbstractBufferedFiles, this causes an initial info() call to figure out the length, but then we do the effectively cat_file, which in most backends doesn't need to file size to operate. So we could reduce the number of requests in many cases by deferring the call to info() until we need it (e.g., seek(x, 2)).

martindurant avatar Oct 18 '23 19:10 martindurant