filesystem_spec
filesystem_spec copied to clipboard
Shortcut to allow sequential read?
A common access pattern is
with fsspec.open(...) as f:
process(f.read())
For AbstractBufferedFiles, this causes an initial info() call to figure out the length, but then we do the effectively cat_file, which in most backends doesn't need to file size to operate. So we could reduce the number of requests in many cases by deferring the call to info() until we need it (e.g., seek(x, 2)).