filesystem_spec icon indicating copy to clipboard operation
filesystem_spec copied to clipboard

Published 20 hours ago • verified publisher icon fsspec

Call cat_ranges in blockcache for async filesystems

Open monken opened this issue 2 years ago • 2 comments

The CachingFileSystem can benefit from calling cat_ranges for asynchronous filesystems to concurrently download blocks. This speeds up filling the cache significantly for filesystems like HTTP and S3FS.

monken avatar Aug 19 '23 20:08 monken

Sorry I let this one slide - I still think it's useful. I have just the one comment above.

martindurant avatar Oct 04 '23 14:10 martindurant

Can you please solve the conflict, and I can have another look?

martindurant avatar Nov 09 '23 19:11 martindurant