Martin Durant

I guess I'll put it somewhere on the my list, but I don't anticipate anything immediate.

@datapythonista , I have implemented this here: https://github.com/martindurant/mnist-data-intake Note that this requires the latest master version of Intake (because the path munging while decompressing has a bug), so the package...

Yes, I considered that, and initially started the work as a PR, but made a separate repo because the main aim of your repo is installable code to be executed...

Hm, I run it many times locally and it always passes, which makes it annoying! Would it be something like the metadata file hasn't made it to the disk yet?

I think it may be that pyarrow is taking the index from global metadata and fastparquet is taking it from the individual files, allowing for some randomisaton. I am not...

Related code suggestion in https://github.com/dask/dask/issues/6074

I have no memory of it, no

[here](https://github.com/fsspec/filesystem_spec/blob/master/fsspec/core.py#L675) ? Yes, it's clearly something that could be optional and would make sense to surface. Seems like `name_function` was indeed meant for this - but doesn't fit the bill....

I don't believe that your asyncio event loop (`_loop`) is ever started. You would want to run that on its own thread, or run the entire application asynchronously with `asyncio.run()`.

Yes, you would think that you can pass any type compatible with that MemoryFile interface, where methods to return sets of bytes from the target were implemented to read on...