filesystem_spec icon indicating copy to clipboard operation
filesystem_spec copied to clipboard

Open the http file system in the wb mode

Open pingsutw opened this issue 2 years ago • 1 comments

We can only use rb mode in the HTTP file system for now. It will be great if we can support wb mode.

https://github.com/fsspec/filesystem_spec/blob/61cdf52cd0aaf524945ffb83502be39fccaee363/fsspec/implementations/http.py#L350-L351

pingsutw avatar Aug 03 '23 05:08 pingsutw

Since we do have a pipe using POST or PUT methods, this is perfectly possible. Because the operation would not be multi-part, the whole of the file would need to be buffered and the upload only happens when the file is closed (.flush(final=True)).

martindurant avatar Aug 05 '23 18:08 martindurant