Error when processing file mode string

[lobis]

In the memory filesystem there is an error if you try to open it with r+b mode saying this mode is not supported.

However this mode is apparently supported (https://github.com/fsspec/filesystem_spec/blob/master/fsspec/implementations/memory.py#L178).

What I think is happening is that the mode=r+b(or mode=rb+) argument is being processed and transformed internally to r+b and then the comparison to rb+ fails.

(I assume that it should work for any order for the characters in the mode string: rb+ or r+b).

Reproducer:

import fsspec
uri = "memory://path/file"
open_file = fsspec.open(uri, mode="rb+")  # r+b doesn't work either
assert open_file.mode == "rb+"
file = open_file.open() # raises ValueError

[martindurant]

I honestly don't know if "r+b" and "rb+" should be considered equivalent or not, and which of these is canonical. In fact, any mode that is not "w" will have the same effect. In general, other filesystems only tend to suppose "r", "w" and sometimes "a". The special case of local files support the full range of options.

[lobis]

I honestly don't know if "r+b" and "rb+" should be considered equivalent or not, and which of these is canonical. In fact, any mode that is not "w" will have the same effect. In general, other filesystems only tend to suppose "r", "w" and sometimes "a". The special case of local files support the full range of options.

Not sure if I get your point. What I am saying here is that neither r+b or rb+ is working because of some internal processing of the string into r+b then trying to compare it with rb+. From my experience I think r+b and rb+ should both work in the same way but there may be some subtly difference I am not aware of. Anyway I think rb+ mode at the very least should work right?

[martindurant]

Anyway I think rb+ mode at the very least should work right?

If you like! In fact, this is what you get with "r" too :)

In [1]: import fsspec
In [2]: fs = fsspec.filesystem("memory")
In [3]: f = fs.open("myfile", "wb")
In [4]: f.write(b"hello")
In [5]: f = fs.open("myfile", "rb")
In [6]: f.write(b"oi")
In [7]: f.seek(0)
In [8]: f.read(5)
Out[8]: b'oillo'