filesystem_spec
filesystem_spec copied to clipboard
ArrowFSWrapper should not use "/" root_marker for all filesystems
Perhaps a bit too hacky to be merged, but Fixes https://github.com/fsspec/filesystem_spec/issues/1464 , in theory. Basic idea is to check if the provided arrow filesystem supports a /
path, then create an ArrowFSWrapper
class dynamically based on the result.
How about https://github.com/fsspec/filesystem_spec/commit/b1298609f437ea57c5a8c1fae18282969c484c7e , does this doe everything, but without type()
?
How about b129860 , does this doe everything, but without
type()
?
Hey @martindurant yes that commit looks great. Dropping the @classmethod
decorator is much cleaner, thanks!
Dropping the @classmethod decorator is much cleaner, thanks!
I don't immediately see any downside, except maybe we should guard against being called as ArrowFSWrapper._strip_protocol
by testing the type of self
. If it is being used as a classmethod, we'll have to try to analyse the path for what backend it is each time.