filesystem_spec icon indicating copy to clipboard operation
filesystem_spec copied to clipboard

ArrowFSWrapper should not use "/" root_marker for all filesystems

Open b-phi opened this issue 1 year ago • 3 comments

Perhaps a bit too hacky to be merged, but Fixes https://github.com/fsspec/filesystem_spec/issues/1464 , in theory. Basic idea is to check if the provided arrow filesystem supports a / path, then create an ArrowFSWrapper class dynamically based on the result.

b-phi avatar Dec 13 '23 17:12 b-phi

How about https://github.com/fsspec/filesystem_spec/commit/b1298609f437ea57c5a8c1fae18282969c484c7e , does this doe everything, but without type() ?

martindurant avatar Jan 08 '24 16:01 martindurant

How about b129860 , does this doe everything, but without type() ?

Hey @martindurant yes that commit looks great. Dropping the @classmethod decorator is much cleaner, thanks!

b-phi avatar Jan 08 '24 19:01 b-phi

Dropping the @classmethod decorator is much cleaner, thanks!

I don't immediately see any downside, except maybe we should guard against being called as ArrowFSWrapper._strip_protocol by testing the type of self. If it is being used as a classmethod, we'll have to try to analyse the path for what backend it is each time.

martindurant avatar Jan 09 '24 15:01 martindurant