FilePathsBase.jl icon indicating copy to clipboard operation
FilePathsBase.jl copied to clipboard

RFC: FilePaths2.jl

Open ExpandingMan opened this issue 2 years ago • 1 comments

Hello all. I have, over the past couple of months, been extremely determined to completely solve what I consider to be an unfortunate state of affairs with S3Path from AWSS3.jl. You've probably all seen my speculative PR's having to do with adding traits to accommodate this. I have recently largely rewritten AbstractTrees.jl partially in service of this goal.

I've ultimately given the situation a lot more thought and have come up with FilePaths2.jl.

The package is currently deliberately incomplete as I would like to solicit opinions. FilePathsBase.jl could be overhauled based on my approach in FilePaths2.jl. I'd be glad to do that, but obviously I do not know what the maintainers would think of that. My current repo of FilePaths2.jl is meant as an example of what I intend to do, please see the README for a more careful exposition of what I did and why.

Please let me know if there is any interest in moving FilePathsBase.jl that way. There are strong technical arguments why what I did there is much better for S3Path, but FilePathsBase works perfectly fine for file system paths so you might decide you just don't want any part of it.

ExpandingMan avatar Jun 22 '22 21:06 ExpandingMan

I'll try to reviewing it more thoroughly later in the week. Some initial thoughts are:

  1. I like the idea of thinking of many filepath APIs as just AbstractTree operations.
  • I wish that package was around when I first started working on this
  • I think it removes any chance of this package getting into base, but I'm fine with that tradeoff if it improves usability/generalisability.
  • A lot of filesystem operations still wouldn't be solved by the abstract tree view (e.g., permissions, IO). It'd be unfortunately, if we couldn't have a common API for work with those as well.
  1. It'll be a lot of work, but we might be able to support most of the current API and deprecate as necessary.
  • Updating AWSS3 might be more of a breaking change we'll need to negotiate
  1. I do worry we're still gonna hit some performance vs correctness implications with this approach (ie: tradeoff between unnecessary network calls vs stale info)?

rofinn avatar Jun 22 '22 22:06 rofinn