Efficient copying via shutil

[four43]

This is a great extension of pathlib who's core describes file system directories and objects. We are looking to efficiently download and upload data from various backends. shutil.copytree and shutil.copy2, which it leans on, are a nice way to copy between Path objects. Is there fanciness in there that's causing these to not work? copytree blew up on a UPath S3 remote path as the destination and seemed just copy things locally after it stringified the path...

Are these nice methods just calling str([path]) and breaking all the nice things? Are there any related helper methods for copying and moving provided?

Thanks a lot!

Related: https://github.com/fsspec/universal_pathlib/issues/175

EDIT: Uh oh, I'm seeing a lot of legacy os.path style stuff in the source: https://github.com/python/cpython/blob/main/Lib/shutil.py#L432

[ap--]

Hi @four43

Sorry for the delayed response. A shutil like interface would be a great addition. Last time I checked the implementations in the stdlib were implicitly assuming local paths. With current efforts of establishing other PathLike implementations I could imagine that might change.

@barneygale are there plans to make shutil.copytree, shutil.rmtree, etc... compatible with PathBase subclasses or maybe are they already?

In the meantime we could sketch out a upath.shutil api compatible submodule and take advantage of https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.generic.rsync in the implementation.

Cheers, Andreas 😃

[barneygale]

@barneygale are there plans to make shutil.copytree, shutil.rmtree, etc... compatible with PathBase subclasses or maybe are they already?

Yes! I'm working on adding copy(), copytree(), rmtree() and move(): https://github.com/python/cpython/issues/73991

copy() and copytree() have already arrived: https://docs.python.org/3.14/library/pathlib.html#copying-renaming-and-deleting

[ap--]

Wonderful! So we can support this natively once it lands in pathlib-abc ☺️ Thanks @barneygale

@four43 I recommend subscribing to #193 to be notified when I find the time to work on moving to PathBase

[four43]

You guys are legends. Thank you so much for all the effort and work. Subbed to that issue and looking forward to it.

[ap--]

.copy .copy_into .move and .move_into are now available in UPath.