lakeFS icon indicating copy to clipboard operation
lakeFS copied to clipboard

HL Python SDK: Support copy object between repositories

Open N-o-Z opened this issue 1 year ago • 1 comments

Prerequisite: https://github.com/treeverse/lakeFS/issues/7446

Support providing a destination repo (Optional for backwards compatibility)

N-o-Z avatar Feb 08 '24 05:02 N-o-Z

This seems doable, but for "common" usage some interactions with garbage collection might be strange.

For externally-managed objects this is obviously easy enough to do. For an internally-managed object -- anything where lakeFS choses a path inside the repository storage namespace -- ownership makes things muddier. The source repository "owns" the object, meaning that it can garbage collect it. So if the source object is never committed it may vanish within a few days, and if committed it can vanish once it falls off of the head of all branches and according to a complex set of rules.

The API does not support zero-copy links even within a single repository because of the associated surprises. We can support cross-repository zero-copy links even without the API, sure, but they be even more surprising.

arielshaqed avatar Feb 08 '24 06:02 arielshaqed