filesystem_spec icon indicating copy to clipboard operation
filesystem_spec copied to clipboard

Update the *hdfs docs about streamed copy

Open isidentical opened this issue 4 years ago • 1 comments

See: https://github.com/intake/filesystem_spec/pull/762#issuecomment-923967545

isidentical avatar Sep 21 '21 13:09 isidentical

Short summary: some implementations don't support a native copy operation, but we can and do emulate it by reading the file and writing to a new remote file. This achieves the same thing, but involves steaming all the data through the local process, which may be surprising to users.

This issue is to discuss if/where we should let users know.

martindurant avatar Sep 21 '21 13:09 martindurant