fs.cp_file: out of memory error (don't store file in RAM)

[casperdcl]

https://github.com/iterative/PyDrive2/blob/378c6e80471d44f4164e64a178efc2f06ecae882/pydrive2/fs/spec.py#L358-L365

ref. https://github.com/iterative/PyDrive2/pull/119#pullrequestreview-753066960 suggestion:

-    """In-memory streamed copy""" 
-    with self.open(lpath) as stream: 
-        # IterStream objects doesn't support full-length 
-        # seek() calls, so we have to wrap the data with 
-        # an external buffer. 
-        buffer = io.BytesIO(stream.read()) 
-        self.upload_fobj(buffer, rpath) 
+        with self.open(lpath) as stream:
+            self.upload_fobj(stream, rpath)

and fix self.open to return a proper stream.

[shcheklein]

@isidentical could you give more context on this?

[isidentical]

Normally a on-server copy was not available when migrating from the dvc's pydrivefilesystem, so I added it with a simple interface for the sake of simplicty in order to share test suites with sshfs. Though we do not depend on that method (it is only used by the DVC when doing external outputs flow AFAIK, and gdrive does not support it).

[casperdcl]

hmm so "external outputs" will affect https://github.com/iterative/dvc/issues/6485, right?

[isidentical]

I don't think that command involves any on the server copy, does it? I think it would be only relevant here https://github.com/iterative/dvc/issues/5811

[efiop]

yeah, it is not related to put-url, not right now at least. Still would be nice not to load the whole thing into memory, of course.

[shcheklein]

PyDrive2 interface has native CopyFile implementation now - https://docs.iterative.ai/PyDrive2/pydrive2/#pydrive2.files.GoogleDriveFile.Copy