rclone icon indicating copy to clipboard operation
rclone copied to clipboard

drive: Batch creation of directories

Open AndrewThrift opened this issue 5 years ago • 2 comments

What is your current rclone version (output from rclone version)?

rclone v1.45

What problem are you are trying to solve?

I have a backup application that creates a large directory structure to store file chunks in (65535 directories at top level). When replicating this, it takes almost 9 hours to recreate the directory structure on Google Drive. It seems that rclone is performing one directory creation per API call.

How do you think rclone should be changed to solve that?

I see Google Drive supports batch operations of up to 100 operations per API call. If it is possible, it would be great if rclone could create 100 directories at a time. This would significantly drop the time needed to create large directory structures.

AndrewThrift avatar Jan 04 '21 07:01 AndrewThrift

Hmm, interesting idea.

I don't think the go SDK supports batch operations (I'd be delighted to be proved wrong!) though :-(

ncw avatar Jan 05 '21 17:01 ncw

drive v3 support batch request: https://developers.google.com/workspace/drive/api/guides/performance and google-api-go-client has defined it: https://github.com/googleapis/google-api-go-client/blob/811ba14d5fc1b87bd0058d49b80293164db986fa/drive/v3/drive-api.json#L40

scal-csimon avatar Nov 12 '25 23:11 scal-csimon