Nick Craig-Wood

I'll just note that this is mentioned in the FAQ https://rclone.org/faq/#rclone-is-using-too-much-memory-or-appears-to-have-a-memory-leak And also that I'm scheduled to implement an out of memory sync later this year which will fix the...

I forgot to say there is a workaround for this written up in the wiki - https://github.com/rclone/rclone/wiki/Big-syncs-with-millions-of-files Which came from this forum post originally - https://forum.rclone.org/t/rclone-sync-s3-to-s3-runs-for-hours-and-copy-nothing/39687/23 It isn't the best...

If anyone would like to try the solution to this, here is rclone with an out of memory sync for directories with lots of entries. This should fix the memory...

@mNantern I noticed that once in my testing but I didn't manage to replicate it. How many files of what size did you have?

@zackees in theory it shouldn't use more than 1 or 2 GB ever. However that hasn't been tested much yet so I'd appreciate hearing about your findings. If you can...

@zackees thanks for the info. It would be useful to know what the distribution of files is in the source bucket. So roughly how many files and how big? I...

@zackees wrote: > The distribution here is WILD. Some files are a few megabytes, some are 300 GB+ OK > Thanks for pointing out `--s3-upload-concurrency` > > I'm a bit...

Thanks for testing everyone. I've merged this to master now which means it will be in [the latest beta](https://beta.rclone.org/) in 15-30 minutes and released in v1.70

You can update credentials via the API Check out https://rclone.org/s3/#set for details.

> rclone rc backend/command command=set fs=obs-mount: -o access_key_id=correct_ak -o secret_acces... You have to use exactly the same `fs` that you used in the mount command. So something like rclone rc...