Eugene Klimov

Unfortunately, issue can't be resolved immediately try to use `https://rclone.org/s3/#tencent-cos` instead of `clickhouse-backup upload / download` or `remote_storage: s3` instead COS directly

@michael-liumh sorry for late reply, could you try to 1.5.0 version and share results for the following command (without sensitive credentials)? ``` clickhouse-backup print-config ```

Thanks a lot for suggestion, I will try! Your original topic https://github.com/AlexAkulov/clickhouse-backup/issues/316#issuecomment-993503130 was about `download` not `upload` failed upload to `cos` is duplicated with https://github.com/AlexAkulov/clickhouse-backup/issues/464

Could you clarify, upload also worked for COS or not?

I really appreciate for your detailed reporting and feedback Could you provide more context? Which clickhouse-backup version do you use for benchmark? Which environment do you use for benchmark? Do...

> I have 16 CPU with 128 GB RAM machines. What are recommended values for S3_CONCURRENCY, DOWNLOAD_CONCURRENCY and UPLOAD_CONCURRENCY? Check how much memory allocated and use DOWNLOAD_CONCURRENCY=8 UPLOAD_CONCURRENCY=8 S3_CONCURRENCY=4 and...

@malcolm061990 any results with lower concurrency numbers? I tried to apply multipart concurrency download implementation, unfortunately it requires allocating additional disk space during download, and we can't apply in-memory streaming...

> But why does it require additional disk space during download? Currently, we use pool of parallel go-routines, each go-routine download one s3://bucket-name/path/backup_name/db/table/disk_name.tar file. S3 allow us to use internal...

>> we need to change remote storage file format and upload files directly > Good idea but its not clear :) What do you mean? Currently each table data create...

@malcolm061990 now, 1.5.x version released which have `upload_by_parts: true` and `download_by_parts: true` in general section could you try it and compare download benchmark? currently, i close issue after inactivity but...