automq icon indicating copy to clipboard operation
automq copied to clipboard

Published 20 hours ago • verified publisher icon AutoMQ

feat(s3stream): adopt objectStorage throttleStrategy for upload WAL task

Open lifepuzzlefun opened this issue 7 months ago • 4 comments

what is changed.

  1. removed UploadWalTask ratelimiter let object storage do the throttle logic.

why

  1. in our poc environment we remove the limit logic at first. the s3.network.baseline.bandwidth is 1GB/s
  2. the upload task cost time drop from 3000ms to 500ms.
  3. the failover duration drop from 5s to 1s per partition

lifepuzzlefun avatar Jul 25 '24 10:07 lifepuzzlefun