coldsnap icon indicating copy to clipboard operation
coldsnap copied to clipboard

Several performance optimizations

Open kdavyd opened this issue 4 years ago • 1 comments

About 4.5x overall increase in download throughput. Most of this is by switching to release build, but other optimizations got a combined ~1.5x increase too, as tested on m6i.8xlarge instances with EBS VPC Endpoint.

Focusing on download performance only. Best performance I got is around 230MB/sec on a fully allocated 100GB snapshot.

Issue #, if available:

Description of changes:

  • Used release build.
  • Fat LTO enabled.
  • Allocate 520KB buffer for each request. (512KB + overhead, 8KB-aligned)
  • Tweaked Tokyo thread settings.
  • Increased worker threads to match Tokyo.
  • Added more retries.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

kdavyd avatar Aug 31 '21 23:08 kdavyd

Thanks for this, @kdavyd! We're giving it a look!

jpculp avatar Sep 03 '21 20:09 jpculp