aws-c-s3 icon indicating copy to clipboard operation
aws-c-s3 copied to clipboard

Optimize the sizes of buffers requested from mem pool

Open DmitriyMusatkin opened this issue 2 months ago • 1 comments

Issue #, if available:

Description of changes: There were a couple corner cases where scheduler would request more mem from mem pool than needed, ex. for a small ranged get, the scheduler requested part sized buffer anyways. With default mem pool, the mem pool will still do part size for that case (we can optimize that later), but for custom mem pools we want to provide an option to do more accurate allocation.

Note: that for put streaming we cant always request an accurate buffer cause we really on finding eof in the stream, so its kinda of a chicken and egg problem

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

DmitriyMusatkin avatar Sep 09 '25 18:09 DmitriyMusatkin

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 88.73%. Comparing base (6c203e4) to head (c0b262a).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #563      +/-   ##
==========================================
+ Coverage   88.72%   88.73%   +0.01%     
==========================================
  Files          22       22              
  Lines        6767     6776       +9     
==========================================
+ Hits         6004     6013       +9     
  Misses        763      763              
Files with missing lines Coverage Δ
source/s3_auto_ranged_get.c 98.01% <100.00%> (+0.01%) :arrow_up:
source/s3_auto_ranged_put.c 92.13% <100.00%> (+0.03%) :arrow_up:
source/s3_client.c 91.83% <100.00%> (+<0.01%) :arrow_up:
source/s3_meta_request.c 91.29% <100.00%> (ø)
source/s3_request.c 77.40% <100.00%> (+0.20%) :arrow_up:
:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov-commenter avatar Sep 09 '25 19:09 codecov-commenter