aws-c-s3 icon indicating copy to clipboard operation
aws-c-s3 copied to clipboard

Delivery exact bytes for read window

Open TingDaoK opened this issue 2 weeks ago • 1 comments

Issue #, if available:

you may still receive some data after the window reaches 0.

  • The current implementation will deliver data by parts, and if you have only 1 bytes in the read window, the whole part will be delivered.
  • Given the downstream may need to buffer the data and this behavior may cause the downstream to overflow the flow control window and lead to oom issue. And as the dynamic sizing may increase the part size under the cover, this behavior makes downstream hard to control the memory usage.

Description of changes:

  • Make sure if the body callback set, we will not delivery more bytes if the window reaches 0
  • The s3 client will still download the full part if the window is less than part size, but only deliver the available bytes, and push the remain back to the queue and wait for the window to be open again.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

TingDaoK avatar Nov 21 '25 22:11 TingDaoK

Codecov Report

:x: Patch coverage is 97.87234% with 1 line in your changes missing coverage. Please review. :white_check_mark: Project coverage is 89.18%. Comparing base (372ffea) to head (fb3b302).

Files with missing lines Patch % Lines
source/s3_meta_request.c 97.87% 1 Missing :warning:
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #600      +/-   ##
==========================================
+ Coverage   89.13%   89.18%   +0.05%     
==========================================
  Files          23       23              
  Lines        7539     7576      +37     
==========================================
+ Hits         6720     6757      +37     
  Misses        819      819              
Files with missing lines Coverage Δ
source/s3_meta_request.c 91.34% <97.87%> (+0.27%) :arrow_up:
:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov-commenter avatar Nov 22 '25 00:11 codecov-commenter