aws-c-s3
aws-c-s3 copied to clipboard
Delivery exact bytes for read window
Issue #, if available:
you may still receive some data after the window reaches 0.
- The current implementation will deliver data by parts, and if you have only 1 bytes in the read window, the whole part will be delivered.
- Given the downstream may need to buffer the data and this behavior may cause the downstream to overflow the flow control window and lead to oom issue. And as the dynamic sizing may increase the part size under the cover, this behavior makes downstream hard to control the memory usage.
Description of changes:
- Make sure if the body callback set, we will not delivery more bytes if the window reaches 0
- The s3 client will still download the full part if the window is less than part size, but only deliver the available bytes, and push the remain back to the queue and wait for the window to be open again.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Codecov Report
:x: Patch coverage is 97.87234% with 1 line in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 89.18%. Comparing base (372ffea) to head (fb3b302).
| Files with missing lines | Patch % | Lines |
|---|---|---|
| source/s3_meta_request.c | 97.87% | 1 Missing :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #600 +/- ##
==========================================
+ Coverage 89.13% 89.18% +0.05%
==========================================
Files 23 23
Lines 7539 7576 +37
==========================================
+ Hits 6720 6757 +37
Misses 819 819
| Files with missing lines | Coverage Δ | |
|---|---|---|
| source/s3_meta_request.c | 91.34% <97.87%> (+0.27%) |
:arrow_up: |
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.