Receive Buffer Refactoring

[nibanks]

Description

Refactors the receive buffer interface to support 3 modes of operation:

1. Single buffer mode - This is used by the crypto stream, and ensures reads are always one continuous block (because TLS doesn't support gather semantics).
2. Circular buffer mode - This is used by the streams, and allows for the read calls to return 2 pointers/lengths that map to different parts of the circular buffers stored internally.
3. Multi-read buffer mode - This will be used (opt-in) by the streams in a future PR. It allows for multiple outstanding reads from a single receive buffer.

The first two "modes" already practically existed, but were defined with slightly different flags. Now the distinction is very clear. There is also a lot of other logic internally required to manage external references in a more generic method.

Testing

Lots of new unit tests added. Also leverages existing CI/CD for any e2e issues.

Documentation

Heavily commented code, but no external docs.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 85.12658% with 47 lines in your changes are missing coverage. Please review.

Project coverage is 85.99%. Comparing base (86e08e5) to head (3080432). Report is 45 commits behind head on main.

:exclamation: Current head 3080432 differs from pull request most recent head 07e4f68. Consider uploading reports for the commit 07e4f68 to get more accurate results

Files	Patch %	Lines
src/core/recv_buffer.c	84.81%	46 Missing :warning:
src/core/stream.c	85.71%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4042      +/-   ##
==========================================
- Coverage   87.38%   85.99%   -1.39%     
==========================================
  Files          56       56              
  Lines       16984    17114     +130     
==========================================
- Hits        14841    14717     -124     
- Misses       2143     2397     +254     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[maolson-msft]

Based on the high-level description, CIRCULAR sounds like just a N=1 case of MULTIPLE. So why have a separate mode?

[nibanks]

Based on the high-level description, CIRCULAR sounds like just a N=1 case of MULTIPLE. So why have a separate mode?

They're related, but not exactly that simple. Only the first buffer is used in a circular fashion. This comes does to the usage pattern, and how draining from the front happens. Only the first buffer is ever drained from, so only it may have a case where partial data is read from, and it makes sense (to eliminate the copy) to just move where we consider the HEAD.

[nibanks]

Let me know, and I can whiteboard it for you.