transport: BBRv2 congestion controller

[WesleyRosenblum]

Background:

s2n-quic currently offers a single congestion controller implementation based on RFC8312, “CUBIC for Fast Long-Distance Networks”. Cubic has been the default congestion control algorithm used in Linux for 15 years, and is widely deployed across the Internet. In 2016, Google proposed the BBR (Bottleneck Bandwidth and Round-trip propagation time) congestion controller and since then most of the discussion on moving beyond Cubic have centered on BBR.

The second version of BBR, BBRv2, was introduced by Google in 2018 to address concerns with fairness and has been in a development state since then. Besides some algorithmic tweaks, BBRv2 incorporates additional signals, including packet loss (v1 ignored packet loss entirely), and ECN (Explicit Congestion Notifications). Given s2n-quic’s use in a variety of network topologies with their associated different buffer capacities, having BBRv2 as an option for customers to use would prove beneficial for those customers with deployments in networks that align with BBRv2’s demonstrated performance advantages.

BBRv2 is currently specified in a draft RFC: https://datatracker.ietf.org/doc/draft-cardwell-iccrg-bbr-congestion-control/

Tasks (WIP):

• [x] Bandwidth estimation #1244 #1245
• [x] Windowed filter #1273
• [x] Network model #1290
• [x] Send quantum #1308
• [x] Main Controller #1405 #1409
• [x] Startup #1405
  • [x] Recovery state #1261
  • [x] Full pipe estimator #1261
• [x] Drain #1405
• [x] ProbeBW #1312
• [x] ProbeRTT #1393
• [x] Packet loss handling #1415
• [x] Idle restart logic #1415
• [x] Compliance citations #1418
• [x] Add random generator to Congestion Controller trait and wire #1387
• [x] Refactor Bandwidth to use nanos_per_byte #1439
• [x] Incorporate ECN notifications #1421 #1425
• [x] Implement the "fast path" optimization #1446
• [x] Add BBRv2 to the public API (initially unstable) #1450
• [x] Fuzz tests to ensure no panics #1452 #1454
• [x] Add #[inline] where appropriate #1465
• [ ] Figure out the correct type (u32 or u64 or a newtype ) for bandwidth and inflight related fields
• [ ] Follow up on some of the RFC bugs/issues or see if there are workarounds
• [x] Unit test coverage
• [x] Monte Carlo simulations to try to simulate the performance implications
• [ ] Comparison to Cubic in Netbench