firedancer icon indicating copy to clipboard operation
firedancer copied to clipboard

quic: inflight packet pool should be shared across conns

Open ripatel-fd opened this issue 10 months ago • 0 comments

Problem

Currently, every QUIC connection owns an exclusive pool of inflight packets. This pool is used to track what has been sent to the peer which is eventually used when processing ACKs.

The fdctl default config file specifies a depth of 65536 packets across 2048 conns, which is about 67 GB (!) of memory. This scales badly with connection table size and limits the per-connection burst rate.

Suggested Fix

Instead, let each quic tile manage a shared pool of inflight packets.

The pool size is determined by the max packet burst rate in a time window of end-to-end latency. E.g. for 1Mpps burst rate and 500ms end-to-end latency, the pool size is 250MB.

ripatel-fd avatar Mar 28 '24 20:03 ripatel-fd