neon
neon copied to clipboard
pageserver: separate concurrency limits for big vs. small PUTs
Two reasons:
- For big puts, there isn't much value in running lots at the same time: assuming a 25gbps link and each flow achieving at least 100mb/s, you don't want more than 25 flows.
- When the queue for layer uploads is long (it can be tens of seconds long after a big compaction as layers trickle into S3), we should not delay small metadata operations required for timeline creation (uploading index_part.json).