replicator icon indicating copy to clipboard operation
replicator copied to clipboard

wiki: Document sink_io_workers vs BDP

Open bobvawter opened this issue 11 months ago • 1 comments

There's a default limit of 32 concurrent webhook requests. This may be overly conservative for high-throughput (i.e. post-split catchup) scenarios. Bandwidth-delay product is troublesome with a small number of outstanding requests.

https://www.cockroachlabs.com/docs/stable/cluster-settings#setting-changefeed-sink-io-workers

https://github.com/cockroachdb/cockroach/blob/55c5b541e1d0bcfca4c4f190b52ab4274b69a797/pkg/ccl/changefeedccl/sink.go#L844-L858

bobvawter avatar Feb 27 '24 17:02 bobvawter

Also worth mentioning is the changefeed.node_throttle_config setting to rate-limit feeds. Config object:

https://github.com/cockroachdb/cockroach/blob/bd34188450945eaea87caa1cb8caa515c7faa2d3/pkg/ccl/changefeedccl/changefeedbase/settings.go#L135-L150

bobvawter avatar Mar 19 '24 14:03 bobvawter