Tobias Grieger
Tobias Grieger
From Sentry and now an escalation: - 24.3: https://github.com/cockroachdb/cockroach/issues/158166 - 24.1: https://github.com/cockroachdb/cockroach/issues/157882 - escalation on 24.1: https://github.com/cockroachlabs/support/issues/3506 - cloud logs[^1] [^1]: https://us5.datadoghq.com/logs?query=%22a%20panic%20has%20occurred%22%20env%3Aproduction%20service%3Acockroachdb%20RangeFeedCheckpoint&agg_m=count&agg_m_source=base&agg_t=count&cols=host%2Cservice&fromUser=true&messageDisplay=inline&refresh_mode=paused&storage=flex_tier&stream_sort=desc&viz=stream&from_ts=1759276800000&to_ts=1765384960102&live=false We ocasionally see crashes when a `MuxRangeFeedEvent` is...
This still filters out the large gossip histograms, but keeps the processed callbacks metric. Epic: none
We suspect a data race in rangefeed checkpoint marshaling where a RangeFeedEvent containing a RangeFeedCheckpoint can be modified while being marshaled. This causes panics with "index out of range" errors...