server: recover from index out of range panic in MuxRangeFeed Send

[tbg]

We suspect a data race in rangefeed checkpoint marshaling where a RangeFeedEvent containing a RangeFeedCheckpoint can be modified while being marshaled. This causes panics with "index out of range" errors because marshaling computes the size needed first, then creates a buffer, and the buffer ends up too small when the underlying data changes.

This commit adds panic recovery in lockedMuxStream.Send to catch these specific panics and convert them to errors instead of crashing. Other panics are still re-raised after logging.

This patch backports with relative ease all the way back to 24.1.

Informs: https://github.com/cockroachdb/cockroach/issues/159166 Informs: https://github.com/cockroachdb/cockroach/issues/157882 Informs: https://github.com/cockroachdb/cockroach/issues/138895 Informs: https://github.com/cockroachdb/cockroach/issues/158166 Informs: https://github.com/cockroachlabs/support/issues/3506

Release note: None

[cockroach-teamcity]

This change is