RedisCommandException during Azure Cluster failover

[MarcinFrankowski]

Configuration:

• StackExchange.Redis 2.5.61
• Azure Redis with clustering enabled, 2 shards (Redis version 6.0.14)

I've noticed RedisCommandException being thrown in WriteMessageToServerInsideWriteLock during the scheduled maintenance window.

"id": "3923059",
"outerId": "17459436",
"type": "StackExchange.Redis.RedisCommandException",
"message": "Command cannot be issued to a replica: HDEL <KEY>",
"parsedStack": [{
  "level": 0,
  "method": "StackExchange.Redis.PhysicalBridge.WriteMessageToServerInsideWriteLock",
  "assembly": "StackExchange.Redis, Version=2.0.0.0, Culture=neutral, PublicKeyToken=c219ff1ca8c2ce46",
  "fileName": "/_/src/StackExchange.Redis/PhysicalBridge.cs",
  "line": 1411
}]

This issue seems to be related to #1504 which should be resolved by #1876. It looks like it's caused by the planned failover. The topology change was not recognized by the library at the time the command was sent. We didn't have any retry policy set up for that particular exception so it caused a brief downtime of a part of the system (less than 30s).

The question here is: should we still expect this exception when rebooting one of the shards in a cluster? Maybe it's worth adding a guideline on how to handle that to the docs.

I've already updated SE.Redis to the latest version and put a retry policy for that in place but wanted to report anyway. Thanks