[Enhancement]: Rewatch once when session keep alive fail

[wayblink]

Is there an existing issue for this?

• [X] I have searched the existing issues

What would you like to be added?

If etcd master node crash, etcd cluster can continue to work but all keep alive will disconnect. Hence, milvus nodes will quit if etcd master node crash. We need to try re-register once when keep alive fail.

Why is this needed?

No response

Anything else?

No response

[wayblink]

/assign

[xiaofan-luan]

Good catch, If etcd failed for some reason, we have to either fail fast or rewatch .

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

[xiaofan-luan]

keep it

[xiaofan-luan]

@wayblink any progress?