etcd icon indicating copy to clipboard operation
etcd copied to clipboard

[release-3.5] server: cleanup zombie membership information

Open fuweid opened this issue 2 weeks ago • 3 comments

There are three places where zombie membership information can be cleaned up:

1. ApplierV2 -> UpdateAttributes

   Ideally, each member updates its own attributes only once before serving
   requests. Since this happens only at startup and at low frequency, it is
   safe to clean up zombie members during this operation.

   Additionally, pending logs are always applied before attributes are updated,
   so there is no data race.

2. Applying a snapshot via Raft message

   When a member receives a snapshot through a Raft message, the v2store is
   synchronized with the v3store. This makes it safe to detect and clean up
   zombie membership entries at that time.

3. Applying ConfChange

   Check and cleanup zombie members after ConfChange. Ideally, there aren't
   too many ConfChange, the impact on performance should be minimal.

Fixes: #20967

Please read https://github.com/etcd-io/etcd/blob/main/CONTRIBUTING.md#contribution-flow.

fuweid avatar Dec 05 '25 16:12 fuweid

@ahrtr for this comment,

As mentioned above, I am thinking probably we should handle the case missing members in v3store (exist in v2, but not in v3) as well, although we haven't seen any reproduction for such case so far.

I think we can keep it as it is right now. based on current situation, v3store has more information than v2store. If we don't have such case like v2store contains more information than v3, we can assume it's not going to happen.

fuweid avatar Dec 10 '25 16:12 fuweid

If we don't have such case like v2store contains more information than v3, we can assume it's not going to happen.

Hopefully we won't have to fix a follow-up issue and generate a follow-up blog.

ahrtr avatar Dec 10 '25 18:12 ahrtr

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahrtr, fuweid

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Dec 10 '25 18:12 k8s-ci-robot