helix icon indicating copy to clipboard operation
helix copied to clipboard

CustomizedView - updates when node shutdown ungracefully

Open desaikomal opened this issue 1 year ago • 0 comments

Describe the bug

Helix Customized View has following behavior

  • participant maintains the customized-state per-instance under /INSTANCE/CUSTOMIZED_VIEW//resource
  • Participant is responsible updating the state.
  • Helix controller aggregate per-instance customized-state and publishes at cluster-level.

So per instance state under given cluster, path - /INSTANCE/CUSTOMIZEDSTATES/<stateName> is maintained purely by instance.

Instance can go gracefully down, in which case they can update the state before going down but in case instance shutdown due to ungraceful shutdown, CUSTOMIZEDSTATE becomes STALE.

From Helix controller side, it is difficult to make out when LIVEINSTANCE ephmeral node is deleted whether: host temporarily disconnected, host gracefully shutdown or host ungracefully went down. Controller can't decide and so it doens't know when aggregating.

If customizedstate maintained session, then also you can't make out as temporary disconnect vs/vs ungraceful will be hard to do it.

So we need to find out if there are any hooks when participant re-joins the cluster, it populates the value.

To Reproduce

Steps to reproduce the behavior.

Expected behavior

A clear and concise description of what you expected to happen.

Additional context

Add any other context about the problem here.

desaikomal avatar Jul 20 '23 21:07 desaikomal