Support removal of specific pod when scaling down a NodeSet

[PhaedrusTheGreek]

Sometimes while managing Elasticsearch, especially under unexpected conditions, it becomes necessary to manually remove a specific node from a cluster.

It seems that there is some momentum here for support in StatefulSet in this issue

[charith-elastic]

I assume you mean remove without replacement. Currently if you delete a pod, it will be replaced automatically with any existing persistent volumes re-attached. Can you elaborate more about your use case where that is not desired?

[PhaedrusTheGreek]

Usually this would be useful during a scale-down, when resources are not uniform, and migrations are days long.

Suppose we are scaling down an unstable cluster, and we have one node that is already holding far less data than the others (due to some failure), and a migration would be faster and safer to move from this node.

[charith-elastic]

Makes sense. If the upstream issue gets resolved and Kubernetes gains support for selective down-scaling, we can consider incorporating that to the ECK down-scaling logic at some point.

[nadeemhussain]

for this we can use how clickhouse operator does, there each node they are creating a specifc sts and each sts has one pod , pro: of this approach is we can remove any node we want without moving data around when scaling down , con: we will end up with same no of sts as there are pod , which should not be an issue