cloud-on-k8s icon indicating copy to clipboard operation
cloud-on-k8s copied to clipboard

The `PHASE` is always `ApplyingChanges` possibly due to volume expansion

Open sissilab opened this issue 1 year ago • 1 comments

Environment:

  • ECK version: 2.8.0
  • Elasticsearch version: 8.7.0
  • kubectl version: v1.23.7
  • ECK Elasticsearch:
    • on premise
    • user local pv
    • mount a shared NFS volume for each elasticsearch node for snapshot

What happened:

After trying to upgrade Elasticsearch version from 8.7.0 to 8.10.2 based on ECK, I found it failed (Elasticsearch version did't change) and saw PHASE turning to ApplyingChanges when get es.

NAME   HEALTH   NODES   VERSION   PHASE             AGE
eck    green  6       8.7.0     ApplyingChanges   15m

Then I restored Elasticsearch version from 8.10.2 to 8.7.0 by restarting eck es (kubectl uninstall -> kubectl install), Elasticsearch was normal. But the PHASE was still ApplyingChanges!!!

The event of es: handle volume expansion: persistentvolumeclaims "elasticsearch-data-eck-es-data-0" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize (BUT I DID NOT CHANGE ANY STORAGE SIZE)

And now, the PHASE is always ApplyingChanges.

Very Thanks!


Update:

  • I have upgraded Elasticsearch version successfully, but the PHASE is still ApplyingChanges.

sissilab avatar Oct 25 '23 03:10 sissilab

I have upgraded Elasticsearch version successfully, but the PHASE is still ApplyingChanges.

Is there anything in the operator's logs that would explain what the operator is trying to reconcile?

barkbay avatar Nov 15 '23 10:11 barkbay

Closing due to inactivity, feel free to reopen if needed.

barkbay avatar Mar 19 '24 08:03 barkbay