openshift-docs icon indicating copy to clipboard operation
openshift-docs copied to clipboard

OBSDOCS-547: Remove explicit mention of local persistent storage volu…

Open eromanova97 opened this issue 10 months ago • 6 comments

Version(s): enterprise-4.12 and later

Issue: OBSDOCS-547

Link to docs preview: Configuring persistent storage

QE review:

  • [ ] QE has approved this change.

Additional information:

eromanova97 avatar Apr 15 '24 12:04 eromanova97

@eromanova97: This pull request references OBSDOCS-547 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.16.0" version, but no target version was set.

In response to this:

Version(s): enterprise-4.12 and later

Issue: OBSDOCS-547

Link to docs preview:

QE review:

  • [ ] QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot avatar Apr 15 '24 12:04 openshift-ci-robot

🤖 Wed Jun 19 06:59:08 - Prow CI generated the docs preview:

https://74645--ocpdocs-pr.netlify.app/ https://74645--ocpdocs-pr.netlify.app/openshift-dedicated/latest/observability/monitoring/configuring-the-monitoring-stack.html https://74645--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/monitoring/configuring-the-monitoring-stack.html https://74645--ocpdocs-pr.netlify.app/openshift-rosa/latest/observability/monitoring/configuring-the-monitoring-stack.html

ocpdocs-previewbot avatar Apr 15 '24 12:04 ocpdocs-previewbot

@eromanova97: This pull request references OBSDOCS-547 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.16.0" version, but no target version was set.

In response to this:

Version(s): enterprise-4.12 and later

Issue: OBSDOCS-547

Link to docs preview: Configuring persistent storage

QE review:

  • [ ] QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot avatar Apr 15 '24 13:04 openshift-ci-robot

@eromanova97: This pull request references OBSDOCS-547 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.16.0" version, but no target version was set.

In response to this:

Version(s): enterprise-4.12 and later

Issue: OBSDOCS-547

Link to docs preview: Configuring persistent storage

QE review:

  • [ ] QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot avatar Apr 15 '24 13:04 openshift-ci-robot

@eromanova97: This pull request references OBSDOCS-547 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.16.0" version, but no target version was set.

In response to this:

Version(s): enterprise-4.12 and later

Issue: OBSDOCS-547

Link to docs preview: Configuring persistent storage

QE review:

  • [ ] QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot avatar May 09 '24 09:05 openshift-ci-robot

seems we should add one Prerequisites when configure PVC https://74645--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/monitoring/configuring-the-monitoring-stack.html#configuring-a-persistent-volume-claim_configuring-the-monitoring-stack * Available StorageClass exists in the cluster. reason: if there is not StorageClass, we can not create PVC

juzhao avatar Jun 06 '24 02:06 juzhao

/retest

eromanova97 avatar Jun 06 '24 08:06 eromanova97

/lgtm

juzhao avatar Jun 06 '24 09:06 juzhao

@eromanova97: This pull request references OBSDOCS-547 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.17.0" version, but no target version was set.

In response to this:

Version(s): enterprise-4.12 and later

Issue: OBSDOCS-547

Link to docs preview: Configuring persistent storage

QE review:

  • [x] QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot avatar Jun 06 '24 10:06 openshift-ci-robot

/retest

eromanova97 avatar Jun 06 '24 10:06 eromanova97

/retest

eromanova97 avatar Jun 06 '24 11:06 eromanova97

/retest

eromanova97 avatar Jun 06 '24 14:06 eromanova97

Hmm, I'm not sure that the expanded storage will be effective without additional steps (like described in https://prometheus-operator.dev/docs/operator/storage/#resizing-volumes).

CC: @machine424

eromanova97 avatar Jun 10 '24 10:06 eromanova97

Hmm, I'm not sure that the expanded storage will be effective without additional steps (like described in https://prometheus-operator.dev/docs/operator/storage/#resizing-volumes).

Which steps more precisely? Here, we ask the user to expand the PVCs by referring them to the storage doc + we ask them to reflect that in the configmap (to adjust the statefulset). I don't think pausing the Prometheus CR makes sense here. Am I missing sth?

machine424 avatar Jun 11 '24 09:06 machine424

I tried the steps in a local cluster and the PV didn't expand. I think that .spec.paused: true isn't necessary but you still need to delete (with orphan mode) the statefulset. Probably QE should verify this.

simonpasquier avatar Jun 11 '24 09:06 simonpasquier

I tried the steps in a local cluster and the PV didn't expand. I think that .spec.paused: true isn't necessary but you still need to delete (with orphan mode) the statefulset. Probably QE should verify this.

The old procedure required updating the configmap first, which would recreate the statefulset and the pods (as the claim part is immutable). It then asked to resize the PVCs and then delete the statefulset in orphan mode which didn’t make sense as the statefulset was already up-to-date at that stage.

Now, we ask to expand the PVC as explained in https://docs.openshift.com/container-platform/4.15/storage/expanding-persistent-volumes.html#expanding-pvc-filesystem_expanding-persistent-volumes, and then update the configmap. This will recreate the statefulset and the pods.

I gave it a try on a clusterbot and it worked as expected. I think Eliska tried it as well, but let’s see if @juzhao can confirm that.

(I’ve discussed this with multiple persons. I think it’d be great if every procedure in the docs is backed up by an automated test. I’ll create a ticket for that and continue the related discussion there.)

machine424 avatar Jun 11 '24 12:06 machine424

~~Ok I'll try again too...~~ @machine424 sorry for the confusion, I was too much focused on the upstream docs and missed that the PR is more about cleaning up the existing procedure :sweat_smile:

simonpasquier avatar Jun 11 '24 12:06 simonpasquier

/test deploy-preview

eromanova97 avatar Jun 12 '24 09:06 eromanova97

I tried the steps in a local cluster and the PV didn't expand. I think that .spec.paused: true isn't necessary but you still need to delete (with orphan mode) the statefulset. Probably QE should verify this.

The old procedure required updating the configmap first, which would recreate the statefulset and the pods (as the claim part is immutable). It then asked to resize the PVCs and then delete the statefulset in orphan mode which didn’t make sense as the statefulset was already up-to-date at that stage.

Now, we ask to expand the PVC as explained in https://docs.openshift.com/container-platform/4.15/storage/expanding-persistent-volumes.html#expanding-pvc-filesystem_expanding-persistent-volumes, and then update the configmap. This will recreate the statefulset and the pods.

I gave it a try on a clusterbot and it worked as expected. I think Eliska tried it as well, but let’s see if @juzhao can confirm that.

(I’ve discussed this with multiple persons. I think it’d be great if every procedure in the docs is backed up by an automated test. I’ll create a ticket for that and continue the related discussion there.)

should edit the PVC size first, then wait for it's resized before resize

$ oc -n openshift-monitoring get pvc
NAME                               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
alertmanager-alertmanager-main-0   Bound    pvc-845a2fe8-8275-4698-860a-b3789572422d   4Gi        RWO            managed-csi    <unset>                 18m
alertmanager-alertmanager-main-1   Bound    pvc-37ea1d39-fbb0-4c3e-950e-d9917b642e02   4Gi        RWO            managed-csi    <unset>                 18m

after resize

$ oc -n openshift-monitoring get pvc
NAME                               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
alertmanager-alertmanager-main-0   Bound    pvc-845a2fe8-8275-4698-860a-b3789572422d   10Gi       RWO            managed-csi    <unset>                 19m
alertmanager-alertmanager-main-1   Bound    pvc-37ea1d39-fbb0-4c3e-950e-d9917b642e02   10Gi       RWO            managed-csi    <unset>                 19m

then edit cluster-monitoring-config configmap

juzhao avatar Jun 12 '24 09:06 juzhao

/lgtm

juzhao avatar Jun 12 '24 11:06 juzhao

/label peer-review-needed

eromanova97 avatar Jun 17 '24 08:06 eromanova97

/label peer-review-in-progress /remove-label peer-review-needed

obrown1205 avatar Jun 18 '24 13:06 obrown1205

A few outstanding questions and nits - overall, great work! /label peer-review-done /remove-label peer-review-in-progress

obrown1205 avatar Jun 18 '24 14:06 obrown1205

New changes are detected. LGTM label has been removed.

openshift-ci[bot] avatar Jun 19 '24 06:06 openshift-ci[bot]

@eromanova97: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci[bot] avatar Jun 19 '24 07:06 openshift-ci[bot]

/label merge-review-needed

eromanova97 avatar Jun 19 '24 07:06 eromanova97

@eromanova97: This pull request references OBSDOCS-547 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.17.0" version, but no target version was set.

In response to this:

Version(s): enterprise-4.12 and later

Issue: OBSDOCS-547

Link to docs preview: Configuring persistent storage

QE review:

  • [x] QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot avatar Jun 20 '24 14:06 openshift-ci-robot

/cherrypick enterprise-4.12

bburt-rh avatar Jun 20 '24 14:06 bburt-rh

/cherrypick enterprise-4.13

bburt-rh avatar Jun 20 '24 14:06 bburt-rh

/cherrypick enterprise-4.14

bburt-rh avatar Jun 20 '24 14:06 bburt-rh