openshift-docs icon indicating copy to clipboard operation
openshift-docs copied to clipboard

[OSDOCS-4353]: Adds GCP workload ID upgrade procedure

Open jeana-redhat opened this issue 2 years ago • 3 comments

Version(s): 4.11+

Issue: OSDOCS-4353

Link to docs preview:

QE review:

  • [ ] QE has approved this change.

Additional information: N/A

jeana-redhat avatar Oct 17 '22 17:10 jeana-redhat

🤖 Updated build preview is available at: https://51752--docspreview.netlify.app

Build log: https://circleci.com/gh/ocpdocs-previewbot/openshift-docs/2088

ocpdocs-previewbot avatar Oct 17 '22 17:10 ocpdocs-previewbot

@abutcher PTAL when you have time

jeana-redhat avatar Oct 17 '22 17:10 jeana-redhat

@jianping-shu PTAL :pray:

jeana-redhat avatar Oct 18 '22 00:10 jeana-redhat

@abutcher @jeana-redhat I tested w/ case OCP-55031, create-all has some issue I tested 2 scenarios, scenario 1, run ccoctl gcp create-all with a new and empty output-dir in upgrade steps, the RSA key pair was re-generated and tls/bound-service-account-signing-key.key is different to that in installation The upgrade hung up for long time jianpingshu@jshu-mac 4.11.7-manifests % oc get clusterversion -w NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.10.34 True True 3h21m Working towards 4.11.7: 637 of 803 done (79% complete), waiting on image-registry

There was the following error for co image-registry image-registry 4.10.34 True True False 170m Progressing: Unable to apply resources: unable to sync storage configuration: Get "https://storage.googleapis.com/storage/v1/b/jshu-gcp6-bsn42-image-registry-us-central1-yanxcjvqmhhnvwxwysj?alt=json&prettyPrint=false&projection=full": oauth2/google: unable to generate access token: Post "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/[email protected]:generateAccessToken": oauth2/google: status code 400: {"error":"invalid_grant","error_description":"Unable to verify the ID Token signature."}...

scenario 2, run ccoctl gcp create-all with the same output-dir in install/upgrade steps, the RSA key pair was reused and tls/bound-service-account-signing-key.key is re-generated but same The upgrade was successful in 1 hour jianpingshu@jshu-mac 4.11.7-manifests2 % oc get clusterversion -w NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.11.7 True False 6m1s Cluster version is 4.11.7

I guess tls/bound-service-account-signing-key.key is used in the created resources by ccoctl, so if it re-generated then it will not match with the one in the cluster. I think the ccoctl output-dir in installation might be long gone when upgrade, so create-service-accounts makes more sense. WDYT?

jianping-shu avatar Oct 18 '22 10:10 jianping-shu

@jeana-redhat Here are some comments not related to this PR. Maybe we can create another ticket for tracking them if they are valid and need some effort.

  1. $ CCO_IMAGE=$(oc adm release info --image-for='cloud-credential-operator' $RELEASE_IMAGE) In my run, it needs "-a ~/.pull-secret" But it may depend on my environment, may be not a real issue
  2. Feature gate in 4.11 vs. feature set in 4.12, this is an general issue
  3. Comparing w/ aws sts procedure, gcp procedure doesn't list the role/permission needed for running ccoctl
  4. In Example output of "ccoctl aws create-identity-provider", "where 02-openid-configuration is a discovery document and 03-keys.json is a JSON web key set file." But neither 02-openid-configuration nor 03-keys.json exists in the output. Should be openid-configuration and keys.json?

jianping-shu avatar Oct 18 '22 10:10 jianping-shu

I think the ccoctl output-dir in installation might be long gone when upgrade, so create-service-accounts makes more sense. WDYT?

Yes, I hadn't considered the output-dir not being around which seems very likely. create-service-accounts makes more sense :+1:.

abutcher avatar Oct 18 '22 13:10 abutcher

Thanks for confirmation! Let's change to create-service-accounts and the procedure will be fine.

jianping-shu avatar Oct 18 '22 13:10 jianping-shu

Thanks for the review! Leaving some comments below:

@jeana-redhat Here are some comments not related to this PR. Maybe we can create another ticket for tracking them if they are valid and need some effort.

1. $ CCO_IMAGE=$(oc adm release info --image-for='cloud-credential-operator' $RELEASE_IMAGE)
   In my run, it needs "-a ~/.pull-secret"
   But it may depend on my environment, may be not a real issue

I actually did this a bit more granularly in a different ccoctl PR I have in-flight, I will add that detail here in case users have the same issue.

2. Feature gate in 4.11 vs. feature set in 4.12, this is an general issue

Yes, I have a card to make this change in 4.12: OSDOCS-4159

3. Comparing w/ aws sts procedure, gcp procedure doesn't list the role/permission needed for running ccoctl

I opened AWS and GCP docs cards at the same time, but the GCP update is waiting on CCO-197.

4. In Example output of "ccoctl aws create-identity-provider",
   "where 02-openid-configuration is a discovery document and 03-keys.json is a JSON web key set file."
   But neither 02-openid-configuration nor 03-keys.json exists in the output. Should be openid-configuration and keys.json?

Good catch! I will fix that while I am in here rather than adding to my to-do list for later :slightly_smiling_face:

jeana-redhat avatar Oct 18 '22 14:10 jeana-redhat

Looks good to me, LGTM

jianping-shu avatar Oct 19 '22 02:10 jianping-shu

/label peer-review-in-progress

sheriff-rh avatar Oct 19 '22 19:10 sheriff-rh

/remove-label peer-review-needed

sheriff-rh avatar Oct 19 '22 19:10 sheriff-rh

/remove-label peer-review-in-progress /label peer-review-done

sheriff-rh avatar Oct 19 '22 19:10 sheriff-rh

Force pushed to force rebuild before merging, no change.

jeana-redhat avatar Oct 19 '22 19:10 jeana-redhat

/cherrypick enterprise-4.12

jeana-redhat avatar Oct 19 '22 19:10 jeana-redhat

/cherrypick enterprise-4.11

jeana-redhat avatar Oct 19 '22 19:10 jeana-redhat

@jeana-redhat: new pull request created: #51878

In response to this:

/cherrypick enterprise-4.12

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@jeana-redhat: new pull request created: #51879

In response to this:

/cherrypick enterprise-4.11

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.