vault-on-gke icon indicating copy to clipboard operation
vault-on-gke copied to clipboard

storage migration check error

Open SrikantPatil88 opened this issue 4 years ago • 5 comments

My Vault cluster was working properly yesterday, after upgrade it stopped working

Getting below error in stack driver,

[WARN] storage migration check error: error="failed to read value for "core/migration": googleapi: got HTTP response code 403 with body: <Error><Code>AccessDenied</Code><Message>Access denied.</Message><Details>Primary: /namespaces/service account with additional claims does not have storage.objects.get access to the Google Cloud Storage object.</Details></Error>"

The status of Vault pod, containers with unready status: [vault]

SrikantPatil88 avatar May 29 '20 09:05 SrikantPatil88

Hi,

Does anyone seen this error in gcp stack driver after upgrade?

SrikantPatil88 avatar Jun 02 '20 06:06 SrikantPatil88

Hi there! This has been automatically marked as stale because it has not had activity in the past 14 days. It will be closed in 14 days if no further activity takes place.

stale[bot] avatar Jun 16 '20 11:06 stale[bot]

@SrikantPatil88 did you manage to fix this ? I have the same issue despite that storage object admin permission is granted

bluemalkin avatar Apr 20 '21 09:04 bluemalkin

@bluemalkin, Yes, We manage to solve the issue.

The issue was the default k8’s service account required binding to the Google service account within the workload Identity.

Workload Identities were added in Kubernetes 1.14 + and are required when the master nodes were upgraded. https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity

SrikantPatil88 avatar Apr 21 '21 05:04 SrikantPatil88

@bluemalkin, Yes, We manage to solve the issue.

The issue was the default k8’s service account required binding to the Google service account within the workload Identity.

Workload Identities were added in Kubernetes 1.14 + and are required when the master nodes were upgraded. https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity

Thanks - I fixed my issue, I used the wrong annotation for the service account. I'm using workload identity too and it works well.

You may want to close this ticket.

bluemalkin avatar Apr 21 '21 05:04 bluemalkin