velero icon indicating copy to clipboard operation
velero copied to clipboard

v1.15 Velero with v1.8 AWS plugin cannot support multiple BSLs under AWS IRSA environment.

Open blackpiglet opened this issue 4 months ago • 6 comments

What steps did you take and what happened:

  • Create an EKS cluster, and deploy a MinIO as the backend of the additional BSL.
  • Install Velero with the AWS S3 as the backend of the default BSL and use the IRSA as the authorization method. The Velero version is the main(v1.15) and the Velero AWS plugin version is v1.8.2
  • Create an additional BSL with the MinIO as the backend, and create a secret as the authorization for the additional BSL.
  • The additional BSL cannot work. The error is
"BackupStorageLocation \"add-bsl\" is unavailable: rpc error: code = Unknown desc = WebIdentityErr: failed to retrieve credentials\ncaused by: RequestError: send request failed\ncaused by: Post \"https://sts.minio.amazonaws.com/\": dial tcp: lookup sts.minio.amazonaws.com on 10.100.0.10:53: no such host",

What did you expect to happen: The additional BSL should work.

The following information will help us better understand what's going on: The reason is that Velero overrides the secret for the additional BSL when IRSA is enabled. The v1.9 and main Velero AWS plugin have this PR to fix the issue: https://github.com/vmware-tanzu/velero-plugin-for-aws/pull/191 But the v1.8 doesn't bump the AWS SDK version to v2, so we cannot simply cherry-pick the PR to fix this issue.

If you are using velero v1.7.0+:
Please use velero debug --backup <backupname> --restore <restorename> to generate the support bundle, and attach to this issue, more options please refer to velero debug --help

If you are using earlier versions:
Please provide the output of the following commands (Pasting long output into a GitHub gist or other pastebin is fine.)

  • kubectl logs deployment/velero -n velero
  • velero backup describe <backupname> or kubectl get backup/<backupname> -n velero -o yaml
  • velero backup logs <backupname>
  • velero restore describe <restorename> or kubectl get restore/<restorename> -n velero -o yaml
  • velero restore logs <restorename>

Anything else you would like to add:

Environment:

  • Velero version (use velero version):
  • Velero features (use velero client config get features):
  • Kubernetes version (use kubectl version):
  • Kubernetes installer & version:
  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release):

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

  • :+1: for "I would like to see this bug fixed as soon as possible"
  • :-1: for "There are more important bugs to focus on right now"

blackpiglet avatar Oct 18 '24 03:10 blackpiglet