k8up icon indicating copy to clipboard operation
k8up copied to clipboard

Allow setting subpaths for S3 backend

Open cwrau opened this issue 3 years ago • 7 comments

Summary

As user I want to specify subpaths for the S3 backend So that I can have a nice structure in my S3

Context

I want the backups to be grouped by namespace, helmRelease and type, resulting in the following;

HelmRelase customer-prod in namespace customer backs up PVC config. I want the backups to land in the following location;

s3://k8s-backup/customer/customer-prod/persistentvolumeclaim/config/

Out of Scope

No response

Further links

  • restic s3 config: https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#amazon-s3

Acceptance Criteria

  • Given a Backup resource's backend.s3, I want to be able to specify something like a prefix

Implementation Ideas

An (optional) prefix field on the S3Spec

cwrau avatar Mar 01 '22 14:03 cwrau

Hello, also need this feature, is this on going ? Thanks

s4ndalHat avatar Oct 21 '24 13:10 s4ndalHat

Hi everyone

You can use subpaths S3 like this:

apiVersion: k8up.io/v1
kind: Backup
metadata:
  name: test
spec:
  backend:
    repoPasswordSecretRef:
      key: password
      name: k8up-repository-password
    s3:
      accessKeyIDSecretRef:
        key: AWS_ACCESS_KEY_ID
        name: backup-bucket-credentials
      bucket: mybucket/path/to/subfolder
      endpoint: http://minio-server.minio.svc:9000
      secretAccessKeySecretRef:
        key: AWS_SECRET_ACCESS_KEY
        name: backup-bucket-credentials

The bucket field is simply concatenated to build the RESTIC_REPOSITORY string as documented here: https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#amazon-s3.

So you can simply add the subfolders via slashes.

Kidswiss avatar Oct 21 '24 14:10 Kidswiss

@Kidswiss thanks! Its getting configured, thank you..

However, I'm facing another issue, I'm getting the following error : image

s4ndalHat avatar Oct 21 '24 14:10 s4ndalHat

@s4ndalHat the ciphertext verification failed error can have two causes I'm aware of:

  • the repository password doesn't match anymore -> maybe the secret was changed by mistake
  • something is corrupt with the repository, in that case you might either want to delete the bucket and then take a fresh backup or get more help in the restic forum

Kidswiss avatar Oct 21 '24 14:10 Kidswiss

This is where I'm lost, it's my first backup and I'm always trying into a new repository and bucket... always the same issue :')

s4ndalHat avatar Oct 21 '24 14:10 s4ndalHat

Can you please open a new issue and add you manifests?

Kidswiss avatar Oct 21 '24 14:10 Kidswiss

My bad, I reverted the endpoint url and bucket typo, I'm still getting some minor error but the job is succeded.

Instead typing: url: myurl and bucket: /path/to/myprefix, I typed url: myurl/bucket and bucket: myprefix

Sorry, but its getting working (next I'll be testing the restore process)

s4ndalHat avatar Oct 21 '24 14:10 s4ndalHat