Restrictive s3 options- missing s3 region

[weaktyper]

Description of the issue:

The k8up application does not provide options to set the region.

I think this makes it only compatible with

• internal minio
• ~s3 on region us-east-1 only~ edit: it will work for s3 regions but not other vendors at this time. ref github.com/minio/minio-go/pull/1188

Error message:

Connection to S3 endpoint not possible: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'xx-yyy'

k8up version: docker.io/vshn/k8up:v0.1.6

Proposed Solution:

Allow region as another env variable

[weaktyper]

So, I have researched this a bit more and discovered a few things:

1. The problem originally resided in restic/restic, however, this PR fixes it and it shipped in the latest release 0.9.6

2. vshn/wrestic does not use restic/restic. It in fact uses a forked branch of restic/restic in vshn/restic called 2319-dump-dir-tar

I wonder is it feasible for vshn to use a release of restic/restic. It keeps the chain of dependencies a bit more manageable and updatable (if it's at all possible).

Update:

I can see the motivation for this change to a fork is described here.

Further Update:

Incorporating the latest changes from release 0.9.6 of restic/restic makes no difference. I suppose this is because vshn/restic seems to use the go client from https://github.com/minio/minio-go rather than directly using the s3 client of restic unless I'm mistaken. Unfortunately, I don't think the minio-go client supports setting the region as it stands.

[Kidswiss]

Hi @weaktyper

Thanks for the feedback!

We're currently avoiding this issue with setting the region by using the region specific AWS url, for example s3 austrialia: https://s3.ap-southeast-2.amazonaws.com

1. Wasn't aware of that change, thanks. I'll have a look at porting 2319-dump-dir-tar to 0.9.6 so we can implement that for k8up/wrestic.

2. You answered that yourself :)

Concering the minio-go client: restic also uses that library, as seen here: https://github.com/restic/restic/pull/2484/files#diff-ff0adb00c85ede1c7bd78ea0561c47d8L69. It may be that it uses a newer version than our 2319-dump-dir-tar fork though, which could cause the issues your experiencing.

EDIT: having said that, wrestic (the container that runs restic), also does some minio-go calls before running restic itself, so it has to be added there as well.

[weaktyper]

Hi @Kidswiss,

Thanks for the reply.

We're currently avoiding this issue with setting the region by using the region specific AWS url, for example s3 austrialia: https://s3.ap-southeast-2.amazonaws.com

Yes, I think you are right about that for amazon urls but unfortunately for scaleway urls or other s3 compatible apis that require the region specifically, I think the minio client is incompatible. It's noted at this issue - github.com/minio/minio-go/pull/1188

In terms of the restic client, I tried setting AWS_DEFAULT_REGION (https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#amazon-s3) inside the wrestic docker container directly but it didn't work. Still got us-east-1.

Not sure there is an easy fix for this other than just using amazon s3 altogether.

[Kidswiss]

Hi @weaktyper

Interesting.

Maybe we need to rebase our fork to 0.9.6 and pull the newest minio-go libraries, as your linked pull request is merged.

k8up is tested and used against following S3 implementations:

• minio
• aws s3
• ceph radosgateway

I'll see if I can carve out some time next week to analyse this. Thanks for bringing it to my attention.

[cdchris12]

We just ran into this issue as well. It seems Google Cloud Storage only has a single endpoint which can be used to access their S3 compatible API.

It seems like specifying the region to be used is natively supported by Restic via the use of the AWS_DEFAULT_REGION environment variable, as shown here: https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#amazon-s3

I'd be happy to test this against our Google Cloud Storage setup if you want/need some testing to be done there. :)