Amazon S3 is deprecating path-style request URIs

[rmetzler]

trafficstars

Output of restic version

restic 0.9.5 compiled with go1.12.4 on darwin/amd64

What should restic do differently? Which functionality do you think we should add?

AWS

AWS is going to change their S3 URI API and stop accepting path-style request on September 30th, 2020.

https://forums.aws.amazon.com/ann.jspa?annID=6776

Amazon S3 currently supports two request URI styles in all regions: path-style (also known as V1) that includes bucket name in the path of the URI (example: //s3.amazonaws.com/<bucketname>/key), and virtual-hosted style (also known as V2) which uses the bucket name as part of the domain name (example: //<bucketname>.s3.amazonaws.com/key). In our effort to continuously improve customer experience, the path-style naming convention is being retired in favor of virtual-hosted style request format. Customers should update their applications to use the virtual-hosted style request format when making S3 API requests before September 30th, 2020 to avoid any service disruptions. Customers using the AWS SDK can upgrade to the most recent version of the SDK to ensure their applications are using the virtual-hosted style request format.

Virtual-hosted style requests are supported for all S3 endpoints in all AWS regions. S3 will stop accepting requests made using the path-style request format in all regions starting September 30th, 2020. Any requests using the path-style request format made after this time will fail.

restic docs

Currently restic docs use the path-style (v1) URI for the repo. I'm not sure if virtual-hosted style (v2) is possible to use. At least it's not documented.

What are you trying to do?

1. It should be possible to use the Virtual-hosted style request in restic.
2. The usage should be documented.
3. A clear upgrade path and (depending on implementation) a warning for users should be documented.

Did restic help you or made you happy in any way?

restic is a great piece of software and In love to use it. I recommend it to everyone who does devops or sysadmin stuff :)

[fd0]

Huh, interesting, thanks for the hint! We're using minio-go for accessing S3, maybe @harshavardhana knows what we need to do here (if anything)? I'm guessing that this already works, but we should amend the documentation.

@rmetzler can you maybe test if restic works with the v2 URLs?

[harshavardhana]

@fd0 minio-go already supports virtual style, look at the minio.Options{BucketLookup: "dns"} so you can support it.

[fd0]

I'll check it out, thanks a lot!

[rmetzler]

# export RESTIC_REPOSITORY="s3:s3.amazonaws.com/${AWS_S3_BUCKET}"
# restic snapshots
password is correct
ID        Date                 Host               Tags        Directory
----------------------------------------------------------------------
[...]
----------------------------------------------------------------------
11 snapshots

# export RESTIC_REPOSITORY="s3:${AWS_S3_BUCKET}.s3.amazonaws.com/"
# restic snapshots
Fatal: unable to open repo at s3:my-restic-test-bucket.s3.amazonaws.com/: minio.NewWithCredentials: Amazon S3 endpoint should be 's3.amazonaws.com'.

[rmetzler]

I don't think this is solved in minio-go and I created an issue there: minio/minio-go#1102

[harshavardhana]

@rmetzler - minio-go already supports virtual DNS style and if it's already S3 endpoint it is automatically using virtual DNS style.

Meaning even if you provide

RESTIC_REPOSITORY="s3:s3.amazonaws.com/${AWS_S3_BUCKET}"

Internally it's using virtual DNS style for S3.

[rmetzler]

@harshavardhana can you point me to the line in code?

[rmetzler]

I started tcpdump and indeed, I can see DNS lookups for my test bucket when I use the path-style bucket. So the claim, that minio-go internally uses virtual-hosted style URIs is reasonable.

My biggest fear - something would break and people would loose data - seems to be unfounded. I'm glad it's this way.

So what I'm asking is: could restic allow the virtual-host style bucket URI as RESTIC_REPOSITORY and update the documentation to match it? I'm pretty sure AWS will change the documentation on their website and it would be easier if users don't have to change the URI. It would also prevent questions like mine.

[rawtaz]

If anyone is up for a PR on this one, that'd be great! ❤️

[MichaelEischer]

The DNS style S3 paths usually also include the region of a bucket. For #2250 only the non region-specific endpoint address worked.

[smessmer]

I'm now unable to use the s3 backend in restic 0.15.1 and restic 0.16.0 with either style.

Using path style:

$ restic cat config --repo s3:s3.amazonaws.com/[bucket.name]/restic-test/subfolder
Fatal: unable to open config file: Stat: 301 Moved Permanently
Is there a repository at the following location?
s3:s3.amazonaws.com/[bucket.name]/restic-test/subfolder

(checking the URL manually, I see that AWS returns a HTTP 301 redirecting to the virtual-hosted style url)

And using virtual hosted style:

$ restic cat config --repo s3:[bucket.name].s3.amazonaws.com/restic-test/subfolder
Fatal: unable to open repository at s3:[bucket.name].s3.amazonaws.com/restic-test/subfolder: minio.New: Amazon S3 endpoint should be 's3.amazonaws.com'.

[smessmer]

nvm, seems this happened because the AWS_ACCESS_KEY was missing from my environment. After setting the key, the path style works again.