s3-resource
s3-resource copied to clipboard
concourse
Generating signed URLs should be done via `put`
Related to https://github.com/concourse/concourse/issues/622 and https://github.com/concourse/s3-resource/issues/47
private: true on get is confusingly named and breaks resource semantics, because the URL it provides has an expiration value, and is different every time it's generated, making get nondeterministic.
Generating the URL should instead be generated by a put, guaranteeing that it'll be a new URL each time. This could look like:
- get: my-bucket-thing
- put: my-bucket-thing
params: {sign: my-bucket-thing, expires_in: 24h}
The put would yield a version like {path: foo, signed_at: <timestamp>} (or version_id if using versioned_file). The get resulting from the put would generate the signed URL. This fixes the hole because the get will be generating a URL valid for that particular timestamp (because it's the start of the expiration countdown), and the timestamp will be different every time.
Hi there!
We use Pivotal Tracker to provide visibility into what our team is working on. A story for this issue has been automatically created.
The current status is as follows:
- [ ] #131941803 Generating signed URLs should be done via
put
This comment, as well as the labels on the issue, will be automatically updated as the status in Tracker changes.
+1 the expires_in portion would also benefit our team as we are running into the 24 hour expiration on signed URLs when re-triggering jobs that use a cached resource that has already passed the 24 hour expiration window.
That seems so... static. From my point of view the signature on the S3 url has nothing to do with the resource the url represents. I would rather see the cached resource expire and the signed url regenerated after a configurable time ( on resource get ).
The cached resource expiring would help us in one sense for immediate usages of the signed URLs.
However, many times we want to be able to download using that signed URL more than a day later for historical purposes, so we would need the expires_in portion.
@amilkh I think a combination of both would be desirable. But I don't know if cache expiration is even possible currently.