Presigned URLs

[metadaddy]

trafficstars

First stab at presigned URLs.

Invoked just like a regular request, but include SignQuery: true and, optionally, additional parameters for aws4. Method returns a string containing the presigned URL rather than the response to the query.

For example, generate presigned URL with a lifetime of one hour:

url = await aws.S3.GetObject({
  Bucket: 'my-bucket',
  Key: 'aws-lite.txt',
  SignQuery: true,
  PresignedUrlExpires: 3600,
})

// https://my-bucket.s3.us-east-1.amazonaws.com/aws-lite.txt?X-Amz-Expires=3600&X-Amz-Date=20240201T213629Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAXSRWQ36HL4DYYSGB%2F20240201%2Fus-west-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=afea02734775bd678875705abc1999ba7811fe5390d4ba4a295ea1261aed2c7f

Rationale for this approach: generating a presigned URL needs all the same processing as a request until the aws4.sign() call. The path of least resistance is to execute that same code path, then bail out after aws4.sign() with the resulting URL, rather than making the request.

I'm very open to suggestions of a better developer experience. Now that this is working, it should be straightforward to refactor to a different approach.

[ryanblock]

Thank you for taking a swing at this! I've done a bit of reading, but I'm going to need to do a little more to have a proper opinion here. A couple of initial thoughts come to mind, in no particular order:

• I really dig the single param concept, much cleaner than how AWS does this
• Related: how AWS does this is totally wild and pretty far from how we should, imo (see SDK docs, example)
• Manually generating AWS v4 signatures is something we have done in the past within the plugin context, see: https://github.com/architect/aws-lite/blob/main/plugins/s3/src/put-object.mjs#L126-L132
• I'm initially inclined (but by no means decided) to avoid letting this use case leak into the client factory / request internals; are there other potential use cases for the signQuery param? (I get that it may be hard to say right now.)
• I'm thinking perhaps this may be best serviced by a separate method or an entirely separate utility plugin

Would love to hear more of your thoughts on this!

[andybee]

I guess the benefit of being here is that any future work on the params will automatically work both for the SDK-performed requests and for the generated URL? e.g. it keeps a single source of truth? (granted, it shouldn't ever change, at least, not from an AWS point of view)

My concern with this approach is how we try and follow the same path for the use case discussed in Discord, e.g. returning the URL + all the key/values you need to hide in an HTML form when doing a web-based upload. Should this be an extra boolean ReturnFormParams etc. that gives you a JSON object response:

{
  "action": <url>
  "fields": {
    "key": <key>
    "success_action_redirect": <success_action_redirect>
    "AWSAccessKeyID": <AWSAccessKeyID>
    "Policy": <Policy>
    "Signature": <Signature>
  }
}

Now I feel we're really moving away from the original design/intention of the "lite" wrapper around the SDK.

[metadaddy]

@ryanblock It should be possible to avoid polluting client factory / request by refactoring somewhat. As I mentioned above, I took the path of least resistance, so I could at least get something working as a start. An alternative approach could have the presigned URL code and client factory code both calling some common method to "prepare" the request, with the presigned URL stuff constructing the URL, and the client factory code calling the request logic.

The way that the AWS SDK does it is understandable when you consider that, in principle, you can presign any AWS request, so you have to be able to specify any of the requests, along with its parameters, and pass that to the presigner. No desirable, but understandable 🙂

@andybee Yep - I think the form post logic should definitely be in its own plugin, rather than the core. There is so much shared code between the regular request path and creating a presigned URL that the basic operation lives in core, I think.

[andybee]

@metadaddy I'd be happy to have a crack at the "HTML form generation" plugin. @brianleroux has done a lot of the initial work already in an Enhance component version, the only thing I'd realistically change from that is just returning a plain ol' object so can manipulate it to a form as you see fit.

I'm thinking this would be best suited as a wrapper function, as in it'd taken the required arguments, then call was-lite to get a presigned URL (in whatever way the above ultimately lands), but also still hold the necessary params to build the form contents and return...?

[brianleroux]

super apprec that @andybee / i'm def distracted with lots of other projects !

[metadaddy]

@andybee Great! I'll try to find some time to refactor the code in this branch so that it doesn't just take a shortcut from the request flow. The overall developer experience seems to have been well received, so it should remain pretty much unchanged, so you could start building on what's here now.

[jchris]

this might be useful for porting the Fireproof uploader to arc.codes

[riderx]

Can we do anything to help on that?

[andybee]

Unfortunately on my end the opportunity I had to work on this as part of another project disappeared shortly after proposing a solution and offering to assist, and I've not had chance to pick this up again unfortunately. If someone else is interested, go for it.