How to create pre-signed URL with metadata in URL query string?

[innix]

Confirm by changing [ ] to [x] below:

• [x] I've gone though the API reference
• [x] I've checked AWS Forums and StackOverflow for answers

---

How do I create an S3 pre-signed URL with metadata in the URL query string, like in the V1 SDK?

According to docs, if I want to attach metadata to an object when it gets upload to S3 via a pre-signed URL, I need to specify the metadata when I create the pre-signed URL and then the client has to send the exact same metadata as headers prefixed with x-amz-meta-. In V1 of the SDK, there was a way to add the metadata to the query string of the pre-signed URL so that the client didn't need to set any headers; they could just upload their object to S3. Here's a code snippet of how it was done in V1 SDK:

package main

import (
	"fmt"
	"os"
	"time"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3"
)

func main() {
	sess := session.Must(session.NewSessionWithOptions(session.Options{
		SharedConfigState: session.SharedConfigEnable,
	}))
	svc := s3.New(sess)

	bucket := "todo" // TODO: replace with valid bucket name
	key := fmt.Sprintf("github_issue_v1_%d", time.Now().Unix())
	req, _ := svc.PutObjectRequest(&s3.PutObjectInput{
		Bucket: aws.String(bucket),
		Key:    aws.String(key),
	})

	// Add some user metadata.
	query := req.HTTPRequest.URL.Query()
	query.Add("x-amz-meta-level", "beginner")
	req.HTTPRequest.URL.RawQuery = query.Encode()

	// Create presigned URL.
	u, err := req.Presign(time.Minute * 30)
	if err != nil {
		fmt.Printf("Failed to create presigned URL for PUT request: %v\n", err)
		os.Exit(1)
	}
	fmt.Println("Presigned URL created:", u)
}

I tried to do something similar with the V2 SDK but I get a SignatureDoesNotMatch error when uploading my object. Here's the V2 sample code:

package main

import (
	"context"
	"fmt"
	"net/url"
	"os"
	"time"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/s3"
)

func main() {
	cfg, err := config.LoadDefaultConfig(context.Background())
	if err != nil {
		fmt.Printf("Could not load default AWS config: %v\n", err)
		os.Exit(1)
	}
	svc := s3.NewPresignClient(s3.NewFromConfig(cfg))

	bucket := "todo" // TODO: replace with valid bucket name
	key := fmt.Sprintf("github_issue_v2_%d", time.Now().Unix())
	input := &s3.PutObjectInput{
		Bucket: aws.String(bucket),
		Key:    aws.String(key),
	}

	preReq, err := svc.PresignPutObject(
		context.Background(),
		input,
		s3.WithPresignExpires(time.Minute*30),
	)
	if err != nil {
		fmt.Printf("Could not generate pre-signed URL: %v\n", err)
		os.Exit(1)
	}

	u, err := url.Parse(preReq.URL)
	if err != nil {
		fmt.Printf("Could not parse the pre-signed URL: %v\n", err)
		os.Exit(1)
	}

	// Add some user metadata.
	query := u.Query()
	query.Add("x-amz-meta-level", "beginner")
	u.RawQuery = query.Encode()

	fmt.Println("Presigned URL created:", u)
}

But as I said, this doesn't work and I get an SignatureDoesNotMatch error when trying to PUT upload my object.

My question: How do I create an S3 pre-signed URL with metadata in the URL query string, like in the V1 SDK?

[KaibaLopez]

Hi @innix , So the way to add Metadata to the Presigned URL was actually simplified for V2, all you have to do is add it to the PutObjectInput before generating the URL like:

input := &s3.PutObjectInput{
        Bucket: aws.String(bucket),
	Key:    aws.String(key),
        Metadata: map[string]string{
		"level": "beginner",
	},
}

Let me know if you still have an error but that should work just fine

[innix]

Hi @KaibaLopez , thanks for looking into this issue for me.

Unfortunately your solution doesn't solve the issue because it would still require the client to send the same metadata as an HTTP header (x-amz-meta-level: beginner) when making the PUT request, which if you see my original comment I am trying to avoid:

In V1 of the SDK, there was a way to add the metadata to the query string of the pre-signed URL so that the client didn't need to set any headers; they could just upload their object to S3.

(Sorry, I didn't describe the issue very clearly in OP).

[KaibaLopez]

Hi @innix , Sorry for the lack of response on this.,truth is it's kind of surprising that this worked for v1... it's kind of unsupported behavior though, we could look into "officially" adding it but as of now I don't really have an answer for you, sorry.

[clausMeko]

I would also really like to have it supported officially as I have the same use case as @innix. For now would I need to use v1.

[chan58]

Sorry to raise this issue from the dead, but could we at least have clarification that adding a map of Metadata as described above is either not supported, or is acknowledged as a bug? It clearly does not work. Adding a map as @KaibaLopez describes above, does add the map key to the URL params, as x-amz-meta-level= but with no value.

[RanVaknin]

@innix , @chan58 , @clausMeko ,

S3's new requirements state that for presigned URLs allowlisted headers will be hoisted into the query param.

If you are curious about which headers are allowlisted, you can see them here.

Let me know if this answers any questions. Thanks, Ran~

[github-actions[bot]]

This issue has not received a response in 1 week. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.

[lucasoares]

Hello. I am resurrecting this thread again.

In the previous version of the SDK we could do what the author said, so the client would not have to know all headers to be set to upload the file. Are you saying that this will never be possible again with v2?

[TakumiHaruta]

@RanVaknin @jasdel

Hi, I have a comment regarding the specifications of PutObjectInput.

X-Amz-Meta- wouldn't be hoisted into the query param as you shared the link, but PutObjectInput contains both key and values as Metadata attribute, which would be a bit confusing for developers.

Since the code was written 4 years ago, I'm just wondering Metadata is just outdated params for PutObjectInput