Outdated code example for R2 with aws-sdk-go-v2

[urgn]

Which Cloudflare product does this pertain to?

R2

Existing documentation URL(s)

The code from this example does not really work. It produces an error:

operation error S3: ListObjectsV2, resolve auth scheme: resolve endpoint: endpoint rule error, Invalid region: region was not a valid DNS name.

What changes are you suggesting?

Update the code listing with working example.

Additional information

No response

[ITheCorgi]

Its just requieres adding WithRegion setting:

        var bucketName = "sdk-example"
	var accountId = "<accountid>"
	var accessKeyId = "<access_key_id>"
	var accessKeySecret = "<access_key_secret>"

	r2Resolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
		return aws.Endpoint{
			URL: fmt.Sprintf("https://%s.r2.cloudflarestorage.com", accountId),
		}, nil
	})

	cfg, err := config.LoadDefaultConfig(context.TODO(),
		config.WithEndpointResolverWithOptions(r2Resolver),
		config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(accessKeyId, accessKeySecret, "")),
                config.WithRegion("auto"),
	)
	if err != nil {
		log.Fatal(err)
	}

	client := s3.NewFromConfig(cfg)

[rhaist]

Both aws.EndpointResolverWithOptionsFunc and aws. Endpoint are now deprecated by AWS and the code should be updated.

It seems like AWS makes it harder to use non-AWS services with their SDK and it could be better to use another S3 client that is compatible with R2 for the example section.

Reference: https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/endpoints/

[plu]

	cfg, err := config.LoadDefaultConfig(context.TODO(),
		config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(accessKeyId, accessKeySecret, "")),
		config.WithRegion("auto"),
	)
	if err != nil {
		log.Fatal(err)
	}

	client := s3.NewFromConfig(cfg, func(o *s3.Options) {
		o.BaseEndpoint = aws.String("https://myaccountid.eu.r2.cloudflarestorage.com")
	})

	listObjectsOutput, err := client.ListObjectsV2(context.TODO(), &s3.ListObjectsV2Input{
		Bucket: &bucketName,
	})
	if err != nil {
		log.Fatal(err)
	}

Also stumbled upon the deprecation warnings, then found this solution ☝️.

// Edit

Sorry, to give some more context: AWS has deprecated the global endpoint configuration in favour of a per-service configuration. So only V1 of the endpoint resolver API is deprecated, but there's a successor V2. More details here: https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/endpoints/

[zaydek]

This is @plu's code that worked for me in case anyone is struggling:

package main

import (
  "context"
  "fmt"
  "log"

  "github.com/aws/aws-sdk-go-v2/aws"
  "github.com/aws/aws-sdk-go-v2/config"
  "github.com/aws/aws-sdk-go-v2/credentials"
  "github.com/aws/aws-sdk-go-v2/service/s3"
  "github.com/sirupsen/logrus"
)

// https://github.com/cloudflare/cloudflare-docs/issues/12043#issuecomment-2198148921
func ConnectToS3(endpoint, accessKey, secretAccessKey string) (*s3.Client, error) {
  cfg, err := config.LoadDefaultConfig(context.TODO(),
    config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(
      accessKey,
      secretAccessKey,
      "",
    )),
    config.WithRegion("auto"),
  )
  if err != nil {
    return nil, fmt.Errorf("failed to load config, %w", err)
  }
  client := s3.NewFromConfig(cfg, func(o *s3.Options) {
    o.BaseEndpoint = aws.String(endpoint)
  })
  return client, nil
}

func main() {
  logrus.Info("connecting to S3")
  r2, err := ConnectToS3(
    fmt.Sprintf("https://%s.r2.cloudflarestorage.com", CLOUDFLARE_R2_ACCOUNT_ID),
    CLOUDFLARE_R2_ACCESS_KEY,
    CLOUDFLARE_R2_SECRET_KEY,
  )
  if err != nil {
    logrus.Fatalf("failed to connect to S3: %v", err)
  }
  logrus.Info("connected to S3")

[KimJ15]

@KianNH something you can help with?

[harshil1712]

Hey all, thank you for reporting this. I tested out the example and it works fine for me. I think it was updated sometime ago, but we missed out on updating this issue.

Closing this issue for now. But if you still face an issue, feel free to open a new one.

Thank you again!