imds.GetInstanceIdentityDocument doesn't honor retry options

[otterley]

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• [x] I've gone though the API reference
• [x] I've checked AWS Forums and StackOverflow for answers
• [x] I've searched for previous similar issues and didn't find any solution

Describe the bug imds.GetInstanceIdentityDocument() doesn't honor retry options, regardless of how they're specified.

Version of AWS SDK for Go? Example: v1.29.22

• get SDK version by printing the output of aws.SDKVersion in your code after importing "github.com/aws/aws-sdk-go-v2/aws"

Version of Go (go version)? 1.16.3

To Reproduce (observed behavior)

Write some code like the following and run it on an EC2 instance after running sudo iptables -A OUTPUT -d 169.254.169.254 -j DROP:

i := imds.NewFromConfig(awscfg, func(options *imds.Options) {
	options.Retryer = retry.AddWithMaxAttempts(options.Retryer, math.MaxInt64)
})
instanceIdentity, err := i.GetInstanceIdentityDocument(context.TODO(), &imds.GetInstanceIdentityDocumentInput{})
if err != nil {
    fmt.Fatal(err)
}

Instead of retrying forever, it fails with:

operation error ec2imds: GetInstanceIdentityDocument, request canceled, context deadline exceeded

Expected behavior I expect it to retry forever.

[skmcgrail]

The SDK will not retry requests if the provided context has been canceled. The purpose of context in the Go language is to provide callers the ability to cancel requests or set timeouts or deadlines by which a request must be finished by. If the context is in such a state where it has been cancelled the SDK must honor that state and not attempt to continue any further attempts.

Can you confirm whether the context you are passing into the call is wrapped with a timeout, deadline, or is cancelable?

See: https://blog.golang.org/context https://golang.org/pkg/context/

[otterley]

Hi @skmcgrail - I should note that the context provided when I built the aws.Config object was context.TODO(). I didn't add a deadline or anything like that. So if something added a deadline, it wasn't me!

	ctx := context.TODO()

	awscfg, err := config.LoadDefaultConfig(ctx)

Also note the context.TODO() that I passed to i.GetInstanceIdentityDocument().

[canassa]

I am also getting this error when I tried to enable the retry system:

cfg, err := config.LoadDefaultConfig(
	context.TODO(),
	config.WithRetryer(func() aws.Retryer {
		return retry.NewStandard(func(o *retry.StandardOptions) {
			o.MaxAttempts = 10
			o.Backoff = retry.BackoffDelayerFunc(func(attempt int, err error) (time.Duration, error) {
				return time.Duration(attempt) * 2 * time.Second, nil
			})
		})
	}))

In my case, it failed with

failed to retrieve AWS credentials. error: [operation error STS: AssumeRole, failed to sign request: failed to retrieve credentials: no EC2 IMDS role found, operation error ec2imds: GetMetadata, canceled, context deadline exceeded]

Does anybody know a workaround for this issue?

[github-actions[bot]]

We have noticed this issue has not received attention in 1 year. We will close this issue for now. If you think this is in error, please feel free to comment and reopen the issue.