An example using a Dead Letter Queue

[simonireilly]

File: docs/reference/pkg/aws/sqs/queue.md

When creating an SQS queue with a Dead Letter Queue the documentation refers to a variable for an existing queue in order to access the ARN:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const queue = new aws.sqs.Queue("queue", {
    delaySeconds: 90,
    maxMessageSize: 2048,
    messageRetentionSeconds: 86400,
    receiveWaitTimeSeconds: 10,
    redrivePolicy: JSON.stringify({
        deadLetterTargetArn: aws_sqs_queue.queue_deadletter.arn,
        maxReceiveCount: 4,
    }),
    tags: {
        Environment: "production",
    },
});

This is a little confusing as the variables aren't included in the example.

Attempts

I wanted to set up a Dead Letter queue for my own SQS queues, and made the following attempts:

import * as aws from '@pulumi/aws'

const ingressQueueDead = new aws.sqs.Queue('ingressQueueDead', {
})

const ingressQueue = new aws.sqs.Queue('ingressQueue', {
  redrivePolicy: JSON.stringify({
    deadLetterTargetArn: ingressQueueDead.arn,
    maxReceiveCount: 2
  })
})

$ pulumi up --yes
Previewing update (dev):
     Type                 Name                    Plan       
     pulumi:pulumi:Stack  lambda-sqs-trigger-dev             
 +   └─ aws:sqs:Queue     ingressQueue            create     
 
Resources:
    + 1 to create
    2 unchanged

Updating (dev):
     Type                 Name                    Status                  Info
     pulumi:pulumi:Stack  lambda-sqs-trigger-dev  **failed**              1 error
 +   └─ aws:sqs:Queue     ingressQueue            **creating failed**     1 error
 
Diagnostics:
  aws:sqs:Queue (ingressQueue):
    error: Error creating SQS queue: InvalidParameterValue: Value {"deadLetterTargetArn":"Calling [toJSON] on an [Output<T>] is not supported.\n\nTo get the value of an Output as a JSON value or JSON string consider either:\n    1: o.apply(v => v.toJSON())\n    2: o.apply(v => JSON.stringify(v))\n\nSee https://pulumi.io/help/outputs for more details.\nThis function may throw in a future version of @pulumi/pulumi.","maxReceiveCount":2} for parameter RedrivePolicy is invalid. Reason: Invalid value for deadLetterTargetArn.
        status code: 400, request id: dfd8181c-cda7-51e4-97a3-4f6b5a12a664
 
  pulumi:pulumi:Stack (lambda-sqs-trigger-dev):
    error: update failed
 
Resources:
    2 unchanged

Duration: 5s

Based on the feedback error I tried:

import * as aws from '@pulumi/aws'

const ingressQueueDead = new aws.sqs.Queue('ingressQueueDead', {
})

const ingressQueue = new aws.sqs.Queue('ingressQueue', {
  redrivePolicy: JSON.stringify({
    deadLetterTargetArn: ingressQueueDead.arn.apply(v => JSON.stringify(v)),
    maxReceiveCount: 2
  })
})

$ pulumi up --yes
Previewing update (dev):
     Type                 Name                    Plan       
     pulumi:pulumi:Stack  lambda-sqs-trigger-dev             
 +   └─ aws:sqs:Queue     ingressQueue            create     
 
Resources:
    + 1 to create
    2 unchanged

Updating (dev):
     Type                 Name                    Status                  Info
     pulumi:pulumi:Stack  lambda-sqs-trigger-dev  **failed**              1 error
 +   └─ aws:sqs:Queue     ingressQueue            **creating failed**     1 error
 
Diagnostics:
  pulumi:pulumi:Stack (lambda-sqs-trigger-dev):
    error: update failed
 
  aws:sqs:Queue (ingressQueue):
    error: Error creating SQS queue: InvalidParameterValue: Value {"deadLetterTargetArn":"Calling [toJSON] on an [Output<T>] is not supported.\n\nTo get the value of an Output as a JSON value or JSON string consider either:\n    1: o.apply(v => v.toJSON())\n    2: o.apply(v => JSON.stringify(v))\n\nSee https://pulumi.io/help/outputs for more details.\nThis function may throw in a future version of @pulumi/pulumi.","maxReceiveCount":2} for parameter RedrivePolicy is invalid. Reason: Invalid value for deadLetterTargetArn.
        status code: 400, request id: 3f35e85c-b56d-542f-b523-bf2b2d6f0703
 
Resources:
    2 unchanged

Duration: 5s

Permalink: https://app.pulumi.com/simonireilly/lambda-sqs-trigger/dev/updates/28

This gives the same error, so finally I thought this was an issue with the dependency tree so I tried:

import * as aws from '@pulumi/aws'

const ingressQueueDead = new aws.sqs.Queue('ingressQueueDead', {
})

const ingressQueue = new aws.sqs.Queue('ingressQueue', {
  redrivePolicy: JSON.stringify({
    deadLetterTargetArn: ingressQueueDead.arn.apply(v => JSON.stringify(v)),
    maxReceiveCount: 2
  })
}, { dependsOn: [ingressQueueDead] })

$ pulumi up --yes
Previewing update (dev):
     Type                 Name                    Plan       
     pulumi:pulumi:Stack  lambda-sqs-trigger-dev             
 +   └─ aws:sqs:Queue     ingressQueue            create     
 
Resources:
    + 1 to create
    2 unchanged

Updating (dev):
     Type                 Name                    Status                  Info
     pulumi:pulumi:Stack  lambda-sqs-trigger-dev  **failed**              1 error
 +   └─ aws:sqs:Queue     ingressQueue            **creating failed**     1 error
 
Diagnostics:
  aws:sqs:Queue (ingressQueue):
    error: Error creating SQS queue: InvalidParameterValue: Value {"deadLetterTargetArn":"Calling [toJSON] on an [Output<T>] is not supported.\n\nTo get the value of an Output as a JSON value or JSON string consider either:\n    1: o.apply(v => v.toJSON())\n    2: o.apply(v => JSON.stringify(v))\n\nSee https://pulumi.io/help/outputs for more details.\nThis function may throw in a future version of @pulumi/pulumi.","maxReceiveCount":2} for parameter RedrivePolicy is invalid. Reason: Invalid value for deadLetterTargetArn.
        status code: 400, request id: 182e3795-d0f4-5715-bba5-60e24826bd62
 
  pulumi:pulumi:Stack (lambda-sqs-trigger-dev):
    error: update failed
 
Resources:
    2 unchanged

Duration: 5s

Permalink: https://app.pulumi.com/simonireilly/lambda-sqs-trigger/dev/updates/29

I understand that the fundamental issue is that I need to resolve the ARN of my queue as an output from the creation of the dead queue, but I can't seem to get this to work. In cloud formation, I would do something like Fn::GetAtt Arn or use a Ref.

What actually Worked

The following worked:

import * as aws from '@pulumi/aws'

const ingressQueueDead = new aws.sqs.Queue('ingressQueueDead', {
})

const ingressQueue = new aws.sqs.Queue('ingressQueue', {
  redrivePolicy: ingressQueueDead.arn.apply(arn => JSON.stringify({
    deadLetterTargetArn: arn,
    maxReceiveCount: 2
  }))
})

But it took a long time for me to figure this out, and I think it might be a common use case, so the documentation might benefit from including it.

Thanks

[vikshab]

I think this would work too:

import * as aws from '@pulumi/aws'

const ingressQueueDead = new aws.sqs.Queue('ingressQueueDead', {
})

const ingressQueue = new aws.sqs.Queue('ingressQueue', {
  redrivePolicy: pulumi.interpolate`{
    "deadLetterTargetArn": "${ingressQueueDead.arn}",
    "maxReceiveCount": "2"
  }
`
})

[brunetto]

Hi, I have a similar issue but with the Go SDK. Following the docs and this issue I tried

name := "queue"
dlqName := "dlq"

// create DLQ
dlq, err := sqs.NewQueue(ctx, dlqName, &sqs.QueueArgs{
	Name:                    pulumi.String(dlqName),
})
if err != nil {
	// ...
}

// get the DLQ ARN property
// from https://www.pulumi.com/docs/intro/concepts/inputs-outputs/#apply
dlqArn := dlq.Arn.ApplyT(func(arn string) string {
	return arn
}).(pulumi.StringOutput)

// print DLQ ARN
err = ctx.Log.Info(fmt.Sprintf("dlq arn %v\n", dlqArn), nil)
if err != nil {
	// ...
}

// JSON encode properties
redrivePolicy, err := json.Marshal(map[string]interface{}{
	"deadLetterTargetArn": dlqArn,
	"maxReceiveCount":     QueueDefaultMaxRecCount,
})
if err != nil {
	return nil, errors.Wrap(err, "can't json encode redrive policy")
}

redriveAllowPolicy, err := json.Marshal(map[string]interface{}{
	"redrivePermission": "byQueue",
	"sourceQueueArns":   []interface{}{dlqArn}})
if err != nil {
	return nil, errors.Wrap(err, "can't json encode redrive allow policy")
}

// create queue
queue, err := sqs.NewQueue(ctx, name, &sqs.QueueArgs{
	Name:                    pulumi.String(name),
	RedrivePolicy:           pulumi.String(redrivePolicy),
	RedriveAllowPolicy:      pulumi.String(redriveAllowPolicy),
},
	// queue depends on dlq
	pulumi.DependsOn([]pulumi.Resource{dlq}),
)
if err != nil {
	// ...
}

but I still get

Diagnostics:
  pulumi:pulumi:Stack (test-dev):
    dlq arn {0xc000261110}
    error: update failed
 
  aws:sqs:Queue (queue-dev):
    error: 1 error occurred:
        * error creating SQS Queue (queue-dev): InvalidParameterValue: Value {"deadLetterTargetArn":{},"maxReceiveCount":3} for parameter RedrivePolicy is invalid. Reason: Invalid value for deadLetterTargetArn.
        status code: 400, request id: c72d34bd-81de-5755-9c73-6ece9df694f6

I tried to get a string from the ARN, but still not working. Basically the ARN I get is either empty or invalid.

Any suggestion?

[ghostsquad]

@brunetto your JSON Marshall operation is happening before the ARN (which is a future) resolves. You must marshal inside of an Apply

[lukehoban]

There are three things I think this is tracking:

1. Improving the example from upstream to include the Queue definition
2. Ensuring that the codegen for the example uses pulumi.jsonStringify so that the unbound variable can be an Output (opened https://github.com/pulumi/pulumi/issues/12354 on this explicitly)
3. Fixing the rendering of examples on this page to be more readable (https://github.com/pulumi/pulumi-terraform-bridge/issues/846)

I'll leave this open to track (1), though I expect it's likely easier and more scalable to "solve" this via addressing (2).