pulumi-awsx
pulumi-awsx copied to clipboard
pulumi
Cannot access properties of `loadBalancer` in `ApplicationLoadBalancer`
What happened?
When trying to setup an ALB, it seems the properties of the underlying loadBalancer are inaccessible. This is preventing me from associating new listeners with the ALB, and makes me nervous that any records I create based on the "undefined" dnsName might spontaneously break.
Example
Here's all the code for the MRE:
import * as aws from '@pulumi/aws';
import * as awsx from '@pulumi/awsx';
import * as pulumi from '@pulumi/pulumi';
const stack = pulumi.getStack();
const cluster = new aws.ecs.Cluster(`test-cluster-${stack}`, {});
const defaultVpc = new awsx.ec2.DefaultVpc('default-vpc');
const serverTg = pulumi.Output.create(new aws.lb.TargetGroup(`test-server-tg-${stack}`, {
vpcId: defaultVpc.vpcId,
port: 80,
protocol: 'HTTP',
targetType: 'ip',
protocolVersion: 'HTTP1',
}));
const lb = new awsx.lb.ApplicationLoadBalancer(`test-lb-${stack}`, {});
new aws.alb.Listener('test-listener', {
loadBalancerArn: lb.loadBalancer.arn,
port: 443,
protocol: 'HTTP',
defaultActions: [{
type: 'forward',
targetGroupArn: serverTg.arn,
}],
});
const serviceSg = new aws.ec2.SecurityGroup(`test-service-sg-${stack}`, {
ingress: [
{
protocol: '-1',
fromPort: 0,
toPort: 0,
cidrBlocks: ['0.0.0.0/0'],
},
],
});
new awsx.ecs.FargateService(`test-server-service-${stack}`, {
cluster: cluster.arn,
networkConfiguration: {
assignPublicIp: true,
securityGroups: [serviceSg.id],
subnets: defaultVpc.publicSubnetIds,
},
taskDefinitionArgs: {
container: {
name: 'server',
image: 'nginx:stable',
cpu: 1024,
memory: 4 * 1024,
essential: true,
portMappings: [
{ targetGroup: serverTg },
],
},
},
});
export const url = lb.loadBalancer.dnsName;
Previewing changes with pulumi preview results in the following output:
Previewing update (dev)
View in Browser (Ctrl+O): https://app.pulumi.com/rpmccarter/bug-repro/dev/previews/71840ef0-7566-4550-8cdb-ae73aff71eb7
Type Name Plan Info
+ pulumi:pulumi:Stack bug-repro-dev create 1 error; 1 warning
+ ├─ awsx:ec2:DefaultVpc default-vpc create
+ ├─ aws:ecs:Cluster test-cluster-dev create
+ ├─ aws:ec2:SecurityGroup test-service-sg-dev create 1 warning
+ ├─ awsx:lb:ApplicationLoadBalancer test-lb-dev create
+ │ ├─ aws:ec2:SecurityGroup test-lb-dev create
+ │ ├─ aws:lb:TargetGroup test-lb-dev create
+ │ ├─ aws:lb:LoadBalancer test-lb-dev create
+ │ └─ aws:lb:Listener test-lb-dev-0 create
+ ├─ aws:lb:TargetGroup test-server-tg-dev create
+ ├─ awsx:ecs:FargateService test-server-service-dev create
+ │ └─ awsx:ecs:FargateTaskDefinition test-server-service-dev create
+ │ ├─ aws:iam:RolePolicyAttachment test-server-service-dev-execution-9a42f520 create
+ │ ├─ aws:cloudwatch:LogGroup test-server-service-dev create
+ │ ├─ aws:iam:Role test-server-service-dev-task create
+ │ └─ aws:iam:Role test-server-service-dev-execution create
└─ aws:alb:Listener test-listener 1 error
Diagnostics:
aws:alb:Listener (test-listener):
error: aws:alb/listener:Listener resource 'test-listener' has a problem: Missing required argument. The argument "load_balancer_arn" is required, but no definition was found.. Examine values at 'test-listener.loadBalancerArn'.
aws:ec2:SecurityGroup (test-service-sg-dev):
warning: Use of inline rules is discouraged as they cannot be used in conjunction with any Security Group Rule resources. Doing so will cause a conflict and may overwrite rules.
pulumi:pulumi:Stack (bug-repro-dev):
warning: Undefined value (url) will not show as a stack output.
error: Running program '/Users/rpmccarter/Documents/codes/pulumi/index.ts' failed with an unhandled exception:
<ref *1> Error: failed to register new resource test-server-service-dev [awsx:ecs:FargateService]: Resource monitor is terminating
at Object.registerResource (/Users/rpmccarter/Documents/codes/pulumi/node_modules/@pulumi/awsx/node_modules/@pulumi/runtime/resource.ts:339:27)
at new Resource (/Users/rpmccarter/Documents/codes/pulumi/node_modules/@pulumi/awsx/node_modules/@pulumi/resource.ts:398:13)
at new ComponentResource (/Users/rpmccarter/Documents/codes/pulumi/node_modules/@pulumi/awsx/node_modules/@pulumi/resource.ts:891:9)
at new FargateService (/Users/rpmccarter/Documents/codes/pulumi/node_modules/@pulumi/ecs/fargateService.ts:84:9)
at Object.<anonymous> (/Users/rpmccarter/Documents/codes/pulumi/index.ts:41:1)
at Module._compile (node:internal/modules/cjs/loader:1376:14)
at Module.m._compile (/Users/rpmccarter/Documents/codes/pulumi/node_modules/ts-node/src/index.ts:439:23)
at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
at Object.require.extensions.<computed> [as .ts] (/Users/rpmccarter/Documents/codes/pulumi/node_modules/ts-node/src/index.ts:442:12)
at Module.load (node:internal/modules/cjs/loader:1207:32) {
code: 1,
promise: Promise { <rejected> [Circular *1] }
}
Of note are the following lines:
error: aws:alb/listener:Listener resource 'test-listener' has a problem: Missing required argument. The argument "load_balancer_arn" is required, but no definition was found.. Examine values at 'test-listener.loadBalancerArn'.
Here, the argument loadBalancerArn is set to lb.loadBalancer.arn, and pulumi is reporting it as undefined.
warning: Undefined value (url) will not show as a stack output.
Here, the export url is set to lb.loadBalancer.dnsName, and pulumi is reporting it as undefined.
Output of pulumi about
CLI
Version 3.112.0
Go Version go1.22.1
Go Compiler gc
Plugins
NAME VERSION
aws 6.28.2
awsx 2.5.0
docker 4.5.3
docker 3.6.1
nodejs unknown
Host
OS darwin
Version 14.4
Arch arm64
This project is written in nodejs: executable='/Users/rpmccarter/.nvm/versions/node/v20.10.0/bin/node' version='v20.10.0'
Current Stack: rpmccarter/bug-repro/dev
Found no resources associated with dev
Found no pending operations associated with dev
Backend
Name pulumi.com
URL https://app.pulumi.com/rpmccarter
User rpmccarter
Organizations rpmccarter
Token type personal
Dependencies:
NAME VERSION
@pulumi/aws 6.28.2
@pulumi/awsx 2.5.0
@pulumi/pulumi 3.109.0
@types/node 16.18.22
typescript 5.3.3
Pulumi locates its logs in /var/folders/dn/z0by0dcj1gnbkjr6_t71hp_m0000gn/T/ by default
Additional context
No response
Contributing
Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).
Updating (dev)
View in Browser (Ctrl+O): https://app.pulumi.com/anton-pulumi-corp/awsx-1253/dev/updates/4
Type Name Status Info
Type Name Status Info
+ pulumi:pulumi:Stack awsx-1253-dev creating (1091s)
+ ├─ awsx:ec2:DefaultVpc default-vpc created (2s)
Type Name Status Info
+ pulumi:pulumi:Stack awsx-1253-dev creating (1092s).
+ ├─ awsx:ec2:DefaultVpc default-vpc created (2s)
+ ├─ aws:ec2:SecurityGroup test-service-sg-dev created (2s) warning: Use of
inline rules is discouraged as they cannot be used in conjunction with any Security Group Rule resources. Doing so will cause
Type Name Status Info
+ pulumi:pulumi:Stack awsx-1253-dev **creating failed (179s)** 1 error
+ ├─ awsx:ec2:DefaultVpc default-vpc created (2s)
+ ├─ aws:ec2:SecurityGroup test-service-sg-dev created (2s) 1 warning
+ ├─ aws:ecs:Cluster test-cluster-dev created (10s)
+ ├─ awsx:lb:ApplicationLoadBalancer test-lb-dev created (0.48s)
+ │ ├─ aws:lb:TargetGroup test-lb-dev created (0.83s)
+ │ ├─ aws:ec2:SecurityGroup test-lb-dev created (2s)
+ │ ├─ aws:lb:LoadBalancer test-lb-dev created (162s)
+ │ └─ aws:lb:Listener test-lb-dev-0 created (0.48s)
+ ├─ aws:lb:TargetGroup test-server-tg-dev created (1s)
+ ├─ awsx:ecs:FargateService test-server-service-dev created (1s)
+ │ ├─ awsx:ecs:FargateTaskDefinition test-server-service-dev created (2s)
+ │ │ ├─ aws:cloudwatch:LogGroup test-server-service-dev created (0.35s)
+ │ │ ├─ aws:iam:Role test-server-service-dev-task created (0.68s)
+ │ │ ├─ aws:iam:Role test-server-service-dev-execution created (0.96s)
+ │ │ ├─ aws:ecs:TaskDefinition test-server-service-dev created (0.38s)
+ │ │ └─ aws:iam:RolePolicyAttachment test-server-service-dev-execution-9a42f520 created (0.50s)
+ │ └─ aws:ecs:Service test-server-service-dev **creating failed** 1 error
+ └─ aws:alb:Listener test-listener created (3s)
Diagnostics:
pulumi:pulumi:Stack (awsx-1253-dev):
error: update failed
aws:ec2:SecurityGroup (test-service-sg-dev):
warning: Use of inline rules is discouraged as they cannot be used in conjunction with any Security Group Rule resources. Doing so will cause a conflict and may overwrite rules.
aws:ecs:Service (test-server-service-dev):
error: 1 error occurred:
* creating urn:pulumi:dev::awsx-1253::awsx:ecs:FargateService$aws:ecs/service:Service::test-server-service-dev: 1 error occurred:
* waiting for ECS Service (arn:aws:ecs:us-east-1:616138583583:service/test-cluster-dev-329aea0/test-server-service-dev-4162b56) create: timeout while waiting for state to become 'tfSTABLE' (last state: 'tfPENDING', timeout: 20m0s)
Outputs:
url: "test-lb-dev-519fb5d-147158593.us-east-1.elb.amazonaws.com"
Resources:
+ 18 created
Duration: 23m8s
Thanks for reporting this! I've tried reproducing and I hit another known issue with the timeout on ECS service but I cannot reproduce the problem you're reporting.
CLI
Version 3.111.1
Go Version go1.22.1
Go Compiler gc
Plugins
NAME VERSION
aws 6.29.0
awsx 2.6.0
docker 4.5.3
docker 3.6.1
nodejs unknown
Host
OS darwin
Version 14.4
Arch x86_64
This project is written in nodejs: executable='/Users/t0yv0/bin/node' version='v18.18.2'
Current Stack: anton-pulumi-corp/awsx-1253/dev
TYPE URN
pulumi:pulumi:Stack urn:pulumi:dev::awsx-1253::pulumi:pulumi:Stack::awsx-1253-dev
pulumi:providers:awsx urn:pulumi:dev::awsx-1253::pulumi:providers:awsx::default_2_6_0
pulumi:providers:aws urn:pulumi:dev::awsx-1253::pulumi:providers:aws::default_6_29_0
awsx:ec2:DefaultVpc urn:pulumi:dev::awsx-1253::awsx:ec2:DefaultVpc::default-vpc
pulumi:providers:aws urn:pulumi:dev::awsx-1253::pulumi:providers:aws::default_6_9_0
aws:ec2/securityGroup:SecurityGroup urn:pulumi:dev::awsx-1253::aws:ec2/securityGroup:SecurityGroup::test-service-sg-dev
awsx:lb:ApplicationLoadBalancer urn:pulumi:dev::awsx-1253::awsx:lb:ApplicationLoadBalancer::test-lb-dev
aws:lb/targetGroup:TargetGroup urn:pulumi:dev::awsx-1253::aws:lb/targetGroup:TargetGroup::test-server-tg-dev
aws:lb/targetGroup:TargetGroup urn:pulumi:dev::awsx-1253::awsx:lb:ApplicationLoadBalancer$aws:lb/targetGroup:TargetGroup::test-lb-dev
aws:ec2/securityGroup:SecurityGroup urn:pulumi:dev::awsx-1253::awsx:lb:ApplicationLoadBalancer$aws:ec2/securityGroup:SecurityGroup::test-lb-dev
aws:ecs/cluster:Cluster urn:pulumi:dev::awsx-1253::aws:ecs/cluster:Cluster::test-cluster-dev
aws:lb/loadBalancer:LoadBalancer urn:pulumi:dev::awsx-1253::awsx:lb:ApplicationLoadBalancer$aws:lb/loadBalancer:LoadBalancer::test-lb-dev
aws:lb/listener:Listener urn:pulumi:dev::awsx-1253::awsx:lb:ApplicationLoadBalancer$aws:lb/listener:Listener::test-lb-dev-0
pulumi:providers:pulumi urn:pulumi:dev::awsx-1253::pulumi:providers:pulumi::default
awsx:ecs:FargateService urn:pulumi:dev::awsx-1253::awsx:ecs:FargateService::test-server-service-dev
awsx:ecs:FargateTaskDefinition urn:pulumi:dev::awsx-1253::awsx:ecs:FargateService$awsx:ecs:FargateTaskDefinition::test-server-service-dev
aws:alb/listener:Listener urn:pulumi:dev::awsx-1253::aws:alb/listener:Listener::test-listener
aws:cloudwatch/logGroup:LogGroup urn:pulumi:dev::awsx-1253::awsx:ecs:FargateService$awsx:ecs:FargateTaskDefinition$aws:cloudwatch/logGroup:LogGroup::test-server-service-dev
aws:iam/role:Role urn:pulumi:dev::awsx-1253::awsx:ecs:FargateService$awsx:ecs:FargateTaskDefinition$aws:iam/role:Role::test-server-service-dev-task
aws:iam/role:Role urn:pulumi:dev::awsx-1253::awsx:ecs:FargateService$awsx:ecs:FargateTaskDefinition$aws:iam/role:Role::test-server-service-dev-execution
aws:ecs/taskDefinition:TaskDefinition urn:pulumi:dev::awsx-1253::awsx:ecs:FargateService$awsx:ecs:FargateTaskDefinition$aws:ecs/taskDefinition:TaskDefinition::test-server-service-dev
aws:iam/rolePolicyAttachment:RolePolicyAttachment urn:pulumi:dev::awsx-1253::awsx:ecs:FargateService$awsx:ecs:FargateTaskDefinition$aws:iam/rolePolicyAttachment:RolePolicyAttachment::test-server-service-dev-execution-9a42f520
aws:ecs/service:Service urn:pulumi:dev::awsx-1253::awsx:ecs:FargateService$aws:ecs/service:Service::test-server-service-dev
Found no pending operations associated with dev
Backend
Name pulumi.com
URL https://app.pulumi.com/anton-pulumi-corp
User anton-pulumi-corp
Organizations anton-pulumi-corp, moolumi, pulumi
Token type personal
Dependencies:
NAME VERSION
@pulumi/aws 6.29.0
@pulumi/awsx 2.6.0
@pulumi/pulumi 3.112.0
@types/node 18.19.29
Pulumi locates its logs in /var/folders/gk/cchgxh512m72f_dmkcc3d09h0000gp/T/com.apple.shortcuts.mac-helper// by default
I notice I have slightly newer versions. Does this still reproduce on latest for you? Does it reproduce consistently or sporadically (need to retry)?
Does this still reproduce on latest for you?
Just tested with the versions shown in your code snippet and was able to reproduce:
Dependencies:
NAME VERSION
@pulumi/aws 6.29.0
@pulumi/awsx 2.6.0
@pulumi/pulumi 3.112.0
@types/node 16.18.22
typescript 5.3.3
Does it reproduce consistently or sporadically (need to retry)?
I could reproduce it consistently in the past - every time, running pulumi preview would cause the error without fail. I then ran pulumi up (failed due to the same error you experienced) followed by pulumi destroy (succeeded), now it seems to be working fine. Let me try to get it reproducing again
@rpmccarter I tried reproducing this but wasn't successful. The timeout issue you're seeing when running pulumi up are caused by the security group blocking egress traffic; this blocks the fargate task from pulling the docker image.
After changing the security group to this the deployment works:
const serviceSg = new aws.ec2.SecurityGroup(`test-service-sg`, {
ingress: [
{
protocol: '-1',
fromPort: 0,
toPort: 0,
cidrBlocks: ['0.0.0.0/0'],
},
],
egress: [
{
protocol: '-1',
fromPort: 0,
toPort: 0,
cidrBlocks: ['0.0.0.0/0'],
}
]
});
Were you successful reproducing it again? If yes, do you mind trying with the latest aws and awsx versions? I tried with the following versions:
Dependencies:
NAME VERSION
@pulumi/aws 6.34.0
@pulumi/awsx 2.9.0
@pulumi/pulumi 3.115.2
@types/node 18.19.33
typescript 5.4.5
Revisiting this as I'm seeing it again. This time, I'm trying to create a Cloudflare record with the following code:
new cloudflare.Record(`api-record-${stack}`, {
zoneId,
name: apiSubdomain,
value: lb.loadBalancer.dnsName,
type: 'CNAME',
proxied: true,
});
Preview works fine, but when deploying, I'm seeing the following error:
cloudflare:index:Record (api-record-dev)
error: 1 error occurred:
* either 'value' (present: false) or 'data' (present: false) must be provided
Can I give someone permission to take a look at my Pulumi update logs to get better insight into this issue? I'm very paranoid of updating anything for this reason, could easily take down our service
I'll try to create an MRE again
Okay I've successfully reproduced a couple times. This must have something to do with deep dependencies because the only way I was able to repro was by copying my yarn lockfile from my real repo to my MRE repo. The flakiness on my end was likely due to me unintentionally switching from yarn to npm and creating a fresh dependency tree.
Here's a repo with everything you should need to reproduce: https://github.com/rpmccarter/pulumi-awsx-lb-bug-repro
It looks like this issue is caused by having multiple versions of @pulumi/pulumi installed.
$ yarn why @pulumi/pulumi
yarn why v1.22.22
[1/4] 🤔 Why do we have the module "@pulumi/pulumi"...?
[2/4] 🚚 Initialising dependency graph...
[3/4] 🔍 Finding dependency...
[4/4] 🚡 Calculating file sizes...
=> Found "@pulumi/[email protected]"
info Has been hoisted to "@pulumi/pulumi"
info This module exists because it's specified in "dependencies".
info Disk size without dependencies: "55.96MB"
info Disk size with unique dependencies: "102.99MB"
info Disk size with transitive dependencies: "143.8MB"
info Number of shared dependencies: 78
=> Found "@pulumi/aws#@pulumi/[email protected]"
info This module exists because "@pulumi#aws" depends on it.
info Disk size without dependencies: "4.19MB"
info Disk size with unique dependencies: "48.12MB"
info Disk size with transitive dependencies: "84.25MB"
info Number of shared dependencies: 91
=> Found "@pulumi/awsx#@pulumi/[email protected]"
info This module exists because "@pulumi#awsx" depends on it.
info Disk size without dependencies: "4.19MB"
info Disk size with unique dependencies: "48.12MB"
info Disk size with transitive dependencies: "84.25MB"
info Number of shared dependencies: 91
=> Found "@pulumi/docker#@pulumi/[email protected]"
info This module exists because "@pulumi#awsx#@pulumi#docker" depends on it.
info Disk size without dependencies: "4.29MB"
info Disk size with unique dependencies: "48.21MB"
info Disk size with transitive dependencies: "84.34MB"
info Number of shared dependencies: 91
=> Found "docker-classic#@pulumi/[email protected]"
info This module exists because "@pulumi#awsx#docker-classic" depends on it.
info Disk size without dependencies: "4.29MB"
info Disk size with unique dependencies: "48.21MB"
info Disk size with transitive dependencies: "84.34MB"
info Number of shared dependencies: 91
✨ Done in 0.65s.
If I add a yarn resolution for a specific version (doesn't matter if it is the old 3.60.0 or the new 3.109.0 version) the issue goes away.
package.json
"resolutions": {
"@pulumi/pulumi": "3.109.0"
}
$ yarn why @pulumi/pulumi
yarn why v1.22.22
[1/4] 🤔 Why do we have the module "@pulumi/pulumi"...?
[2/4] 🚚 Initialising dependency graph...
[3/4] 🔍 Finding dependency...
[4/4] 🚡 Calculating file sizes...
=> Found "@pulumi/[email protected]"
info Has been hoisted to "@pulumi/pulumi"
info Reasons this module exists
- Specified in "dependencies"
- Hoisted from "@pulumi#aws#@pulumi#pulumi"
- Hoisted from "@pulumi#awsx#@pulumi#pulumi"
- Hoisted from "@pulumi#awsx#@pulumi#docker#@pulumi#pulumi"
- Hoisted from "@pulumi#awsx#docker-classic#@pulumi#pulumi"
info Disk size without dependencies: "55.96MB"
info Disk size with unique dependencies: "102.99MB"
info Disk size with transitive dependencies: "144.07MB"
info Number of shared dependencies: 48
✨ Done in 0.22s.
@rpmccarter you may be able to use the yarn resolution workaround for the time being, but I'll need to spend some more time figuring out why this cause the issue.
Looks like this is caused by https://github.com/pulumi/pulumi/issues/13223
We believe this should be closed by https://github.com/pulumi/pulumi/issues/13223 but please open another issue and link to this one if you are still having trouble. Appreciate the patience!