terraform-cdk
terraform-cdk copied to clipboard
Unit testing failing to find resources for azurerm provider
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
cdktf & Language Versions
cdktf 0.8.6 terraform v1.1.3 on windows_amd64 nodejs v16.13.1 Note: I tested both directly under windows command prompt and under WSL, and I get the same results in both cases.
Affected Resource(s)
PublicIp is what I tested, but I expect all will be affected
Debug Output
N/A (problem in unit testing, it didn't generate a debug log file)
Expected Behavior
There are actually two problems here. I'm including them in the same issue because we think there's a good chance they have the same cause. The actual cdktf code deploys just fine; the problem is only in unit testing.
There are two kinds of test cases failing. We think they may be related.
import { PublicIp } from "../.gen/providers/azurerm/public-ip";
it ("is TEST CASE #1", () => {
expect(
Testing.synthScope((scope) => {
new AppGatewayStack(scope, QA_PARAMS); // AppGatewayStack does create a PublicIp
})
).toHaveResource(PublicIp);
})
it ("is TEST CASE #2", () => {
const app = new App();
const stack = new AppGatewayStack(app, QA_PARAMS);
expect(Testing.fullSynth(stack)).toMatchSnapshot();
})
Expected behavior: Both tests should have passed.
Actual Behavior
TEST CASE #1 actual behavior: It fails with the error
FAIL __tests__/main-test.ts (12.786 s)
My CDKTF Application
Test the public IP
× should have a public IP (59 ms)
● My CDKTF Application › Test the public IP › should have a public IP
Type resource not found in stack
58 | // }
59 | })
> 60 | ).toHaveResource(PublicIp);
| ^
61 | })
62 | });
63 | // // All Unit testst test the synthesised terraform code, it does not create real-world resources
at getAssertElementWithProperties (node_modules/cdktf/lib/testing/matchers.ts:108:13)
at toHaveResourceWithProperties (node_modules/cdktf/lib/testing/matchers.ts:186:64)
at Object.toHaveResource (node_modules/cdktf/lib/testing/adapters/jest.ts:72:65)
at __EXTERNAL_MATCHER_TRAP__ (node_modules/expect/build/index.js:386:30)
at Object.throwingMatcher [as toHaveResource] (node_modules/expect/build/index.js:387:15)
at Object.<anonymous> (__tests__/main-test.ts:60:9)
TEST CASE #2 actual behavior: The first time you run it, it passes and generates a snapshot file (though, see below). The second time and after, it fails with the error below.
✕ should not change the snapshot (24 ms)
● AppGateway CDKTF Application › Test the public IP › should not change the snapshot
expect(received).toMatchSnapshot()
Snapshot name: `AppGateway CDKTF Application Test the public IP should not change the snapshot 1`
Snapshot: "/tmp/cdktf.outdir.sMGHRL"
Received: "/tmp/cdktf.outdir.fElhCW"
This second error actually makes sense in a way, because the snapshot file generated seems to be wrong. Rather than a representation of the stack resources, the __snapshots__/main-test.ts.snap
file looks like this:
// Jest Snapshot v1, https://goo.gl/fbAQLP
exports[`AppGateway CDKTF Application Test the public IP should not change the snapshot 1`] = `"/tmp/cdktf.outdir.sMGHRL"`;
The JSON with actual stack resources IS being generated under that location.
Steps to Reproduce
Install dependencies and run npm run test
Important Factoids
References
- #0000
I tried to reproduce this, on OSX it seems to work, can you check if this production case fails as well for you? https://github.com/DanielMSchmidt/cdktf-issue-1506
Huh, it does work - I can run npm run test
on that repo, and it works under windows. There's something weird going on here. One thing I did in your test is I changed the snapshot test to be a "normal" one:
expect(
Testing.synthScope((scope) => {
new MyConstruct(scope, "myConstruct");
})
).toMatchSnapshot();
Not only did this work, but the snapshot it generated looks correct. There's something weird going on with mine. I note that I include a number of other resources, as well as an Azure provider and S3 backend construct. I'll try to get it down to the minimum difference.
So I figured out what was wrong - sort of. At least, I figured out how to make it work, by comparing against your example. Problem was, when I was calling
Testing.synthScope((scope) => {
new AppGatewayStack(scope, QA_PARAMS); // AppGatewayStack does create a PublicIp
})
AppGatewayStack
is defined as:
export class AppGatewayStack extends TerraformStack {
constructor(scope: Construct, params: Parameters) { // , options: AppGatewayStackOptions
super(scope, params.stackName);
// etc.
}
}
But you had it synthing a subclass of Construct, not TerraformStack. Now, given that TerraformStack extends Construct, I would expect that to work, also. But it doesn't. I instead modified it to:
Testing.synthScope((scope) => {
new AppGatewayConstruct(scope, QA_PARAMS);
})
...
export class AppGatewayConstruct extends Construct {
constructor(scope: Construct, params: Parameters) { // , options: AppGatewayStackOptions
super(scope, `${params.stackName}-construct`);
// etc.
}
}
and now everything works!
Testing.synthScope
internally creates a new stack. I don't believe there's been any testing done on nested stacks, so I'm not surprised that they don't work correctly. We could either make sure nested stacks work properly (probably best in the long term) or make Testing.synthScope
more obvious that a stack can't be passed (probably fails with an App
as well).
Your second test case comes down to Testing.fullSynth
just always returns a temporary directory. Looks like the intent was to chain it with other functions (such as toBeValidTerraform
. At a minimum there is some more documentation needed there. Should probably consider renaming the function as well.
Testing.synthScope
internally creates a new stack. I don't believe there's been any testing done on nested stacks, so I'm not surprised that they don't work correctly. We could either make sure nested stacks work properly (probably best in the long term) or makeTesting.synthScope
more obvious that a stack can't be passed (probably fails with anApp
as well).
I don't think the immediate issue is best solved by making nested stacks work. Because the thing that's in the way here is the fact that Testing.synthScore
creates a new stack regardless. I'm pretty sure the common case is going to be testing whether a Stack has specific resources, rather than a Construct.