Generating template for GCP provider with TypeScript consistently leads to V8 module hitting OOM error

[tylerburdsall]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

cdktf & Language Versions

• cdktf version: 0.11.2
• @cdktf/provider-aws: 8.0.10
• @cdktf/provider-google: 0.9.26
• typescript: 4.4.2

%cat cdktf.json
{
  "language": "typescript",
  "app": "npx ts-node bin/main.ts",
  "projectId": "<removed>",
  "terraformProviders": [],
  "terraformModules": [],
  "context": {
    "excludeStackIdFromLogicalIds": "true",
    "allowSepCharsInLogicalIds": "true"
  }
}

Affected Resource(s)

Debug Output

https://gist.github.com/tylerburdsall/3b2ffd87ec07e4b5ba43e508ee161601

Expected Behavior

cdktf should have been able to synthesize project without OOM error

Actual Behavior

The process fails with an OOM error for the Node V8 engine

Steps to Reproduce

1. Create a project that uses GCP as a provider
2. Build project on an AWS CodeBuild machine (Linux, 15 GB memory, 8 vCPUs)
3. Run cdktf synth without adding any additional Node environment variables to increase heap size

Important Factoids

It looks like this is a known issue that has happened with other providers: https://github.com/hashicorp/terraform-cdk/issues/1264

I can also see that there is a warning in the library itself: https://github.com/hashicorp/terraform-cdk/blob/main/packages/%40cdktf/provider-generator/lib/get/constructs-maker.ts#L320

Is there a reason the NODE_OPTIONS flag isn't set across providers (perhaps here for reference)?

References

[cprice404]

Another data point on this issue; we were synthing fine with cdktf v0.10.3, so this seems like a regression that occurred sometime between those two releases.

[DanielMSchmidt]

You can set NODE_OPTIONS yourself as a temporary solution. Besides it seems like the amount of code we generate is too much for TS to handle. While there is not much besides improving JSII we can do regarding cdktf get performance you could look into changing your app command to be something more performant for the compilation process.

esbuild could be a good alternative, in the end all the app command needs to do is to run your program so it can synthesize the configuration. Deno could be an option as well.

[ansgarm]

Hi @tylerburdsall 👋

The error log you posted seems to be something new and unrelated to the generator code and previous issues you found. Previously, we only saw issues while generating the code for the Terraform providers (i.e. while running cdktf get).

The error you got seems to be from invoking ts-node (which is the app command in the TypeScript template that is invoked when cdktf synth is run). We originally switched to ts-node to fix an issue with stale transpiled code. However, since then, we implemented more features which required more generated code for the providers, probably leading up to this issue. As the provider bindings now have more code and ts-node caches everything in memory, using both the AWS provider and the Google provider (which both are quite large) seems to exceed the default memory limit of 2 GB that NodeJS runs with.

As Daniel already mentioned, you can work around this by setting the NODE_OPTIONS environment variable when running ts-node (i.e. when running cdktf synth). You could even add that one to your app script in the cdktf.json config file:

"app": "NODE_OPTIONS='--max-old-space-size=4048' npx ts-node bin/main.ts"

(I thing 4 GB might suffice already, but maybe that one needs to be increased further)

That said, we should still find a solution for this problem by either replacing ts-node with something else that removes unused code and does not load that into the memory of the invoked NodeJS process, or we should at least update our TypeScript template with that fix.

Adding to this: I'm suspecting recursive type configs in the "large" Terraform providers to be causing a lot of the size of those packages. So tackling that could also make this problem disappear. The underlying reason for this is that the Terraform provider JSON schema does not contain information about such recursion but instead just explicitly renders it 50 levels deep or so. This also causes other problems (e.g. with naming).

[ansgarm]

I just tried to reproduce your error but failed to do so (when using the prebuilt providers from NPM). If you are able to share a reproduction example that would be great.

However, I was able to reproduce the OOM error while synthesizing using locally built providers (via cdktf get). I suspect that they were able to fail because locally generated provider bindings are in TypeScript whereas the published pre-built providers contain JS code (which seems to be less heavy on ts-node).

Reproduction (for future reference)

Run

cdktf init --local --template typescript
cdktf provider add --force-local azurerm aws google # I also had to add Azure for it to fail with an out of memory error

Add this to the main.ts

import * as aws from "./.gen/providers/aws";
import * as google from "./.gen/providers/google";
import * as azurerm from "./.gen/providers/azurerm";
// and in the constructor below "define resources here"
new aws.AwsProvider(this, "aws");
new google.GoogleProvider(this, "google");
new azurerm.AzurermProvider(this, "azurerm", { features: {} });

Then run

cdktf synth

(should fail if your NodeJS version defaults to 2 GB memory or less) else you can force the error by running

NODE_OPTIONS="--max-old-space-size=2024" cdktf synth

Running NODE_OPTIONS="--max-old-space-size=4048" cdktf synth should succeed.

[ansgarm]

Hi 👋 We recently released CDKTF 0.13 which puts each resource into their own namespace and due to this, synthesizing should take way less memory as ts-node does not need to go over the whole provider at once (if using direct imports with /lib/ in the import path. Head over to https://cdk.tf/0.13 to learn more about this recent change and how to adjust the imports.

However, if you still run into performance issues in certain cases, don't hesitate to file a new issue and we'll try to reproduce and improve those cases as well 🕵️

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days. This helps our maintainers find and focus on the active issues. If you've found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.