(aws-cdk-lib): Use a streaming json implementation when serializing the tree.json file

[bmoffatt]

Describe the feature

The tree.json file should be able to be arbitrarily large. Currently it is limited to ~512mb, node's max string size!

Use Case

I have a growing CDK application which is running into limitations of node.js when the tree.json file is being written during synthesis. I'm hitting the max string size!

RangeError: Invalid string length
    at JSON.stringify (<anonymous>)
    at TreeMetadata._synthesizeTree (/<redacted>/node_modules/aws-cdk-lib/core/lib/private/tree-metadata.js:1:1457)

With either of my workarounds, my tree.json ends up being ~630mb.

Proposed Solution

I tried patching aws-cdk-lib to use json-stream-stringify, and while this appeared to resolve my synthesis failures, it didn't pass all the unit tests, presumably because the file write no longer happens synchronously. I'd be willing to spend more time opening a pull-request and iterating on this approach if ya'll think it's reasonable.

https://github.com/bmoffatt/aws-cdk/commit/91edfec0365ab5ff2f479df675d9afc6f3a54fab

---

The way I've actually unblocked myself before developing the above library change was patching my node.js build - I went this way because I knew I'd be able to validate it faster than learning how to modify and test and contributions to the aws-cdk. Obviously I don't want to maintain either patch long term :)

diff -ruN dist/node-v18.16.1/deps/v8/include/v8-primitive.h dist/node-v18.16.1-patched/deps/v8/include/v8-primitive.h
--- dist/node-v18.16.1/deps/v8/include/v8-primitive.h	2023-06-20 12:50:23.000000000 +0000
+++ dist/node-v18.16.1-patched/deps/v8/include/v8-primitive.h	2023-07-27 21:13:09.852080196 +0000
@@ -123,7 +123,7 @@
 class V8_EXPORT String : public Name {
  public:
   static constexpr int kMaxLength =
-      internal::kApiSystemPointerSize == 4 ? (1 << 28) - 16 : (1 << 29) - 24;
+      internal::kApiSystemPointerSize == 4 ? (1 << 28) - 16 : (1 << 30) - 24;
 
   enum Encoding {
     UNKNOWN_ENCODING = 0x1,

---

An alternative solution to my problem would be making the generation of the tree.json optional. I'm new-ish to CDK development and don't really know what it's for 🤷‍♂️ - my light googling makes me think it's an optional bit of metadata for use with visualizers

Other Information

No response

Acknowledgements

• [X] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

CDK version used

v2.91.0

Environment details (OS name and version, etc.)

macOS

[peterwoodworth]

Thanks for the request @bmoffatt,

I have never seen an error related to generating tree.json before. I'm curious if you are able to share a way to reproduce this? If this is reasonable to run into in some use cases we can take this as a bug report

[bmoffatt]

Here's an (unreasonable) reproduction of the error :) - https://github.com/bmoffatt/cdk-tree-limit/tree/main - 9001 stacks each with 101 sqs queues!

The real-world application I'm working with is several large stacks, some with 100s of resources, fanned out to every AWS region.

[mikewrighton]

Seems like there's a way to avoid the tree.json being generated, with:

const app = new cdk.App({
  treeMetadata: false
});

Is this a reasonable workaround for you?

[bmoffatt]

Oh nice!

Trying it out briefly, I can see it'll break a workflow used for diff-ing changes during development and code review. I'll have to check with my team to see how feasible it is to change or go without.

[bmoffatt]

So while no-one has complained yet about losing access to cdk diffs, someone on my team recently ran into an issue with an internal platform tool barfing on not being able to find and walk the tree file. For now, we're back to using my shady node patch workaround.

[rowantran]

+1 on this issue -- one of our applications recently passed the threshold to trigger this issue, and setting treeMetadata to false breaks some of our workflows that rely on the tree.json.

We may be able to use something like @bmoffatt 's patch for now to get around this, but we definitely don't want to rely on a patched NodeJS longterm either :)

Would appreciate if the CDK team is able to prioritize this fix!

[SaadRehmanCS]

+1 our team has encountered this issue and we're currently working around it with the above mentioned ways but we're waiting for a long term fix :)

[rix0rrr]

We can switch to a streaming implementation for the writer, but this is used in the AWS plugin for VSCode which is also written in JS, and then it will refuse to read that large file.

As another way to make sure the file doesn't grow too large (a technique we could also apply to the manifest.json), how about the following:

• After the tree is built, we go through it breadth-first.
• As soon as we are approaching the size limits (we'll count that simply by counting nodes, not serialized bytes that's too finicky) we are going to replace nodes at the next level with a "your tree is in another castle" node, with a type, message and filename indicating that.
• We write the tree sharded over files.

That way, consumers still have something, they just lose browseable detail if they have humongous trees, until the code is changed to deal with the indirection.

[bmoffatt]

Expressing continued interest in this :) A member of my team recently bumped into the limit of what's achievable with my approach of patching nodejs.

[sorawee]

Not from the CDK team, but I submitted #33784, which is a very straightforward change to remove whitespaces from tree.json. Eventually if an application is large enough, we will hit the same error again, but with a lot of spaces gained from this change, that should not happen anytime soon.

[github-actions[bot]]

Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.