Unable to bundle CDK lambdas when executed through vitest

[Poliziano]

Describe the bug

Using vitest to test CDK stacks fails when the stack constructs a nodes lambda. When using jest the tests run without issue. I have no idea if the is a problem with CDK or vitest, or something else entirely.

Failed to bundle asset MyTestStack/my-lambda/Code/Stage, bundle output is located at /private/var/folders/wf/0n_3mjr91114bj87w744lyvm0000gn/T/cdk.outRqpmmh/bundling-temp-31e6d62c5dd7767fc416ae362fc44caa01c8c1f300c419c61d1f3663ad52c8ff-error: TypeError [ERR_INVALID_ARG_VALUE]: The argument 'stdio' is invalid. Received WritableWorkerStdio {
  _writableState: WritableState {
    objectMode: false,
    highWaterMark: 16384,
    finalCalled: false,...
 ❯ AssetStaging.bundle node_modules/aws-cdk-lib/core/lib/asset-staging.js:2:584
 ❯ AssetStaging.stageByBundling node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:4201
 ❯ stageThisAsset node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:1609
 ❯ Cache.obtain node_modules/aws-cdk-lib/core/lib/private/cache.js:1:242
 ❯ new AssetStaging node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:2004
 ❯ new Asset node_modules/aws-cdk-lib/aws-s3-assets/lib/asset.js:1:736
 ❯ AssetCode.bind node_modules/aws-cdk-lib/aws-lambda/lib/code.js:1:4731
 ❯ new Function node_modules/aws-cdk-lib/aws-lambda/lib/function.js:1:2803
 ❯ new NodejsFunction node_modules/aws-cdk-lib/aws-lambda-nodejs/lib/function.js:1:1171

Reproduction

I have created a repository that demonstrates the issue. For comparison, I have added one test that runs using vitest, and another test that runs using jest.

https://github.com/Poliziano/cdk-vitest-bug

System Info

System:
  OS: macOS 12.4
  CPU: (10) arm64 Apple M1 Pro
  Memory: 261.09 MB / 16.00 GB
  Shell: 5.8.1 - /bin/zsh
Binaries:
  Node: 16.13.2 - ~/Library/pnpm/node
  npm: 8.1.2 - ~/Library/pnpm/npm
Browsers:
  Safari: 15.5
npmPackages:
  vitest: ^0.16.0 => 0.16.0

Used Package Manager

npm

Validations

• [X] Follow our Code of Conduct
• [X] Read the Contributing Guidelines.
• [X] Read the docs.
• [X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• [X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• [X] The provided reproduction is a minimal reproducible example of the bug.

[Poliziano]

I opened this same issue with the aws-sdk team. They have responded with the following, if it helps: https://github.com/aws/aws-cdk/issues/20873#issuecomment-1171528320:

@Poliziano it looks like this is due to vitest not handling the stdout to stderr redirect. Under the hood we are doing something like this:

import { spawnSync } from 'child_process';
spawnSync(command, {
  stdio: [
    'ignore',
    process.stderr,
    'inherit',
  ],
});

For some reason it is the second value of process.stderr that is causing the issue with vitest. I'm not sure why, but I think it may have something to do with their use of workerpools.

Unfortunately I'm not sure that there is anything we can do to help with this.

[elthrasher]

I came across this as well. I'm leaving the comment here because this is the open issue, but I think this probably needs to be fixed on the aws-cdk side. We were able to use vitest with NodejsFunction up until version 0.13 by means of the --no-threads arg. This commit broke that capability. It seems that using child_process the way CDK does, doesn't work inside a worker thread. I made a minimal no-dependencies reproduction to demonstrate this.

https://github.com/elthrasher/worker_child

Here's why I think CDK owns this fix - they may wish to have the capability to run in worker threads and right now they don't, whether we use vitest or not. One mitigation would be to use isMainThread to detect whether or not the output should be redirected:

import { spawnSync } from 'child_process';
import { isMainThread } from 'worker_threads';
spawnSync(command, {
  stdio: isMainThread ? [
    'ignore',
    process.stderr,
    'inherit',
  ] : 'inherit', // or undefined or whatever is appropriate
});

Alternately vitest could restore the ability to run tests in the main thread, but seems like that doesn't meet with the vision of the library.