turborepo, when running sst deploys, leads to env vars not being available to underlying processes

[n-batalha]

Verify canary release

• [x] I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

https://github.com/n-batalha/turbo-repro-issue-10353/blob/main/README-error.md

Which canary version will you have in your reproduction?

2.5.1-canary.1

Environment information

turbo 2.5.1-canary.1

CLI:
   Version: 2.5.1-canary.1
   Path to executable: /Users/nelson/git/REDACTED/node_modules/.pnpm/[email protected]/node_modules/turbo-darwin-arm64/bin/turbo
   Daemon status: Running
   Package manager: pnpm9

Platform:
   Architecture: aarch64
   Operating system: macos
   WSL: false
   Available memory (MB): 3845
   Available CPU cores: 12

Environment:
   CI: None
   Terminal (TERM): xterm-ghostty
   Terminal program (TERM_PROGRAM): ghostty
   Terminal program version (TERM_PROGRAM_VERSION): 1.1.3
   Shell (SHELL): /bin/zsh
   stdin: false

Expected behavior

I need to run this command to deploy an app: pnpm exec sst --stage preview deploy, inside an sst app e.g. /app/myapp-infra.

For full context, it deploys an sst app, includes for instance the resource below for a next app:

export const nextjs = new sst.aws.Nextjs("MyApp-Web", {
  path: "./../myapp-web",
  // buildCommand: "pnpm exec open-next build",
  vpc,
  warm: 0,
  link: [relationalDatabase, sourceImagesBucket],
  environment: {
    DB_HOSTNAME: relationalDatabase.host,
    DB_PORT: relationalDatabase.port.apply((v) => String(v)),
    ...
  },
  dev: {
    command: "turbo dev --filter=@workspace/myapp-web...",
    url: "http://localhost:3000/",
  },
});

pnpm exec sst --stage preview deploy works well in the terminal, as well as via pnpm run deploy with package.json having:

  "scripts": {
     ...
    "deploy": "pnpm exec sst --stage preview deploy",
    ...
  },

I would expect turbo run deploy to work too.

Actual behavior

Running turbo run deploy, even with --env-mode=loose, seems to make it such that environment variables are not available during the next app build process.

Unlike pnpm run deploy.

E.g. we edit the next config of the app mentioned above:

// next.config.mjs

import pkg from '@next/env';

const projectDir = process.cwd();
pkg.loadEnvConfig(projectDir);

const { SOME_ENV_FROM_SST } = process.env;

console.log("ENV is: ", SOME_ENV_FROM_SST)

The result is ENV is: undefined. It's correct via pnpm directly.

(by setting SOME_ENV_FROM_SST under environment for the next app config above).

To Reproduce

(covered above)

Additional context

No response

[anthonyshew]

Please provide a reproduction. --env-mode=loose should be allowing the variables into the runtime.

Have you also tried using the envMode key in turbo.json?

[n-batalha]

Thanks for reviewing! I will need some time to prepare a reproduction.

I tried with envMode in turbo.json.

But correct me if I'm wrong, that is about the passage of env vars in the first 2 arrows below right?

(user in terminal / CI/CD) --> turbo --> sst --> `next build`

Here the problem is in the last arrow.

sst deploy launches processes like next build and provides it with env vars on its own (not on the host system), but when sst deploy is launched via turbo, those are gone.

[anthonyshew]

I tried with envMode in turbo.json.

Did this work? That would indicate there's a turbo invocation elsewhere in the stack of CLI calls that didn't get the flag applied.

Here the problem is in the last arrow.

If you see the environment variables past the turbo boundary in your repository, then turbo isn't involved in the behavior you're seeing. If the variables are in the sst portion of your process stack, that would mean that Turborepo is no longer involved and wouldn't block the environment.

[n-batalha]

I tried with envMode in turbo.json.

Did this work?

Sorry I forgot to add that it didn't work.

If you see the environment variables past the turbo boundary in your repository, then turbo isn't involved in the behavior you're seeing. If the variables are in the sst portion of your process stack, that would mean that Turborepo is no longer involved and wouldn't block the environment.

Makes total sense. But treating these tools as black boxes, this issue only happens when sst deploy is launched through turbo (no variable changing on the sst side).

[anthonyshew]

I’m not sure how sst works or what it does to manipulate the environment. If it does something to mutate the environment at a weird moment, it could be causing problems. But I’m not sure how that would be happening or how that would work. The reproduction will help.

[n-batalha]

The reproduction will help

Attempt and instructions: https://github.com/n-batalha/turbo-repro-issue-10353/blob/main/README-error.md

[anthonyshew]

In that reproduction, I don't see any of:

• --env-mode=loose on the turbo invocation
• An envMode key in turbo.json
• An env key with the list of environment variables that should be passed into the task

Under these circumstances, what you're seeing is expected behavior.

[n-batalha]

@anthonyshew sorry, I added a commit doing these 3 and retested.

(it's partly because I interpreted those settings as impacting the first arrows, rather than the 3rd, but you're right)