Missing '$amplify/env/xy' upon deploying

[hnrkdwl]

Environment information

System:
  OS: macOS 14.5
  CPU: (8) arm64 Apple M3
  Memory: 124.56 MB / 24.00 GB
  Shell: /bin/zsh
Binaries:
  Node: 20.14.0 - /opt/homebrew/opt/node@20/bin/node
  Yarn: undefined - undefined
  npm: 10.7.0 - /opt/homebrew/opt/node@20/bin/npm
  pnpm: undefined - undefined
NPM Packages:
  @aws-amplify/backend: 1.0.3
  @aws-amplify/backend-cli: 1.0.4
  aws-amplify: 6.3.7
  aws-cdk: 2.147.0
  aws-cdk-lib: 2.147.0
  typescript: 5.5.2
AWS environment variables:
  AWS_STS_REGIONAL_ENDPOINTS = regional
  AWS_NODEJS_CONNECTION_REUSE_ENABLED = 1
  AWS_SDK_LOAD_CONFIG = 1
No CDK environment variables

Description

Simplified structure

amplify
├───data
│      ├───resource.ts
│      └───test-mutation-handler
│                    └───testMutationHandler.ts
├───functions
│      └───daily-function
│                    ├───handler.ts
│                    └───resource.ts
├───backend.ts
└───tsconfig.json

Code

amplify/data/resource.ts

import { type ClientSchema, a, defineData, defineFunction } from '@aws-amplify/backend';
...
import { dailyFunction } from '../functions/daily-function/resource';

const testMutationHandler = defineFunction({
  entry: './test-mutation-handler/testMutationHandler.ts'
});

const schema = a.schema({
...
  testMutation: a.mutation()
  .arguments({
    arg1: a.string().required(),
    arg2: a.string().required()
  })
  .returns(a.string())
  .authorization(allow => allow.authenticated())
  .handler(a.handler.function(testMutationHandler)),
...
})
.authorization(allow => [
  allow.resource(testMutationHandler),
  allow.resource(dailyFunction)
...
]);

amplify/data/test-mutation-handler/testMutationHandler.ts

import type { Schema } from '../resource';
import { env } from "$amplify/env/testMutationHandler";
...

Amplify.configure({...})
const client = generateClient<Schema>({
  authMode: "iam"
});
export const handler: Schema["testMutation"]["functionHandler"] = async (event, context) => {
...
};

amplify/functions/daily-function/handler.ts

import type { Handler } from "aws-lambda";
import type { Schema } from '../../data/resource';
import { env } from "$amplify/env/daily-function";
...

Amplify.configure({...})
const client = generateClient<Schema>({
  authMode: "iam"
});
export const handler: Handler = async (event, context) => {
...
};

amplify/functions/daily-function/resource.ts

import { defineFunction } from '@aws-amplify/backend';

export const dailyFunction = defineFunction({
  name: 'daily-function',
  environment: {
    test: "2"
  }
});

amplify/backend.ts

import { defineBackend } from '@aws-amplify/backend';
import * as events from 'aws-cdk-lib/aws-events';
import * as targets from 'aws-cdk-lib/aws-events-targets';
import { auth } from './auth/resource';
import { data } from './data/resource';
import { storage } from './storage/resource';
import { dailyFunction } from './functions/daily-function/resource';

const backend = defineBackend({
  auth,
  data,
  storage,
  dailyFunction
});

const rule = new events.Rule(
  backend.dailyFunction.resources.lambda,
  'Rule',
  {
    description: 'Rule that triggers the lambda function every day at 00:00',
    schedule: events.Schedule.cron({minute: "0", hour: "0"}),
  },
);

rule.addTarget(
  new targets.LambdaFunction(backend.dailyFunction.resources.lambda),
);

amplify/tsconfig.json

{
  "compilerOptions": {
    "target": "es2022",
    "module": "es2022",
    "moduleResolution": "bundler",
    "resolveJsonModule": true,
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "skipLibCheck": true,
    "paths": {
      "$amplify/*": [
        "../.amplify/generated/*"
      ]
    }
  }
}

Problem

What is expected to happen

The necessary environment files are correctly generated using the sandbox for local development as well as in the cloud-backend upon deploying.

What actually happens

Using the sandbox everything works as expected. However, upon deploying, the build-process fails with the following statements:

2024-07-06T12:32:57.614Z [INFO]: # Executing command: npx ampx pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID
2024-07-06T12:33:08.889Z [INFO]:
2024-07-06T12:33:15.365Z [INFO]: amplify/data/test-mutation-handler/testMutationHandler.ts(2,21): error TS2307: Cannot find module '$amplify/env/testMutationHandler' or its corresponding type declarations.
amplify/functions/daily-function/handler.ts(3,21): error TS2307: Cannot find module '$amplify/env/daily-function' or its corresponding type declarations.

Although there have already been similar issues (e.g. #1374), those seem to be connected to developing a Next.js app, as where I am developing a swift app (hence there is no tsconfig.json file at the root of the project).

One last side note: I have also implemented a PostConfirmationTriggerHandler according to the documentation which also uses import { env } from "$amplify/env/post-confirmation";. Strangely, this seems to be the only case for which there is no error in the build-console after deploying.

[ykethan]

Hey @hnrkdwl, thank you for reaching out. Tried reproducing the issue on sandbox and console build but did not observe this error message. Did observe packages are a bit out of date.

@aws-amplify/backend: 1.0.3
@aws-amplify/backend-cli: 1.0.4

Could you try upgrading to the latest versions and retry the build.

"@aws-amplify/backend": "^1.0.4",
"@aws-amplify/backend-cli": "^1.1.0",

[Jay2113]

Hi @hnrkdwl 👋 , just checking in to see if you are still experiencing this issue with the latest package versions?

[hnrkdwl]

Hi, thank you for your answers! Unfortunately upgrading to the latest packages did not resolve the issue. The error has not changed.

[amalhub]

I had a similar error and resolved it like below: When you do

npm run build

in your project, are you getting this same error? If you do, then probably you are missing the tsconfig exclude in your nextjs tsconfig.json file:

"exclude": ["node_modules", ".amplify", "./amplify"]

Make sure you modify the tsconfig of the nextjs project and not in the Amplify directory project. Insights: During the deployment, the whole project gets build and nextjs doesn't know how to build Amplify modules (directories). Amplify modules (backend) gets build separately by the AWS services. So it tends to throw a similar error you are experiencing right now. Excluding those Amplify derectries will ignore them during the nextjs build.

Hope this helps. :) Let me know.

[ErlendHer]

We have the same issue. Next project. npm run build runs fine (also it fails in the pipeline-deploy step), not the frontend build step.

[allxie]

Experiencing the same issue with an amplify/expo app.

Some details: Amplify-backend: 1.0.2 (later upgraded to 1.0.4) Using Expo, React Native, EAS. Not Next. No type errors locally. I have nothing in frontend build script since the frontend is a mobile app that I'm not hosting on amplify. I also have a post-confirmation trigger which is not erroring.

I included some logs above the type error about dotenvx. I'd added dotenvx to be able to use env variables locally in the same PR that broke the rest of the build. The logs are a little curious -- it says I'm injecting 0 variables when I am supposed to have 2.

Logs

2024-07-22T23:41:08.317Z [INFO]: # Executing command: npx ampx pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID
2024-07-22T23:41:16.111Z [INFO]: [[email protected]] injecting env (0) from .env
2024-07-22T23:41:19.053Z [INFO]:
2024-07-22T23:41:24.598Z [INFO]: amplify/functions/utils/dbUtils/dbConnect.ts(3,21): error TS2307: Cannot find module '$amplify/env/api-function' or its corresponding type declarations.

My build was working (though not yet with env variables) until a commit where I did the following:

• Added import { env } from '$amplify/env/api-function'; and env usage to a db util file.
• Removed amplify_outputs* from gitignore (to enable EAS to pick it up) (but did not commit amplify_outputs itself).
• Added an env.d.ts file with two variables, extending env.
• Added environment key with two variables to my functions/api-function/resource.ts defineFunction.
• I created two environment variables in the amplify console.
• added dotenvx

What I've tried:

• I added the following to my tsconfig.json, and also to my amplify/tsconfig.json. No change.
  • "exclude": ["node_modules", ".amplify", "./amplify","amplify/**/*", ".amplify/**/*"],
• Checked to make sure that amplify console had all env variables correct for the correct branch/environment.
• Realized I'd left the second variable out of the Hosting > Build Settings > amplify.yml and added them to be echoed into .env. This resulted in dotenvx saying it was injecting 1 instead of 0 variables -->
  • [[email protected]] injecting env (0) from .env --> [[email protected]] injecting env (1) from .env
    • Makes me wonder why it's saying 0 and 1 instead of 2. I have 2. Is there something clearing out .env? At some point it went back to 0. Not sure what changed.
• Tried adding "$amplify/*": ["./.amplify/generated/*"] to paths in root tsconfig.json. No change. FYI I already have "$amplify/*": ["../.amplify/generated/*"] in the amplify/tsconfig.json.
• Upgraded:
  • "@aws-amplify/backend": "^1.0.4",
  • "@aws-amplify/backend-cli": "^1.1.0"
• Tried adding an import of {env} to lambda function handler file instead of a more nested util --> importing into that file breaks too.
  • amplify/functions/api-function/handler.ts(7,21): error TS2307: Cannot find module '$amplify/env/api-function' or its corresponding type declarations.
  • But amplify/env.d.ts has the import of {env} and doesn't error. I added an import of something that really shouldn't exist into amplify/env.d.ts and it doesn't break/log, so we just must not be getting there in the build process.
  • Added include env.d.ts to amplify/tsconfig.json ...stabbing in the dark 😅 ... much more exciting errors, at least.

[ykethan]

Hey folks, wanted to provide to provide some information on the error Cannot find module '$amplify/env/api-function' or its corresponding type declarations.

If the error is occurring in your frontend build step, refer to the documentation here: https://docs.amplify.aws/react/build-a-backend/troubleshooting/cannot-find-module-amplify-env/

If the error is occurring in your backend build step:

1. Ensure the project is utilizing the latest backend packages
2. Check the env import path used in the function, for example if the name of the function was changed the path may be referring to an older version of the generated file.

On updating the name to say-hello -> say-hello1 the .amplify/generated generated a new env file

[allxie]

@ykethan

I have upgraded:

• "@aws-amplify/backend": "^1.0.4",
• "@aws-amplify/backend-cli": "^1.2.0",
• (and updated package-lock.json)

I can confirm that I locally have:

• Sandbox working
• generated/env/api-function.ts
  • this contains:

type AmplifyBackendEnvVars = {
  MY_VAR_ONE: string;
  MY_VAR_TWO: string;
};

• project/amplify/functions/api-function/resource.ts as follows:

import { defineFunction } from "@aws-amplify/backend";
import * as dotenvx from "@dotenvx/dotenvx";
dotenvx.config();

export const myApiFunction = defineFunction({
  name: "api-function",
  environment: {
    MY_VAR_ONE: process.env.MY_VAR_ONE,
    MY_VAR_TWO: process.env.MY_VAR_TWO,
  },
});

amplify/env.d.ts

import { env } from "$amplify/env/api-function";

declare namespace NodeJS {
  export interface ProcessEnv extends env {
    MY_VAR_ONE: string;
    MY_VAR_TWO: string;
  }
}

• the util that my function handlers/utils call contains:
  • import { env } from "$amplify/env/api-function";
  • ... env.MY_VAR_ONE ...

Can confirm that building main branch on amplify console still errors with:

2024-07-23T18:19:21.305Z [INFO]: # Executing command: npx ampx pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID
2024-07-23T18:19:29.430Z [INFO]: [[email protected]] injecting env (0) from .env
2024-07-23T18:19:32.476Z [INFO]:
2024-07-23T18:19:38.144Z [INFO]: amplify/functions/utils/dbUtils/dbConnect.ts(3,21): error TS2307: Cannot find module '$amplify/env/api-function' or its corresponding type declarations.

amplify.yml (in console)

version: 1
backend:
    phases:
        build:
            commands:
                - echo "MY_VAR_ONE=$MY_VAR_ONE" >> .env
                - echo "MY_VAR_TWO=$MY_VAR_TWO" >> .env
                - cat .env
                - 'npm ci --cache .npm --prefer-offline'
                - 'npx ampx pipeline-deploy --branch $AWS_BRANCH --app-id $AWS_APP_ID'
frontend:
  phases:
    # IMPORTANT - Please verify your build commands
    build:
      commands: 
          - echo "No Frontend Builds"
  artifacts:
    # IMPORTANT - Please verify your build output directory
    baseDirectory: /
    files:
      - '**/*'
  cache:
    paths: []

amplify/tsconfig.json

{
  "compilerOptions": {
    "target": "es2022",
    "module": "es2022",
    "moduleResolution": "bundler",
    "resolveJsonModule": true,
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "skipLibCheck": true,
    "paths": {
      "$amplify/*": ["../.amplify/generated/*"]
    }
  }
}

project root tsconfig.json

{
  "extends": "expo/tsconfig.base",
  "compilerOptions": {
    "strict": true
  },
  "include": ["**/*.ts", "**/*.tsx", ".expo/types/**/*.ts", "expo-env.d.ts"],
  "paths": {
    "@/*": ["./*"]
  },
  "exclude": ["node_modules", ".amplify", "./amplify"] // fails with or without this line
}

[ErlendHer]

In our case the deploy failed because of an environment variable that wasn't set in the console. This caused an error since process.env["VARIABLE_NAME"] was undefined. Apparently pipeline-deploy doesn't bubble up any errors when running the compiled JavaScript to deploy the backend, instead it just failed to generate the env files and that was the only logs we were seeing.

One way to debug if a JavaScript runtime error during building the backend is causing the issue is to simply commit the .amplify/generated/env folder and read the deploy logs. If it fails during build now you will be able to see the error message that is causing the issue.

[allxie]

I committed .amplify/generated/env and then main deployed fine, no errors. I imagine that implies that .amplify/generated/env is not getting generated correctly during build?

I'm investigating whether dotenvx is getting in the way, since dotenvx is saying that it is importing (0) variables. The (very new!!) documentation says to use dotenvx locally for env variables, but doesn't say anything about how to make sure it doesn't get in the way for deployments.

[allxie]

After deleting the local .amplify/generated/env folder, the generated folder isn't re-generating all of the files in .ampify/generated/env (but is generating some of the files) and I'm now getting the same type error locally.

Update later: They are now generating again locally. Not sure why this is happening intermittently.

[ykethan]

Hey @allxie @ErlendHer, thank you for the information. Tried reproducing the issue using the dotenvx package and other permutations but did not observe the error message. Could you provide us a minimal reproduction steps or sample repository with the reproduction this would help us in diving deeper into this issue.

[Jay2113]

@allxie @ErlendHer 👋 , just checking in to see if you are still blocked? If so, can you share minimal reproduction steps or a sample repository as it can help us dive deeper into the issue.

[nhilbert]

Summary

Check if you have env imports in other files except for the actual handler. This could pose a problem.

---

I experienced the same issue today. Sorry for not being able to describe steps for reproduction.

I did quite a few changes in sandbox, then deployed to dev branch. I good the error on the cloud deployment. I deleted the sandbox (where I had no errors), restarted the sandbox and got the same problem. Deleted .amplify and not all files were created for some reason.

I then commented out the missing env file imports, removed all references (replaced with dummies), then the sandbox stabilized again and the env file was created (locally). Now the cloud build ran successfully (it actually failed, but 5 minutes later at the frontend stage)

Reintroduction of the imports locally did not lead to errors anymore in the sandbox, but once I deleted the .amplify I had the same problem.

As excpected the cloud build also failed with the same error.

I then removed the imports of the env from other (utility) files to which I had moved some helper functions. This also seemed to help in the sandbox.

Now the cloud build seems to run successfully too.

Maybe it's not possible to import env from a lambda in another file? (Seems a bad idea, now that I think about it as I may be transporting envs out of their scope.) Any thoughts on this? Maybe this helps others too.

[allxie]

The workaround of committing the .amplify/generated/env files worked for me, and I needed to move on to other issues. Hoping to come back to resolve this soon, since it doesn't seem good to have those in git.

Like, @nhilbert, I suspected that it was an issue with importing in other files from my lambdas so I put everything back in the lambdas, but that didn't solve the issue. The issue for me is that the .amplify/env files just don't reliably get generated in the branch environments on launch, leading to them being not findable. They don't always get generated locally, either, but I've had fewer problems locally/with sandbox.

[ErlendHer]

@allxie @ErlendHer 👋 , just checking in to see if you are still blocked? If so, can you share minimal reproduction steps or a sample repository as it can help us dive deeper into the issue.

Hi. Yes we're still being blocked. Unable to come up with a MVP unfortunately. We haven't been able to identify what is causing the issue.

We're currently unable to deploy unless we commit the generated env files.

We're also seeing the issue in the sandbox at times. To resolve it we sometimes need to manually add shims for the missing files (deleting the .amplify folder and restarting sandbox doesn't work), or uncomment the imports from the lambda, deploy that and then re-add the imports after a successful deploy.

Edit: In this instance an error that didn't bubble up was not the issue (as deploying succeeded when we included the generated shims)

[ErlendHer]

@allxie @ErlendHer 👋 , just checking in to see if you are still blocked? If so, can you share minimal reproduction steps or a sample repository as it can help us dive deeper into the issue.

Hi. Yes we're still being blocked. Unable to come up with a MVP unfortunately. We haven't been able to identify what is causing the issue.

We're currently unable to deploy unless we commit the generated env files.

We're also seeing the issue in the sandbox at times. To resolve it we sometimes need to manually add shims for the missing files (deleting the .amplify folder and restarting sandbox doesn't work), or uncomment the imports from the lambda, deploy that and then re-add the imports after a successful deploy.

Edit: In this instance an error that didn't bubble up was not the issue (as deploying succeeded when we included the generated shims)

Actually we figured it out just now. Turns out one of our function handlers imported a constant declared in a different function handler. So when the constant was imported, the env file was also attempted to be imported (but this is before the deploy step had generated shims for that function so it failed with the error). This is also why manually adding the shims worked.

Removing any cross-function dependencies worked (which is probably a code smell regardless)

[ykethan]

Marking this as bug, it appears the initial synth is not exposing the error message and fails to generate a env file which causes typescript check to then fail on this.

[Amplifiyer]

Simple reproduction steps:

1. Delete the .amplify/generated directory or start with a new app that has never been deployed.
2. Update the code to introduce an esbuild error such as

 export const node16Func = defineFunction({
    name: "node16Function",
    entry: "./func-src/handler_node16.ts",
    environment: {
      TEST_VARS: process.env['non_existent_var']!,
    }
  });

3. Run sandbox or deploy to pipeline and observe the error mentioned in introduction.

[yamatatsu]

In my case, I was able to solve this issue by either adding "type": "module" to the root package.json file or by creating a new package.json with { "type": "module" } inside the amplify/ directory. And not needed to modify tsconfig.json and to put comment @ts-noncheck.

I discovered this solution by checking the behavior of npm create amplify.