Failed to pull the backend. - ENOENT: no such file or directory, realpath

[iyz91]

Before opening, please confirm:

• [X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
• [X] I have searched for duplicate or closed issues.
• [X] I have read the guide for submitting bug reports.
• [X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
• [X] I have removed any sensitive information from my code snippets and submission.

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

16.14.2

Amplify CLI Version

7.6.25

What operating system are you using?

Windows 11

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No manual changes made.

Amplify Categories

auth, storage, function, api

Amplify Commands

pull

Describe the bug

Project was previously worked on a different device. Setting this up on a new device for the first time created this issue.

The ENOENT error appears to involve a function lambda layer looking for the file/directory realpath in "function<layerName>\lib\nodejs\node_modules\browser-resolve\node_modules\resolve\test\pathfilter\deep_ref\node_modules\deep\deeper". It appears the sub-directory "deep_ref" does not exist but the sub-directories after that do exist.

Deleting any files related to amplify does not solve the issue.

Expected behavior

Amplify should be able to pull the project environment without error, including for any lambda layers.

Reproduction steps

Not sure if this will be reproducible for others as it appears to involve a lambda layer issue.

1. Using an existing backend with a lambda layer in the function category, pull an environment on a new device

GraphQL schema(s)

No response

Log output

Backend environment 'dev' found. Initializing...
Failed to pull the backend.
ENOENT: no such file or directory, realpath 'C:\...\amplify\.temp\#current-cloud-backend\function\<layerName>\lib\nodejs\node_modules\browser-resolve\node_modules\resolve\test\pathfilter\deep_ref\node_modules\deep\deeper'

Additional information

No response

[iyz91]

It seems that there's been numerous updates to the CLI since I last used it, such as for lambda layers and the gql transformer. At this point, I cannot follow the recommended steps to update lambda layers since I cannot run amplify push as the project cannot be properly initialized. What would be the recommended steps to resolve this?

[0afcode]

I am also experiencing the same issue on same version of cli. I also tried reverting as far back as 7.6.20 and still persisting. I have diagnosed that this is happening due to a windows limitation for file path length. My project pulls fine on a linux machine but does not work on windows anymore after we updated and installed more node packages in one of our lambda layers in our amplify project.

I tried enabling long file path on windows but that did not seem to help, however I was able to find a work around by moving the project root folder into a short path (ex: C:\MyProject..) and that allowed me to pull successfully.

[josefaidt]

To add on to @0afcode 's comment, there appears to be a MAX_PATH variable we can set to increase the path constraint from 260. First, can you confirm whether the printed path exceeds 260 characters, and if so can you try increasing this constraint via the variable or move the project up a few levels?

reference: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd

[iyz91]

@josefaidt The path is indeed quite long but long paths were already enabled in W11 so not sure why the issue still happened. I've decided to reinitialize the amplify stack since we had a lot of refactoring to do anyways and to take advantage of the toolchain changes. That exact same lambda layer with the same folder depth in the new stack does not have the issue, so it doesn't actually appear to be an issue with the path length limit in the OS, it's something with the framework. Again, looking at my original post above, the framework added a subdirectory that doesn't exist, "deep_ref" so maybe the issue lies there.

[josefaidt]

Hey @iyz91 :wave: unfortunately I have not been able to reproduce this by using a Lambda Layer with browser-resolve as a dependency among many others. I have a few follow-up questions:

• was the layer modified in the cloud?
• can you post the full path of the npm module that was causing issues? the path in the original post seems obfuscated
• how was this project initialized on the separate device?

[iyz91]

@josefaidt

• The layer was not modified in the cloud
• The full path would have been something akin to C:\Users\Abc\Documents\Projects\Abcdefgh\Abcdefghijk\Code\Abcdefghijklmn\amplify\backend\function<layerName>\lib\nodejs\node_modules\browser-resolve\node_modules\resolve\test\pathfilter\deep_ref\node_modules\deep\deeper
• The project was first pulled from a repo and then I ran amplify pull

It's important to note I think that the lambda layer was created using an older CLI version before the lambda layer changes that I linked in the first comment.

[josefaidt]

Hey @iyz91 thanks for the clarification!

The project was first pulled from a repo and then I ran amplify pull

Does this issue occur each time we attempt to clone the repo and pull? Does this also occur with init?

[iyz91]

@josefaidt As far as I remember it occurred with both pull and init.

[josefaidt]

Hey @iyz91 :wave: unfortunately I have not been able to reproduce this issue. Have you continuously experienced this? Re-reading this note from an earlier reply

I've decided to reinitialize the amplify stack since we had a lot of refactoring to do anyways and to take advantage of the toolchain changes. That exact same lambda layer with the same folder depth in the new stack does not have the issue

Was this project recreated from scratch or was it pulled into an empty directory?

[iyz91]

@josefaidt The issue came up when the project repo was pulled into another device and then amplify pull was run. I remember trying different approaches to initializing it etc. (can't say for certain what they were at this point) but everything gave me the same error.

When I decided to recreate the stack from scratch I didn't have this problem at all even with the exact same folder structure and nesting depth. The key difference though is the recreation was done using the latest CLI version, while the earlier errored attempts were using the latest CLI on an amplify backend that was previously created probably 2 major CLI versions ago.

[josefaidt]

Hey @iyz91 given my inability to reproduce this issue and since you were able to mitigate the issue I will close this for now, however if this occurs again please reply back to this thread and we can re-open to investigate further.

[iyz91]

Hey @josefaidt Looks like this issue popped up again... I created a seperate repo for an admin interface on our project and following the instructions here. This gave the error below:

Failed to pull the backend.
🛑 ENOENT: no such file or directory, realpath 'C:\Users\<user>\Documents\Projects\stuff\stuff\Code\stuff\amplify\.temp\#current-cloud-backend\function\stuffv3GeneratorLayer\lib\nodejs\node_modules\browser-resolve\node_modules\resolve\test\pathfilter\deep_ref\node_modules\deep\deeper'

To be clear, this was done in a brand new repo, pulling from an already existing and pushed amplify backend that was created with the amplify CLI v8.

[iyz91]

@josefaidt Seems to be a bug with node's fs.promises.realpath, see https://github.com/actions/upload-artifact/issues/240 and https://github.com/nodejs/node/issues/39721

[josefaidt]

Hey @iyz91 great callout on the Node issue. I will re-open this and mark this as a dependency issue

[kylekirkby]

@josefaidt - any update on this? Got hit by this again today :/ The weird thing is that everything has been going great with the CLI and then suddenly this old issue is back.

Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/

ZipExtractFault: Failed to extract zip file:
    at extractZip (C:\snapshot\amplify-cli\build\node_modules\@aws-amplify\amplify-provider-awscloudformation\lib\zip-util.js:43:11)
    at async Object.run (C:\snapshot\amplify-cli\build\node_modules\@aws-amplify\amplify-provider-awscloudformation\lib\initialize-env.js:44:25)

ENOENT: no such file or directory, realpath 'C:\Users\kylek\Documents\XXX\amplify\.temp\#current-cloud-backend\function\XXXXXXX\lib\nodejs\node_modules\@aws-sdk\s3-presigned-post\node_modules\@aws-sdk\credential-provider-ini\node_modules\@smithy\shared-ini-file-loader\node_modules\@smithy'
Error: ENOENT: no such file or directory, realpath 'C:\Users\kylek\Documents\XXX\amplify\.temp\#current-cloud-backend\function\XXXXXXX\lib\nodejs\node_modules\@aws-sdk\s3-presigned-post\node_modules\@aws-sdk\credential-provider-ini\node_modules\@smithy\shared-ini-file-loader\node_modules\@smithy'

[naedx]

I'm also having this issue now.

PullBackendFault: Failed to pull the backend.
    at attachBackend (C:\snapshot\amplify-cli\build\node_modules\@aws-amplify\cli-internal\lib\attach-backend.js:55:15)
    at async Object.run (C:\snapshot\amplify-cli\build\node_modules\@aws-amplify\cli-internal\lib\commands\pull.js:64:9)
    at async Object.executeAmplifyCommand (C:\snapshot\amplify-cli\build\node_modules\@aws-amplify\cli-internal\lib\index.js:194:9)
    at async executePluginModuleCommand (C:\snapshot\amplify-cli\build\node_modules\@aws-amplify\cli-internal\lib\execution-manager.js:139:5)
    at async executeCommand (C:\snapshot\amplify-cli\build\node_modules\@aws-amplify\cli-internal\lib\execution-manager.js:37:9)
    at async Object.run (C:\snapshot\amplify-cli\build\node_modules\@aws-amplify\cli-internal\lib\index.js:121:5)

ENOENT: no such file or directory, realpath 'C:\Users\windowsuser\Projects\XXXXXXXXXXX\packages\lib_package_name_xxxxxxxxx\amplify\.temp\#current-cloud-backend\function\apinameLayer1473473xxxxxxxxxxxxxxxxxxxxx\lib\nodejs\node_modules\@aws-crypto\ie11-detection\node_modules\tslib\test\validateModuleExportsMatchCommonJS'  
Error: ENOENT: no such file or directory, realpath 'C:\Users\windowsuser\Projects\XXXXXXXXXXX\packages\lib_package_name_xxxxxxxxx\amplify\.temp\#current-cloud-backend\function\apinameLayer1473473xxxxxxxxxxxxxxxxxxxxx\lib\nodejs\node_modules\@aws-crypto\ie11-detection\node_modules\tslib\test\validateModuleExportsMatchCommonJS'

Amplify version: 12.10.1 (latest)

[THARUE]

Having this issue as well. Amplify version: 12.11.1

[THARUE]

no such file or directory, realpath 'DynamodbStreamsLambdaLayer\lib\nodejs\node_modules@aws-sdk\client-cognito-identity-provider\node_modules@aws-crypto\sha256-browser\node_modules\tslib\test\validateModuleExportsMatchCommonJS''