serverless-python-requirements
serverless-python-requirements copied to clipboard
Cannot Deploy; EMFILE: too many open files
Are you certain it's a bug?
- [X] Yes, it looks like a bug
Is the issue caused by a plugin?
- [X] It is not a plugin issue
Are you using the latest version?
- [X] Yes, I'm using the latest version
Is there an existing issue for this?
- [X] I have searched existing issues, it hasn't been reported yet
Issue description
When running serverless deploy
through CLI, I get some variation of the below error.
Serverless Error ----------------------------------------
Cannot read file node_modules\rxjs\operator\groupBy.d.ts due to: EMFILE: too many open files, open 'C:\Users\gcper\Code\190\Website\lambda\node_modules\rxjs\operator\groupBy.d.ts'
I cannot deploy to my lambda functions.
Service configuration (serverless.yml) content
service: team190
frameworkVersion: '2'
provider:
name: aws
runtime: python3.8
iamRoleStatements:
- Effect: "Allow"
Action:
- dynamodb:Query
- dynamodb:Scan
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:UpdateItem
- dynamodb:DeleteItem
Resource: "*"
plugins:
- serverless-python-requirements
custom:
pythonRequirements:
dockerizePip: non-linux
functions:
login:
handler: handler/handler.login
choose:
handler: handler/handler.assign_role
request:
handler: handler/student_handler.request
get_requests:
handler: handler/uber_handler.get_requests
get_records:
handler: handler/handler.get_records
confirm_requests:
handler: handler/uber_handler.confirm_requests
create_poll:
handler: handler/uber_handler.create_poll
get_users:
handler: handler/uber_handler.get_all_users
create_event:
handler: handler/uber_handler.create_event
get_pending_requests:
handler: handler/student_handler.get_pending_requests
delete_pending_requests:
handler: handler/student_handler.delete_pending_requests
Command name and used flags
serverless deploy
Command output
Serverless: Running "serverless" installed locally (in service node_modules)
Serverless: Deprecation warning: Resolution of lambda version hashes was improved with better algorithm, which will be used in next major release.
Switch to it now by setting "provider.lambdaHashingVersion" to "20201221"
More Info: https://www.serverless.com/framework/docs/deprecations/#LAMBDA_HASHING_VERSION_V2
Serverless: Generated requirements from C:\Users\gcper\Code\190\Website\lambda\requirements.txt in C:\Users\gcper\Code\190\Website\lambda\.serverless\requirements.txt...
Serverless: Using static cache of requirements found at C:\Users\gcper\AppData\Local\UnitedIncome\serverless-python-requirements\Cache\b729430ab9c5421667932cf132fc2b7c75df8e7e25338978deedaf75b5fec293_slspyc ...
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless Error ----------------------------------------
Cannot read file node_modules\rxjs\operator\groupBy.d.ts due to: EMFILE: too many open files, open 'C:\Users\gcper\Code\190\Website\lambda\node_modules\rxjs\operator\groupBy.d.ts'
Get Support --------------------------------------------
Docs: docs.serverless.com
Bugs: github.com/serverless/serverless/issues
Issues: forum.serverless.com
Your Environment Information ---------------------------
Operating System: win32
Node Version: 14.16.1
Framework Version: 2.64.1 (local)
Plugin Version: 5.5.0
SDK Version: 4.3.0
Components Version: 3.17.1
Environment information
Serverless: Running "serverless" installed locally (in service node_modules)
Framework Core: 2.64.1 (local)
Plugin: 5.5.0
SDK: 4.3.0
Components: 3.17.1
@GrantPerkins great thanks for reporting. Can you post the output but when command is run with SLS_DEBUG=*
flag?
Hi i have the same problem and to my knowledge it is not dependend on config files. This error is really hard to reproduce, and the higher chance to do so is when node_module is filled with packages. My coleagues running the same command on the same repo does not have this problem. But unfortunetly it does exists and its stops me from developing :( For eg. part of my package.json file:
"devDependencies": {
"@babel/core": "^7.15.5",
"@babel/preset-env": "^7.15.6",
"@babel/register": "^7.15.3",
"prettier": "^2.4.1",
"serverless-lift": "^1.9.0"
},
"dependencies": {
"@sentry/serverless": "^6.13.3",
"aws-sdk": "^2.997.0",
"node-fetch": "^2.6.5",
"node-fetch-retry": "^2.0.0",
"yup": "^0.32.11"
}
This is my output after running sls deploy -s prod
:
Serverless: To ensure safe major version upgrades ensure "frameworkVersion" setting in service configuration (recommended setup: "frameworkVersion: ^2.64.1")
Serverless: Load command interactiveCli
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command config:tabcompletion
Serverless: Load command config:tabcompletion:install
Serverless: Load command config:tabcompletion:uninstall
Serverless: Load command create
Serverless: Load command install
Serverless: Load command package
Serverless: Load command deploy
Serverless: Load command deploy:function
Serverless: Load command deploy:list
Serverless: Load command deploy:list:functions
Serverless: Load command invoke
Serverless: Load command invoke:local
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command metrics
Serverless: Load command print
Serverless: Load command remove
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command slstats
Serverless: Load command plugin
Serverless: Load command plugin
Serverless: Load command plugin:install
Serverless: Load command plugin
Serverless: Load command plugin:uninstall
Serverless: Load command plugin
Serverless: Load command plugin:list
Serverless: Load command plugin
Serverless: Load command plugin:search
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command upgrade
Serverless: Load command uninstall
Serverless: Load command lift
Serverless: Load command lift:eject
Serverless: Load command HooksQueue:logs
Serverless: Load command HooksQueue:send
Serverless: Load command HooksQueue:failed
Serverless: Load command HooksQueue:failed:purge
Serverless: Load command HooksQueue:failed:retry
Serverless: Load command login
Serverless: Load command logout
Serverless: Load command generate-event
Serverless: Load command test
Serverless: Load command dashboard
Serverless: Load command output
Serverless: Load command output:get
Serverless: Load command output:list
Serverless: Load command param
Serverless: Load command param:get
Serverless: Load command param:list
Serverless: Load command studio
Serverless: Skipping variables resolution with old resolver (new resolver reported no more variables to resolve)
Serverless: Configuration warning at 'constructs.HooksQueue': unsupported configuration format
Serverless:
Serverless: Learn more about configuration validation here: http://slss.io/configuration-validation
Serverless:
Serverless: Invoke deploy
Serverless: Invoke package
Serverless: Invoke aws:common:validate
Serverless: Invoke aws:common:cleanupTempDir
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Excluding development dependencies...
Serverless: Excluding development dependencies...
Serverless Error ----------------------------------------
ServerlessError: Cannot read file node_modules\@types\lodash\fp\thru.d.ts due to: EMFILE: too many open files, open 'C:\PRACA_STUDIA\PRACACODEWAVE\webhooks-execution\node_modules\@types\lodash\fp\thru.d.ts'
at C:\Users\DominioPianino\AppData\Roaming\nvm\v15.0.1\node_modules\serverless\lib\plugins\package\lib\zipService.js:126:15
at tryCatcher (C:\Users\DominioPianino\AppData\Roaming\nvm\v15.0.1\node_modules\serverless\node_modules\bluebird\js\release\util.js:16:23)
at Promise._settlePromiseFromHandler (C:\Users\DominioPianino\AppData\Roaming\nvm\v15.0.1\node_modules\serverless\node_modules\bluebird\js\release\promise.js:547:31)
at Promise._settlePromise (C:\Users\DominioPianino\AppData\Roaming\nvm\v15.0.1\node_modules\serverless\node_modules\bluebird\js\release\promise.js:604:18)
at Promise._settlePromise0 (C:\Users\DominioPianino\AppData\Roaming\nvm\v15.0.1\node_modules\serverless\node_modules\bluebird\js\release\promise.js:649:10)
at Promise._settlePromises (C:\Users\DominioPianino\AppData\Roaming\nvm\v15.0.1\node_modules\serverless\node_modules\bluebird\js\release\promise.js:725:18)
at _drainQueueStep (C:\Users\DominioPianino\AppData\Roaming\nvm\v15.0.1\node_modules\serverless\node_modules\bluebird\js\release\async.js:93:12)
at _drainQueue (C:\Users\DominioPianino\AppData\Roaming\nvm\v15.0.1\node_modules\serverless\node_modules\bluebird\js\release\async.js:86:9)
at Async._drainQueues (C:\Users\DominioPianino\AppData\Roaming\nvm\v15.0.1\node_modules\serverless\node_modules\bluebird\js\release\async.js:102:5)
at Immediate.Async.drainQueues [as _onImmediate] (C:\Users\DominioPianino\AppData\Roaming\nvm\v15.0.1\node_modules\serverless\node_modules\bluebird\js\release\async.js:15:14)
at processImmediate (node:internal/timers:462:21)
Get Support --------------------------------------------
Docs: docs.serverless.com
Bugs: github.com/serverless/serverless/issues
Issues: forum.serverless.com
Your Environment Information ---------------------------
Operating System: win32
Node Version: 15.0.1
Framework Version: 2.64.1
Plugin Version: 5.5.0
SDK Version: 4.3.0
Components Version: 3.17.1
and when there are some fixes for a linux based environment, but it is hard to achieve on a windows. According to: https://stackoverflow.com/questions/8965606/node-and-error-emfile-too-many-open-files
Note: File that is shown in a error output is not always the same one.
@DominikIlski we have EMFILE prevention baked in. Still it cannot handle eventual sync fs calls which we practice only when reading user configuration.
Other corner case scenario, where eventually EMFILE may sneak unhadled, is if any shared dependency between serverless
and .e.g @servleress/components
requires fs
and attaches to it's functions in module initialization.
If there is such scenario anywhere, then this will be fixed with next release (as with https://github.com/serverless/serverless/pull/10131 we've ensured to not require other CLI's modules if they're not run)
I could deploy on my PC but not on the laptop. The next day, without making any updates, no changes, I can no longer deploy both. I get the same "EMFILE: too many open files" error
Today updated nodejs, serverless, and still with issue.
FWIW - I was/am experiencing this with Python. I created a virtual environment for each Lambda handler - and it looks like serverless is including all of the /Lib
and /Scripts
directories for each virtual environment, thereby inflating the file count needed to package.
Adding exclusions "solved" this for me.
I just also ran into this while working on Windows 10 randomly.
Nothing changed on my system or project except for an npm audit fix
that upgraded a single package.
I tried reinstalling my packages from scratch but had no joy.
Framework Core: 2.71.0 (local) Plugin: 5.5.3 SDK: 4.3.0 Components: 3.18.1
I ended up using [email protected] and it solved the problem. It also dramatically decreased build time. I'm not a fan of having the extra dependency though, but it does seem to be a viable work around.
I've moved this issue to https://github.com/serverless/serverless-python-requirements repository, as it seems specific to this plugin
@medikoo I don't believe that's the case - this issue can be encountered without having serverless-python-requirements
installed. It seems more closely related to the combination of serverless
with a virtual environment in the directory being packaged.
@FrodoBaggyMcBaggins can you create a sample reproduction case, that doesn't involve any plugins, and provide a link?
@medikoo - I can guarantee it's not specific to the serverless-python-requirements plugin (I'm not using that one at all). I can't rule out other plugins having similar issues though, as I'm using:
plugins:
- serverless-s3-sync
- serverless-offline
- serverless-export-env
@jvanalst are you capable to prepare a reproduction test case that doesn't involve any plugins? Or if not, can you single out which plugin causes that?
@medikoo Checking now if I can replicate it if I roll back my project a few commits...
Yup... replicable. And I can trigger it without any Serverless Plugins at all:
org: bananaweb
app: bananarego
service: ${file(./config.js):common.service}
frameworkVersion: '2'
provider:
name: aws
runtime: nodejs14.x
region: us-west-2
functions:
api:
handler: api/index.apiHandler
That api/index.js file can be empty, BTW. Nothing in there affects what I'm experiencing.
Currently looking through the package.json to see if anything in there affects it. Stripping the serverless-plugins from devDependencies (with clean npm install of course) ... no effect. Stripping ALL the other devDependencies... also no effect.
So... starting to strip out dependency dependencies.... also no effect. Well this is annoying.
{
"name": "bananarego",
"version": "0.0.0-alpha",
"private": true,
"dependencies": {
"serverless": "^2.65.0"
},
"devDependencies": {
}
}
Is it a problem with Serverless 2.65? If it is... 2.72.2 has the same exact bug in it. So that's not it probably. So far all we've managed to change is which node module shows up in the error message.
Serverless Error ---------------------------------------- Cannot read file node_modules\archiver-utils\node_modules\safe-buffer\index.js due to: EMFILE: too many open files, open 'C:\Users...\node_modules\archiver-utils\node_modules\safe-buffer\index.js'
I guess I'll start randomly removing files from the the project's base folder... Maybe it's trying to include something weird like include the frontend that's explicitly blocked in the original code.
Serverless Error ----------------------------------------
Cannot read file node_modules\aws-sdk\lib\services\cloudfront.d.ts due to: EMFILE: too many open files, open 'C:\Users...\node_modules\aws-sdk\lib\services\cloudfront.d.ts'
Okay... wiped the project down to package-lock.json, package.json, api/index.js, and node_modules containing only a serverless v2.27.2 install....
Serverless Error ----------------------------------------
Cannot read file node_modules\kafka-node\node_modules\debug.travis.yml due to: EMFILE: too many open files, open 'C:\Users...\node_modules\kafka-node\node_modules\debug.travis.yml'
Wow. Seriously? Still broken? Am I doing something crazy? Okay fine... blow away the package-lock and package.json just in case... so bare node_modules with an api/index.js (empty) and the serverless.yml above.
Serverless Error ----------------------------------------
Cannot read file node_modules@sindresorhus\is\dist\index.d.ts due to: EMFILE: too many open files, open 'C:\Users...\node_modules@sindresorhus\is\dist\index.d.ts'
Still broken..... Last shot. Gonna try it all in a completely different folder on a completely different hard drive. I'm also going to start from scratch in a totally new folder.
Serverless Error ----------------------------------------
Cannot read file node_modules\traverse\examples\stringify.js due to: EMFILE: too many open files, open 'C:\Users...\node_modules\traverse\examples\stringify.js'
Still? I'm not nuts am I? Okay... let's get out of the Users folder entirely. Maybe Windows is doing something weird like trying to sync OneDrive (even though I'm pretty sure that second SSD isn't sunk). Oh also, I figured out you don't actually need that api/index.js file. It'll still break on "Excluding development dependencies..."
Serverless Error ----------------------------------------
Cannot read file node_modules\d\node_modules\type\CHANGELOG.md due to: EMFILE: too many open files, open 'C:\test\node_modules\d\node_modules\type\CHANGELOG.md'
Okay... I'm at a total loss. All I can think of is that a bug got introduced to Serverless at some point. That or I'm a bizarre edge case.
Also tried installing a few older versions of Serverless: v2.50.0 - same issue. v2.25.0 - same issue. v2.0.0 - same issue.
I wonder if something in Windows changed.
@jvanalst can you post a reproduction case on GitHub as a project (please also include the content of node_modules
) ? That will allow us hopefully provide some solution if we happen to reproduce that on our side
https://github.com/jvanalst/emfile-serverless-win10-bug
So i closed my project on VS code, and opened a terminal window in the project directory. I Ran sls deploy
and everything worked fine. Could it possibly be a compatibility issue with VS code ?
I get the error regardless of VS Code vs PowerShell vs Command Prompt.
On Mon, Feb 7, 2022 at 10:49 AM 0xD1x0n @.***> wrote:
So i closed my project on VS code, and opened a terminal window in the project directory. I Ran sls deploy and everything worked fine. Could it possibly be a compatibility issue with VS code ?
— Reply to this email directly, view it on GitHub https://github.com/serverless/serverless-python-requirements/issues/662#issuecomment-1031746382, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACGSXKMFQ6W3LSAOWRI4SLU2AA4LANCNFSM5L3H26IA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID: @.*** com>
Issues still persists, here is my output:
Error: EMFILE: too many open files, open 'C:\Users\olli9\Documents\coding\graphql-ms\node_modules\@amplitude\types\dist\src\ampli.js'
Also tried installing a few older versions of Serverless: v2.50.0 - same issue. v2.25.0 - same issue. v2.0.0 - same issue.
I wonder if something in Windows changed.
Dont bother with version 3. The frameworks STILL broken
Same here:
Cannot read file node_modules\rxjs\dist\types\internal\operators\sequenceEqual.d.ts due to: EMFILE: too many open files, open 'C:\Users\...\Desktop\...\...\node_modules\rxjs\dist\types\internal\operators\sequenceEqual.d.ts'
can someone look at this issue?
@medikoo Looks like this is still an issue -- I've run into a couple of times before and have been able to work around it with either one or both of the following plugins: serverless-plugin-common-excludes and serverless-plugin-include-dependencies.
However, this issue seems to just randomly come up when trying to run sls deploy. Have you been able to look into why this issue still persists despite saying multiple times that it should be fixed next release or that it's currently fixed? Because, truth be told it is not fixed yet.
As I was writing this comment, I was JUST able to get past the problem by running npm install [email protected] specifically. I was using this package at version 5.0.0 earlier with no luck. However, my project is now running properly using version 4.1.0.
I hope this helps someone else who is experiencing this very frustrating issue with serverless framework!
@patrick-long thanks for the suggested work-around. I've also had this issue occurring from time to time. I had a good run of no issues and then had the EMFILE error pop up again. This was without any changes to dependencies in package.json. Reverting to [email protected] didn't do the trick in this case. Also, deleting and re-installing the packages in the node_modules folder didn't make a difference (although it seems to do so previously).
Hello, I got the EMFILE error when launching the serverless "deploy" script. I used graceful-fs like this to solve the issue :
- install graceful-fs:
npm install graceful-fs
- go to file:
node_modules/serverless/lib/plugins/package/lib/zip-service.js
- replace this line :
const fs = BbPromise.promisifyAll(require('fs'));
by this:
var realFs = require('fs')
var gracefulFs = require('graceful-fs')
gracefulFs.gracefulify(realFs)
const fs = BbPromise.promisifyAll(realFs);
@medikoo ... have been able to work around it with either one or both of the following plugins: serverless-plugin-common-excludes and serverless-plugin-include-dependencies.
Just installing serverless-plugin-common-exludes worked for me. Thanks! 🙇
A colleague saw the EMFILE: too many files open
error recently, and the error was only showing up on their computer. It took a while for us to figure it out what the issue was, but it ended up being a package glob issue.
We had manually listed exclude glob patterns in the serverless.yml
like this:
package:
individually: true
exclude:
- .git/**
- node_modules/**
- .venv/**
The colleague had created their virtual environment in a directory called venv
instead of .venv
, so serverless didn't know to skip their virtual environment when packaging the lambda. It was trying to read everything in their virtual environment which blew up the memory.
We updated the glob pattern to the following:
package:
individually: true
include:
- '!*'
- '!**'
- src/*.py
This pattern prevents anything other than python files in the src dir from getting packaged. It turns out we got speed benefits with this new syntax as well, as the glob pattern is much faster when you use !
in an include
rather than listing files in the exclude
because the glob takes a while to find every matching file that you want to exclude.
Delete _node_modules _, package.lock.json file and .serverless . after that reinstall all dependencies by npm install. and problem fixed.
A colleague saw the
EMFILE: too many files open
error recently, and the error was only showing up on their computer. It took a while for us to figure it out what the issue was, but it ended up being a package glob issue.We had manually listed exclude glob patterns in the
serverless.yml
like this:package: individually: true exclude: - .git/** - node_modules/** - .venv/**
The colleague had created their virtual environment in a directory called
venv
instead of.venv
, so serverless didn't know to skip their virtual environment when packaging the lambda. It was trying to read everything in their virtual environment which blew up the memory.We updated the glob pattern to the following:
package: individually: true include: - '!*' - '!**' - src/*.py
This pattern prevents anything other than python files in the src dir from getting packaged. It turns out we got speed benefits with this new syntax as well, as the glob pattern is much faster when you use
!
in aninclude
rather than listing files in theexclude
because the glob takes a while to find every matching file that you want to exclude.
It worked like a charm 🥇
For me I've just deleted the node_modules and runned npm install again, after that the deploy worked.
A colleague saw the
EMFILE: too many files open
error recently, and the error was only showing up on their computer. It took a while for us to figure it out what the issue was, but it ended up being a package glob issue. We had manually listed exclude glob patterns in theserverless.yml
like this:package: individually: true exclude: - .git/** - node_modules/** - .venv/**
The colleague had created their virtual environment in a directory called
venv
instead of.venv
, so serverless didn't know to skip their virtual environment when packaging the lambda. It was trying to read everything in their virtual environment which blew up the memory. We updated the glob pattern to the following:package: individually: true include: - '!*' - '!**' - src/*.py
This pattern prevents anything other than python files in the src dir from getting packaged. It turns out we got speed benefits with this new syntax as well, as the glob pattern is much faster when you use
!
in aninclude
rather than listing files in theexclude
because the glob takes a while to find every matching file that you want to exclude.It worked like a charm 🥇
This reminded me to check the exclude path list and fixed the issue. Thank you.