Bug in pipenv v2022.4.21 is preventing generation of requirements.txt from Pipfiles

[guyc]

This issue is caused by a bug in a dependency, not by serverless-python-requirements. However it took quite a bit of digging to track down this issue, so maybe this will help others facing the same issue.

There is a recent bug in pipenv that will cause this plugin to fail if the following conditions are met:

• you are using the latest version of pipenv (v2022.4.30)
• you are using Pipfiles

An error was introduced into pipenv by this PR https://github.com/pypa/pipenv/pull/5069 which causes pipenv lock --requirements to write a deprecation warning message to stdout, which then gets written to the requirements.txt file.

The unintentionally injected text looks like this, and it makes the generated requirements.txt file unparseable.

            Warning: The lock flag -r/--requirements will be deprecated in a future version
            of pipenv in favor of the new requirements command. For more info see
            https://pipenv.pypa.io/en/latest/advanced/#generating-a-requirements-txt
            NOTE: the requirements command parses Pipfile.lock directly without performing any
            locking operations. Updating packages should be done by running pipenv lock

As a result serverless deploy terminates like this:

...
Serverless: Generating requirements.txt from Pipfile...
Serverless: [AWS sts 200 0.998s 0 retries] getCallerIdentity({})
Serverless: Parsed requirements.txt from Pipfile in /home/vsts/work/1/s/api/auth/.serverless/requirements.txt...
Serverless: Installing requirements from /home/vsts/.cache/serverless-python-requirements/48be0dccf397293edb052749ee84e17db11e4a514a061b07d1509d06c0c5b39a_x86_64_slspyc/requirements.txt ...
Serverless: Using download cache directory /home/vsts/.cache/serverless-python-requirements/downloadCacheslspyc
Serverless: Running ...
 
 Error ---------------------------------------------------
 
  Error: `python3.7 -m pip install -t /home/vsts/.cache/serverless-python-requirements/48be0dccf397293edb052749ee84e17db11e4a514a061b07d1509d06c0c5b39a_x86_64_slspyc -r /home/vsts/.cache/serverless-python-requirements/48be0dccf397293edb052749ee84e17db11e4a514a061b07d1509d06c0c5b39a_x86_64_slspyc/requirements.txt --cache-dir /home/vsts/.cache/serverless-python-requirements/downloadCacheslspyc` Exited with code 1
      at ChildProcess.<anonymous> (/home/vsts/work/1/s/api/auth/node_modules/child-process-ext/spawn.js:38:8)
      at ChildProcess.emit (node:events:526:28)
      at ChildProcess.emit (node:domain:475:12)
      at maybeClose (node:internal/child_process:1092:16)
      at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5)
 
  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com
 
  Your Environment Information ---------------------------
     Operating System:          linux
     Node Version:              16.14.2
     Framework Version:         2.70.0
     Plugin Version:            5.5.3
     SDK Version:               4.3.0
     Components Version:        3.18.1
 
make: *** [../../tools/serverless.mak:2: deploy] Error 1

Running the failing command in isolation confirms the issue, pip install is failing because of the spurious lines in the requirements.txt file.

# python3.7 -m pip install -t /root/.cache/serverless-python-requirements/48be0dccf397293edb052749ee84e17db11e4a514a061b07d1509d06c0c5b39a_x86_64_slspyc -r /root/.cache/serverless-python-requirements/48be0dccf397293edb052749ee84e17db11e4a514a061b07d1509d06c0c5b39a_x86_64_slspyc/requirements.txt --cache-dir /root/.cache/serverless-python-requirements/downloadCacheslspyc
ERROR: Invalid requirement: 'NOTE: the requirements command parses Pipfile.lock directly without performing any' (from line 2 of /root/.cache/serverless-python-requirements/48be0dccf397293edb052749ee84e17db11e4a514a061b07d1509d06c0c5b39a_x86_64_slspyc/requirements.txt)

The issue has been addressed in pipenv with this PR https://github.com/pypa/pipenv/pull/5091 so the problem should be resolved when it gets released.

[guayasamin]

thanks for the great summary, we are using github actions and ran across this same issue. As a temporary workaround, we added a step to generate the requirements.txt file with

      - name: Setup env
        run: |
          echo "\nENV_NAME=staging" >> .env
          pip install pipenv==2022.4.30

      - name: Create Deployment Dependencies File
        run: pipenv requirements > requirements.txt

and then disabled the pipenv integration with

custom:
  pythonRequirements:
    usePipenv: false

[guyc]

Pipenv admin has confirmed the fix is included in pipenv v2022.5.2.

[DasMagischeToastbrot]

I personally just preinstalled the latest version of pipenv (2022.5.2) before I use the serverless-python-requirements plugin and now it works.

[dnp1]

same here

[rafsanbhuiyan12]

1. Frist uninstall pipenv: 'pip uninstall pipenv` OR 'brew uninstall pipenv' OR 'sudo uninstall pipenv'
2. Second install pipenv==2022.5.2: sudo pip install pipenv==2022.5.2

pipenv: 2022.5.2 works great!! ⚡️🚀🤘😃

[yesthesoup]

Thanks for this thread. I am using CircleCI on an older project, python 3.8.6 image with default installed pipenv 2020.11.05. Not an option right now to update the python version, so I had to:

1. set usePipenv: false in serverless.yml as was suggested earlier in the thread

2. and change the deploy step in CircleCI config.yml from:

in the build step

- run:
          name: Install Python dependencies
          command: |
            pipenv install --dev --ignore-pipfile

to

 - run:
          name: Install Python dependencies
          command: |
            sudo pip uninstall -y pipenv
            pip install pipenv
            pipenv sync --dev

and the deploy step

- run:
          name: Deploy to staging
          command: |
           # ... aws configure steps
           pipenv run sls deploy --stage staging

to:

      - run:
          name: Deploy to staging preview
          command: |
            sudo pip uninstall -y pipenv
            pip install pipenv
            pipenv requirements > requirements.txt
            . /home/circleci/repo/.venv/bin/activate; echo "COMPLETED" # `pipenv shell` would hang because CI didn't see any return code
            # ... aws configure steps
            sls deploy --stage staging