Values in Pipfile seem to be ignored

[paulalex]

I am currently unsure if this is a bug, but I am using the plugin to build requirements from a Pipfile, and I see the following message:

Serverless: Generating requirements.txt from Pipfile...
--
Serverless: Parsed requirements.txt from Pipfile in /codebuild/output/src226277008/src/github.com/my-org/my-service/.serverless/requirements.txt...
Serverless: Installing requirements from /root/.cache/serverless-python-requirements/844ade38cf22332a92d3662e1b0f852366c5dc9d600ec37332c7f2ecdb1515a5_x86_64_slspyc/requirements.txt ...
 Serverless: Using download cache directory /root/.cache/serverless-python-requirements/downloadCacheslspyc
Serverless: Running ...

This is what the Pipfile contains (I have pinned pandas and numpy as they are causing me a build issue):

[packages]
jsons = "*"
pandas = "1.2.1"
openpyxl = "*"
xlsxwriter = "*"
xlrd = "*"
numpy = "1.19.5"
boto3 = "*"
botocore = "*"
requests = "*"
aws-requests-auth = "*"
aws-lambda-powertools = "*"

However when I download the layer and look at the requirements.txt file included in it, the versions are not honoured:

jsons==1.6.0
numpy==1.22.0rc1
openpyxl==3.0.9
pandas==1.3.4

You can see that the versions defined in the Pipfile are being ignored for some reason, is there a chance that if there is a Pipfile.lock present it is actually using this file?

[paulalex]

I thought this was caused by the missing == from my dependencies, but even after adding this in the plugin still ignores these values and justs gets the latest version and I dont understand why.

[pgrzesik]

Hello @paulalex 👋 Could you please share the final pipfile that you're currently using? Also, what version of plugin are you using? Additionally, could you try to just install your dependencies with pipenv in some venv and see how it's going to resolve these? Plugin internally uses pipenv itself to generate requirements and does not manipulate it in any way, so it might be either an issue with pipenv or your configuration.

[paulalex]

Hi, thanks!

The plugin version is:

[email protected]

Here is the output of pip list from the virtual environment I am using for that service:

(shape) ind-openit-shape-service (feature/OPEN-2349) $ pip list
Package                           Version
--------------------------------- ----------
appdirs                           1.4.4
astroid                           2.6.6
attrs                             21.2.0
aws-lambda-powertools             1.22.0
aws-requests-auth                 0.4.3
aws-xray-sdk                      2.8.0
backports.entry-points-selectable 1.1.1
bandit                            1.7.1
black                             21.11b1
boto3                             1.20.18
botocore                          1.23.18
certifi                           2021.10.8
cfgv                              3.3.1
chardet                           4.0.0
charset-normalizer                2.0.8
click                             8.0.3
coverage                          6.2
distlib                           0.3.3
et-xmlfile                        1.1.0
fastjsonschema                    2.15.1
filelock                          3.4.0
future                            0.18.2
gitdb                             4.0.9
GitPython                         3.1.24
identify                          2.4.0
idna                              3.3
iniconfig                         1.1.1
isort                             5.10.1
jdcal                             1.4.1
jmespath                          0.10.0
jsonpickle                        2.0.0
jsons                             1.6.0
lazy-object-proxy                 1.6.0
mccabe                            0.6.1
mypy                              0.910
mypy-extensions                   0.4.3
nodeenv                           1.6.0
numpy                             1.19.5
openpyxl                          3.0.9
packaging                         21.3
pandas                            1.2.1
pathspec                          0.9.0
pbr                               5.8.0
pip                               20.1.1
pipenv                            2020.11.15
platformdirs                      2.4.0
pluggy                            1.0.0
pre-commit                        2.16.0
py                                1.11.0
pylint                            2.9.6
pyparsing                         3.0.6
pytest                            6.2.5
pytest-mock                       3.6.1
python-dateutil                   2.8.2
pytz                              2021.3
PyYAML                            6.0
regex                             2021.11.10
requests                          2.26.0
s3transfer                        0.5.0
setuptools                        49.1.0
six                               1.16.0
smmap                             5.0.0
stevedore                         3.5.0
toml                              0.10.2
tomli                             1.2.2
typed-ast                         1.4.2
typing-extensions                 4.0.1
typish                            1.9.3
urllib3                           1.26.7
virtualenv                        20.10.0
virtualenv-clone                  0.5.4
wheel                             0.34.2
wrapt                             1.13.3
xlrd                              2.0.1
XlsxWriter                        3.0.2

Here is the entire Pipfile

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]
bandit = "1.6.2"
mypy = "0.790"
pre-commit = "2.8.2"
coverage = "5.2"
black = "*"
pylint = "~=2.9.1"
pytest = "~=6.2.4"
pytest-mock = "~=3.6.1"

[packages]
jsons = "*"
pandas = "==1.2.1"
openpyxl = "*"
xlsxwriter = "*"
xlrd = "*"
numpy = "==1.19.5"
boto3 = "*"
botocore = "*"
requests = "*"
aws-requests-auth = "*"
aws-lambda-powertools = "*"

[requires]
python_version = "3.8"

[pipenv]
allow_prereleases = true

[paulalex]

I do think I might have been a little hasty, when testing with the ==, could you confirm the behaviour if the == are missed from the dependency in the pip file?

Could you tell me whether in this instance the default is the latest version for that dependency?

I have actually found in my build logs the following warning which could be the cause of this:

npm WARN [email protected] requires a peer of serverless@^2.32 but none is installed. You must install peer dependencies yourself.

I think I should possibly pin the version until we upgrade to serverless 2.

[pgrzesik]

Hey @paulalex - as far as I know with pipenv if you dont specify the version properly e.g. with == then it will just use the latest available version.

The warning you're listing has nothing to do with how pipenv works. You can try it on your own, without serverless-python-requirements, try to install with pipenv without == in specific versions and you'll see how it will be installed.