aws-sam-cli

aws-sam-cli copied to clipboard

Description

Trying to create Python Lambda function from PyCharm not working

Steps to reproduce

Invoke Lambda creation using "Create new AWS Lambda..." Runtime: Python 3.7 requirements.txt: https://pastebin.com/HpU1it63 Run using strict template file: template.yml:

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: An AWS Serverless Specification template describing your function.
Resources:
  test3:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: order.create_order
      Runtime: python3.7
      CodeUri: .
      Description: ''
      MemorySize: 128
      Timeout: 300
      Role: 'arn:aws:iam::123456789:role/service-role/test-role'
      VpcConfig:
        SecurityGroupIds:
          - sg-62e1c42f
        SubnetIds:
          - subnet-26e9c26e
          - subnet-68a3c632
          - subnet-b8edb5de

Observed result

Used graphical create, not CLI, error:

Build Failed
Error: PythonPipBuilder:ResolveDependencies - {msgpack-python==0.5.6(sdist), pyrsistent==0.15.7(sdist), simplejson==3.17.0(wheel), pyyaml==5.3(sdist)}

Full log: https://pastebin.com/asgCa9D4

Expected result

Successful build

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

OS: macOS Mojave 10.14.6 SAM: SAM CLI, version 0.43.0 P: Python 3.7.6 IDE: PyCharm 2019.3 AWS: aws-cli/2.0.1 Python/3.7.4 Darwin/18.7.0 botocore/2.0.0dev5 used wheel: wheel 0.34.2 used pip: pip 19.0.3

I solved this using sam build --use-container, however this works only when template.yml file is present. Using the graphical form does not allow to insert --use-container and it fails, so I do not count this as a solution, but workaround instead... // Also since I am unable to build app locally in my IDE, I am unable to run/debug it, so it makes no sense to use this workaround.

Are you using AWS Toolkit for Pycharm?

I'm seeing something similar:

$ sam build
Building resource 'PaymentsDataStreamFunction'
Running PythonPipBuilder:ResolveDependencies

Build Failed
Error: PythonPipBuilder:ResolveDependencies - {pyyaml==5.3.1(sdist), pyrsistent==0.15.7(sdist)}

Both pyyaml & pyrsistent are inside my requirements.txt file, which have been installed inside a virtualenv.

Do you have wheel installed? if not sam build --use-container may be the best bet.

wheel is installed. --use-container also fails with a similar error but with different deps. I tried creating a brand new project which worked fine. So, it's probably something within my project that is messed up.

@c2tarun yes, I am using this plugin

Same here - Used another dep that depends on jsonschema and bumped into this. Can't use --use-container as I have a shared dep two directories up. For that to work I'd need to copy the wheel to every service I have, or have a way to mount a volume during sam build.

Thanks for opening the issue, I'll spend a couple of days rewriting the lib and build a wheel from that instead

I'm getting same issue when I use sam build --use-container (Windows, Python 3.6, SAM CLI version 0.47.0): Build Failed Error: PythonPipBuilder:ResolveDependencies - {tensorflow==1.15.0(wheel), grpcio==1.28.1(wheel)}

Same. It seems like it fails to build anything with tensorflow or better yet grpcio

My current workaround is to manually add grpcio to the requirements.txt file, but fixed to an older version.

grpcio==1.27.2

It seems to fail with version 1.28.

I'm seeing a similar error with the following in requirements.txt pyorc==0.2.0

I am running:

sam build --use-container

Output: Starting Build inside a container Building resource 'CdcFunction'

Fetching lambci/lambda:build-python3.7 Docker container image...... Mounting /path/to/my/app as /tmp/samcli/source:ro,delegated inside runtime container

Build Failed Running PythonPipBuilder:ResolveDependencies Error: PythonPipBuilder:ResolveDependencies - {pyorc==0.2.0(wheel)}

wheel is installed. I'm getting the same error when I add psycopg2 to the requirements. Here is my template.yaml for reference

Globals:
  Function:
    Timeout: 180

Resources:
  CdcFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: lambda/
      Handler: lambda.lambda_handler
      Runtime: python3.7

+1

+1

Error: PythonPipBuilder:ResolveDependencies - {pyrsistent==0.16.0(sdist), pyyaml==5.1.2(sdist)}

If your dependencies have C bindings involved, building within the container is the way forward in those cases. Toolkits have an explicit checkbox that can be enabled to allow for all builds to be within a build container.

Closing this issue, please re-open if you feel this doesn't answer your original question.

got the following error started appearing during sam build

Error: PythonPipBuilder:ResolveDependencies - {protobuf==3.17.1(wheel)}

SAM CLI, version 1.20.0 Python 3.8.7 pip 21.1.2

--use-container doesn't solve the problem.

UnsatisfiedLinkError: com.amazonaws.kinesisvideo.producer.jni.NativeKinesisVideoProducerJni.getNativeLibraryVersion()Ljava/lang/String I am following this workspace https://github.com/backdoorcodr/amazon-kinesis-video-streams-producer-sdk-java Expectation is to run kinesis video stream using webcam in java. Please help

I was running into Error: PythonPipBuilder:ResolveDependencies - {cytoolz==0.11.0(wheel)} with sam 0.45.0. Upgrading sam to latest (1.27.2) solved it

Build Failed
Error: PythonPipBuilder:ResolveDependencies - {pillow==8.3.1(wheel)}

Solved updating SAM CLI from version 1.22.0 to version 1.29.0

I'm having the same issue

sam build --use-container --template ./my_template.yaml --debug

produces

Error: PythonPipBuilder:ResolveDependencies - Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-download-ufgf7ow5/fastparquet/

However, installing the requirements.txt with pip in the host machine produces no error.

I also tried to update my sam version from 1.27.2 -> 1.31.0 and I get the same error.

I have the same issue. Tried installing wheel, using --use-container, updating sam to 1.31. None is helpful.

The most weird thing is it was working last week, but suddenly broke today. Nothing has changed at all!

Reopening since this still seems to be an issue. Potentially related: https://github.com/aws/aws-lambda-builders/issues/280

+1 Error: PythonPipBuilder:ResolveDependencies - {olefile==0.46(sdist), future==0.18.2(sdist), wrapt==1.12.1(sdist)}

I was having a similar problem just running from the command line with SAM cli 1.15. A little behind, I know!

Here's what I had before using a python pipenv virtual environment.

OS: macOS Monterey 12.0.1 SAM: SAM CLI, version 1.15.0 P: Python 3.8.7 IDE: 1.61.2 AWS: aws-cli/2.1.21 Python/3.7.4 Darwin/21.1.0 exe/x86_64 prompt/off used wheel: wheel 0.37.0 used pip:21.3.1

I upgraded to 1.34 and build seems to work.

Thanks @ryanaklein ! Unfortunately in my case it's already version 1.34.1, though the other versions are different (for instance, python 3.9).

My environment:

• macOS Catalina
• Python 3.9.7
• pip 21.3.1 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)
• SAM CLI, version 1.35.0

I am building this project: https://github.com/jojo786/awswhatsnew, which works perfectly locally, but not with sam.

If i do a sam build without a requirements.txt, its successful. But with this requirements.txt file of:

feedparser
requests
python-twitter
aws_lambda_powertools

I get this when doing a sam build

Running PythonPipBuilder:ResolveDependencies

Build Failed
Error: PythonPipBuilder:ResolveDependencies - {wrapt==1.13.3(wheel)}

These are the things I have tried unsuccessfully:

• start with a new clean venv, and install all dependencies
• use sam build with --use-container
• install wheel as mentioned in https://github.com/aws/aws-lambda-builders/issues/280
• include wheel in requirements.txt
• download wrapt wheel manually and install

Linked issues:

• https://github.com/aws/aws-sam-cli/issues/1380
• https://github.com/aws/aws-lambda-builders/issues/208
• https://github.com/aws/aws-sam-cli/issues/2291
• https://github.com/aws/aws-lambda-builders/issues/201

This fixed the ResolveDependencies error for me pip install wheel

I am also seeing a problem. I can deploy to AWS without anything in my requirements file, but when I add boto3 (for instance to my requirements file I get the following error)

Build Failed
Running PythonPipBuilder:ResolveDependencies
Error: PythonPipBuilder:ResolveDependencies - 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

My requirements.txt looks like this...

boto3==1.18.0 botocore==1.21.65 jmespath==0.10.0 python-dateutil==2.8.2 s3transfer==0.5.0 six==1.16.0 urllib3==1.26.7

I've tried upgrading pip and wheel but this has not helped.

This definitely seems to be some sort of pip issue around string encoding, lots of results like this: https://stackoverflow.com/questions/41906206/pip-unicodedecodeerror-utf8-codec-cant-decode-byte

I'm unfortunately not sure what we can do about this, we're reliant on pip for Python builds, and this doesn't seem to come from any pass-through behavior. Does it behave differently if you run pip directly?

Thanks for coming back to me. You are right it was an encoding issue of the requirements.txt file. To check it I opened it and printed the encoding info.

f=open('requirements.txt, 'w') print(f.encoding)

The result was cp1252.

I altered this by changing it in the Windows control panel

Control panel (make sure you are set to category view)> Clock and Region > Region > Adminsitrative > Change System Locale

and ticked the box that says 'Beta: Use Unicode UTF-8 for worldwide language support'. Following a restart, my encoding had changed to 'cp65001' which I believe is equivalent to UTF8 and I was able to create my Lambda Function.

Glad you were able to resolve this! FWIW I've seen this periodically with Python builds especially, pip seems to be very sensitive to encoding issues. Unfortunately, nothing we can do on our end about that.