Miserlou
prebuilt cryptography for python 3.6?
I just got cryptography working in 3.6/lambda -- i'll try to PR it today.
nice adam..I have to pm opencv and numpy. Maybe I should share here so that you can pm all :)
Pycrptodome is py36 compatible and backward compatible with Pycrypto. https://github.com/Legrandin/pycryptodome
Getting Unable to import module <handler> /var/task/cryptography/hazmat/bindings/_constant_time.abi3.so: invalid ELF header. I have tried recompiling from the docker image and the AWS 2016.09 docker image with no luck.
I believe this requires a py3.6 version of CFFI.
File "/var/task/cryptography/hazmat/primitives/constant_time.py", line 9, in <module>
from cryptography.hazmat.bindings._constant_time import lib
ModuleNotFoundError: No module named '_cffi_backend'
Here's cffi 3.6 (normally under root of site-packages)
_cffi_backend.cpython-36m-x86_64-linux-gnu.so.zip
Here's crypto 3.6 (normally under cryptography/hazmat/bindings/)
@alvinwan Packaging with your cffi and cryptography SO files above, I get:
File "/var/task/cryptography/hazmat/primitives/constant_time.py", line 11, in <module>
from cryptography.hazmat.bindings._constant_time import lib
ImportError: libffi-d78936b1.so.6.0.4: cannot open shared object file: No such file or directory
Has anyone run into this?
Having the same issue following similar steps to the above @RevolutionTech
Personally I've given up on trying to use prebuilt object files in zappa projects. Instead, I use a Docker container based off of one of the lambci/lambda containers to run my zappa commands and build any object files I need.
For example, for Python 3.8 I use the lambci/lambda:build-python3.8 container. Here is a custom Dockerfile I based off of that to get SO files for SQLite that are compatible with the latest versions of Django:
https://github.com/RevolutionTech/opstrich/blob/main/docker/zappa/Dockerfile
And here's a project where I've used it: https://github.com/RevolutionTech/revolutiontech.ca/blob/master/Dockerfile
I hope this helps unblock someone!
Using @RevolutionTech's method, I put together a Dockerfile that should create a usable cryptography package.
FROM amazon/aws-lambda-python:3.8
WORKDIR /packaged
RUN yum install -y libffi libffi-devel gcc python3-devel openssl11 openssl11-devel tar gzip
RUN pip install cryptography --target /packaged --no-binary cryptography --no-dependencies
RUN tar -zcvf ../cryptography.tar.gz *
Generate the archive
> docker build .
> IMAGE_ID=`docker images | awk 'FNR == 2 {print $3; exit}'`
> docker cp $(docker create --rm $IMAGE_ID):/cryptography.tar.gz .
Thanks @cspollar - aws updated so - you'll want to match the 3.10 to whatever version they're on.
FROM amazon/aws-lambda-python:3.10
WORKDIR /packaged RUN yum install -y libffi libffi-devel gcc python3-devel openssl11 openssl11-devel tar gzip RUN pip install cryptography --target /packaged --no-binary cryptography --no-dependencies RUN tar -zcvf ../cryptography.tar.gz *
Hi All, The new and official Zappa repo is here here. We have added support for Python 3.10 environments while using the zip and in a docker deployment, you can use any version => 3.7.
if someone can successfully create a lamda layer - and publish the ARN as open / available to public - that would be a big help. I love zappa - but this cryptography lambda layer is pain point. related https://github.com/pyca/cryptography/pull/8994#issuecomment-1569983867
If you have a docker setup, I can definitely work to add cryptography and if you don't I will have to see how to compile it for python 3.6.
Because lambda out of the box as of this writing is 3.10 makes more sense to target that
Here I'm using a docker hack to generate the zip. I then went to import but gave up when it couldn't find it.
mkdir -p python/lib/python3.10/site-packages echo "cryptography" > requirements.txt sudo docker run -v "$PWD":/var/task "public.ecr.aws/sam/build-python3.10:latest" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.10/site-packages/; exit" zip -r lambda_function.zip .
Hi @johndpope and @wweevv-johndpope I built a docker image based on Debian with Python 3.10 and I was able to run cryptography without requiring any shared objects. I just added a cryptography package to the requirements and it worked. Here is the link to my deployment, if you want to see https://nu1a421aqf.execute-api.ap-south-1.amazonaws.com/lambda_docker_flask/crypto/
I just installed using pip inside a Debian-based container and it worked then I imported the cryptography module and produced a response using just django
from django.contrib import admin
from django.http import HttpResponse
from django.urls import path
import cryptography
CRYPTO_VERSION = cryptography.__version__
urlpatterns = [
path('admin/', admin.site.urls),
path('crypto/', lambda request: HttpResponse(f'cryptography version: {CRYPTO_VERSION}'))
]
Here is the Dockerfile
FROM XXXXXXXX.dkr.ecr.ap-south-1.amazonaws.com/debian-based-lambda-images:latest
ARG FUNCTION_DIR="/var/task/"
# Setup Python environment
COPY ./requirements.txt ${FUNCTION_DIR}
RUN pip install wheel
RUN pip install -r ${FUNCTION_DIR}requirements.txt
# RUN POETRY_VIRTUALENVS_CREATE=false poetry install --no-root
COPY ./ ${FUNCTION_DIR}
# Grab the zappa handler.py and put it in the working directory
RUN ZAPPA_HANDLER_PATH=$( \
python -c "from zappa import handler; print (handler.__file__)" \
) \
&& echo $ZAPPA_HANDLER_PATH \
&& cp $ZAPPA_HANDLER_PATH ${FUNCTION_DIR}
CMD [ "handler.lambda_handler" ]
interesting - after doing a google for lambda + docker came across this article https://medium.com/swlh/how-to-run-docker-containers-on-aws-lambda-c9bedd25fdf4 when I added a lambda function - there's a simple add layer button - from here you can point to shared layers that others have uploaded - this was what I was originally requesting - but can see a lot of benefit using docker under the hood. thanks for sharing.