[Migrated] slim_handler: true fails in Windows, possibly due to file slashes in tarball

[jneves]

Originally from: https://github.com/Miserlou/Zappa/issues/2145 by CaseGuide

It is not possible to use Zappa on Windows with slim_handler = true. This is likely due to the slashes used when tarring the file. This issue is well known, probably easy to fix, and has cropped up in #1870 and #1358 with a PR for #1358 in #1570 that seemed to not get integrated.

Context

Using venv and Python 3.6 on Widows 10, I was able to successfully deploy a project and get the expected results. Upon installing pandas, my lambda project exceeded what Lambda would accept. After pip install pandas and the failed deployment I changes zappa_settings.json to add slim_handler : true, then deployed. I'm now encountering an issue returning the error errorMessage": "Unable to import module 'index'". My handler function is a regular python function that is in index.py and was otherwise working fine.

Expected Behavior

Expect this deployment to be successful and return a 200 and some JSON via my API Gateway

Actual Behavior

I receive a 502 {"message": "Internal server error"} from the API Gateway endpoint and zappa tail returns many instances of: [1596019020990] Unable to import module 'index': No module named 'index'. My function is in index.py. "lambda_handler": "index.handler",

Possible Fix

• Redo your project in Linux, on Windows can try WSL.
• Consider editing cli.py line 228 to use zip instead of tar. If I do this might also need to change the file name string
• Consider editing the path itself in core.py around line 766

Steps to Reproduce

1. Create venv, pip install zappa, etc. on a Windows 10 machine
2. zappa deploy with "slim_handler": true, "delete_local_zip": false,
3. Run zappa tail expected output should be something like: Unable to import module <handler_function>: No module named <your import>
4. Using the .tar.gz created on Windows, list the files in it using tar --list --verbose --file=<project>-<env>-<time>.tar.gz on Linux. The output, in my case from WSL Ubuntu, proves that there are indeed mixed slashes

-rw-rw-rw- 0/0               0 2020-07-28 16:54 djangocms_icon/migrations/__pycache__/__init__.py
-rwxr-xr-x 0/0            4610 2020-07-28 16:54 djangocms_icon\\static\\djangocms_icon\\css\\djangocms-icon.css
-rwxr-xr-x 0/0             274 2020-07-28 16:54 djangocms_icon\\static\\djangocms_icon\\js\\base.js
[...]
-rwxr-xr-x 0/0            2670 2020-07-28 16:54 ~umpy\\tests\\__pycache__\\test_warnings.cpython-36.pyc
-rwxr-xr-x 0/0             168 2020-07-28 16:54 ~umpy\\tests\\__pycache__\\__init__.cpython-36.pyc
-rw-rw-rw- 0/0               0 2020-07-28 16:54 ~umpy/tests/__pycache__/__init__.py
[...]
-rwxr-xr-x 0/0            8140 2020-07-28 16:54 ~umpy\\__pycache__\\__init__.cpython-36.pyc
-rw-rw-rw- 0/0               0 2020-07-28 16:54 ~umpy/__pycache__/__init__.py
[...and so on]

Your Environment

• Zappa version used: 0.51.0
• Operating System and Python version: Windows 10, Python 3.6
• The output of pip freeze: I have a bloated pip freeze here, I'm not actually using django for this particular function, but its part of a set of lambdas that are all on the same venv. Posted my requirements.txt for brevity:

zappa==0.51.0
boto3==1.14.22

Django==2.0.6
mysqlclient==1.3.12
pymysql

### Created by djangocms-installer
django-cms>=3.7,<3.8
djangocms-admin-style>=1.5,<1.6
django-treebeard>=4.0,<5.0
djangocms-text-ckeditor>=3.7,<4.0
djangocms-link>=2.5,<2.7
djangocms-icon>=1.4,<1.6
djangocms-style>=2.2,<2.4
djangocms-googlemap>=1.3,<1.5
djangocms-snippet>=2.2,<2.4
djangocms-video>=2.1,<2.4
djangocms-file>=2.3,<2.5
djangocms-picture>=2.3,<2.5
djangocms-bootstrap4>=1.5,<1.7
easy_thumbnails
django-filer>=1.3
django-classy-tags>=0.9
django-sekizai>=1.0
django-mptt>0.9
html5lib>=1.0.1
Pillow>=3.0
six
pytz
### Added later
djangorestframework
django-s3-storage==0.13.2

### What I'm using for this part of the project.
selenium==3.13.0
pandas==0.25.1 # This version of pandas is needed for compatibility with zappa's python-dateutils requirement
python-dateutil==2.6.1 # This overlaps the reqs of zappa 0.51.0 and pandas 0.25.1

• Your zappa_settings.json:

{
    "dev": {
        "aws_region": "us-east-1",
        "profile_name": "default",
        "project_name": "<name>",
        "runtime": "python3.6",
        "s3_bucket": "<bucket>",
        "use_precompiled_packages": true,
        "lambda_handler": "index.handler",
        "memory_size": 2048,
        "timeout_seconds": 300,
        "cloudwatch_log_level": "INFO",
        "slim_handler": true,
        "delete_local_zip": false, // Delete the local zip archive after code updates. Default true.
        "delete_s3_zip": false // Delete the s3 zip archive. Default true.
    }
}

EDIT: I noted that this issue doesn't seem to affect the zip file. Running unzip -l handler_<project>-<env>-<time>.zip in WSL on the Windows-created handler zip works without issue. See output:

    25185  1980-01-01 00:00   troposphere-2.6.2.dist-info/METADATA
    18383  1980-01-01 00:00   troposphere-2.6.2.dist-info/RECORD
       12  1980-01-01 00:00   troposphere-2.6.2.dist-info/top_level.txt
       97  1980-01-01 00:00   troposphere-2.6.2.dist-info/WHEEL
    14461  1980-01-01 00:00   urllib3/connection.py
    35725  1980-01-01 00:00   urllib3/connectionpool.py
     7172  1980-01-01 00:00   urllib3/exceptions.py
     8553  1980-01-01 00:00   urllib3/fields.py

[MartijnGevaert]

This is an issue for me as well because:

1. It's not made clear that this functionality does not work on Microsoft. Took me some time to find this github issue.
2. I'm working in Pycharm community edition which does not support WSL interpreter. Now looking into other ways I can still get Zappa working.

[CaseGuide]

This is still an issue in Zappa 0.54.0/Python3.7/Win 10. Had a legacy project using Zappa. Likely just going to more everything to docker the AWS SAM CLI and deploy that way with a docker container. No more running into the stupid 250MB limit.

[github-actions[bot]]

Hi there! Unfortunately, this Issue has not seen any activity for at least 90 days. If the Issue is still relevant to the latest version of Zappa, please comment within the next 10 days if you wish to keep it open. Otherwise, it will be automatically closed.

[github-actions[bot]]

Hi there! Unfortunately, this Issue was automatically closed as it had not seen any activity in at least 100 days. If the Issue is still relevant to the latest version of Zappa, please open a new Issue.