[Migrated] The "zappa update <stage>" command disconnects API Gateway and Lambda function.

[jneves]

Originally from: https://github.com/Miserlou/Zappa/issues/2207 by hidekuma

When creating events for scheduling in zappa_settings.json, the API Gateway and Lambda function are disconnected. The "zappa unschedule " command also disconnects the two services, is there a solution?

Context

Python 3.7, with pyenv, virtualenv

"events": [{
    "function": "app.cron",
    "expression": "rate(1 minute)"
}],

then

zappa update <stage>

API Gateway and Lambda functions are disconnected.

Expected Behavior

The connection between API Gateway and Lambda is not interrupted, and the registered event bridges must be applied.

Actual Behavior

API Gateway and Lambda functions are disconnected.

Possible Fix

Steps to Reproduce

1. Add events to zappa_settings.json
2. zappa update stage
3. API Gateway and Lambda functions are disconnected.

Your Environment

• Zappa version used: 0.52.0
• Operating System and Python version: python 3.7.6
• The output of pip freeze:

Package             Version
------------------- ----------
aniso8601           8.0.0
argcomplete         1.9.3
attrs               19.3.0
aws-xray-sdk        2.4.3
awscli              1.16.309
boto3               1.10.34
botocore            1.13.45
certifi             2019.11.28
cfn-flip            1.2.2
chalice             1.12.0
chardet             3.0.4
Click               7.0
colorama            0.4.1
docutils            0.15.2
durationpy          0.5
enum-compat         0.0.3
Flask               1.1.1
flask-restplus      0.13.0
future              0.16.0
greenlet            1.0.0
hjson               3.0.1
idna                2.8
importlib-metadata  1.2.0
itsdangerous        1.1.0
jedi                0.18.0
Jinja2              2.10.3
jmespath            0.9.3
jsonpickle          1.2
jsonschema          3.2.0
kappa               0.6.0
lambda-packages     0.20.0
MarkupSafe          1.1.1
more-itertools      8.0.2
msgpack             1.0.2
neovim              0.3.1
parso               0.8.1
pip                 21.0.1
pip-tools           5.5.0
placebo             0.9.0
pyasn1              0.4.8
pynvim              0.4.2
pyrsistent          0.15.6
python-dateutil     2.6.1
python-slugify      1.2.4
pytz                2019.3
PyYAML              5.1.2
requests            2.22.0
rsa                 3.4.2
s3transfer          0.2.1
setuptools          53.0.0
six                 1.13.0
toml                0.10.0
tqdm                4.19.1
troposphere         2.5.2
Unidecode           1.1.1
urllib3             1.25.7
Werkzeug            0.16.0
wheel               0.36.2
wrapt               1.11.2
wsgi-request-logger 0.4.6
zappa               0.52.0
zipp                0.6.0

• Your zappa_settings.json:

{
  "dev": {
    "profile_name": "xxx",
    "lambda_description": "xxx",
    "apigateway_description": "xxx",
    "tags": {
      "service": "xxx",
      "stage": "xxx"
    },
    "s3_bucket": "xxx",
    "environment_variables": {},
    "keep_warm": false,
    "debug": true,
    "log_level": "DEBUG",
    "api_key": "xxx",
    "remote_env": "s3://xxx/env.json",
    "domain": "xxx.com",
    "certificate_arn": "arn:aws:acm:us-east-1:xxx:certificate/xxx",
    "cloudwatch_log_level": "OFF",
    "vpc_config": {},
    "xray_tracing": false,
    "events": [
      {
        "function": "app.cron",
        "expression": "rate(1 minute)"
      }
    ],
    "app_function": "app.app",
    "project_name": "xxx",
    "slim_handler": false,
    "num_retained_versions": 20,
    "runtime": "python3.7",
    "memory_size": 128,
    "timeout_seconds": 60,
    "exclude": [
      ".git",
      ".gitignore",
      ".aws/*",
      ".ceph/*",
      "__pycache__",
      "CHANGELOG.md",
      "requirements.txt",
      "*.ini",
      "policy/*",
      "*.gz",
      "*.rar",
      "*.pyc",
      "*.yml",
      "dockerfiles/*",
      "Dockerfile",
      "*.Dockerfile",
      "*.swp"
    ],
    "endpoint_configuration": [
      "EDGE"
    ],
    "touch": true,
    "manage_roles": false,
    "role_name": "xxx",
    "cors": true,
    "aws_region": "xxx",
    "base_path": "v1",
    "api_key_required": true,
    "cloudwatch_data_trace": true,
    "cloudwatch_metrics_enabled": true
  }
}

[ararage]

The Same here, just update your environment with new code or whatever, run Zappa update & the API Gateway stops working retrieving a 500 Internal Server Error, for now temporally i'm running first $ zappa undeploy and $ zappa deploy setting up the configurations & updating the URL given by the API Gateway (Re deploying my API as a new App), a month ago this was fine i don't know if it's an AWS internal configuration or something with Cloud Formation :(

[ghost]

We are facing the same issue. It's the first time we are using it, but after updating the code, API Gateway and Lambda functions are disconnected, with a 500 Internal Server Error.

[jvilar-greenspot]

Same here! At first we thought it was a problem with our setup in AWS and were driving us crazy :S We are at an early development stage with our mobile app and need to update our project frequently. It is a nightmare to undeploy, update and deploy manually everytime (plus the time the system is down). It would be great to have a solution 🙏

[gbartosz]

We're also facing this issue. The problem seems to be with newly created Lambda not granting API Gateway permission to trigger the Lambda. So far we're doing this manually in the AWS Console after every deploy, but the fix would be more than welcome! I can provide more details if necessary - just tell me what you need.

[gbartosz]

We're also facing this issue. The problem seems to be with newly created Lambda not granting API Gateway permission to trigger the Lambda. So far we're doing this manually in the AWS Console after every deploy, but the fix would be more than welcome! I can provide more details if necessary - just tell me what you need.

We got tired of manually fixing it after every deploy so we automated it. Our pipeline runs below script right after zappa deploy. Maybe you'll find it useful.

`import boto3 import os if name == 'main':

print('Creating permission to trigger a lambda...')
lamb = boto3.client('lambda')

lambda_name = os.getenv('LAMBDA_NAME')
gateway_id = os.getenv('API_GATEWAY_ID')

print(f'    Lambda name: {lambda_name}')
print(f'    Api Gateway id: {gateway_id}')

lamb.add_permission(FunctionName=lambda_name,
                    StatementId=f'AllowLambda{lambda_name}ToBeTriggeredByApiGateway{gateway_id}',
                    Action='lambda:InvokeFunction',
                    Principal='apigateway.amazonaws.com',
                    SourceArn=f'arn:aws:execute-api:eu-north-1:xxxxxxxxxxxxx:{gateway_id}/*')

`

[deceze]

My guess: it's the call to Zappa._clear_policy as part of unschedule_events which removes all existing permission statements uncritically. I'm not sure if that conflicts with anything Zappa sets up itself, but it certainly conflicts with any extra policies one might have set up. Can that be made more selective to only touch Zappa-created policies…?

[deceze]

FWIW, my discovered facts and solution:

• Our app uses a separate WebSocket API Gateway in addition to the default API Gateway that Zappa deploys. This is being set up with a callbacks.settings function that monkey patches the stack template Zappa creates; though that's not relevant, you may also set it up manually.
• The issue is that Zappa unschedules all events and re-creates them when using keep_warm; as part of this unscheduling, it also removes all policies connected with that Lambda function.
• We've set up the permission for the WebSocket API Gateway to make calls to the Lambda function via an explicit Permission entry in the stack template, i.e. something that creates a policy and associates it with the Lambda function. This policy got cleared as part of the event unscheduling.
• The correct way to set up that permission is to use the IAM role policy that Zappa creates as Execution Role/Lambda Invoke Role/CredentialsArn (it's called different things in different places), so the API Gateway assumes that role, which includes permission to call the Lambda function. This doesn't require the creation of a separate policy which then also won't get cleared. This is zappa.zappa.credentials_arn in the settings callback, if you need it.

Hope that helps someone. I'd still question whether unschedule_events should be a bit more selective about what policies it removes.

[github-actions[bot]]

Hi there! Unfortunately, this Issue has not seen any activity for at least 90 days. If the Issue is still relevant to the latest version of Zappa, please comment within the next 10 days if you wish to keep it open. Otherwise, it will be automatically closed.

[github-actions[bot]]

Hi there! Unfortunately, this Issue was automatically closed as it had not seen any activity in at least 100 days. If the Issue is still relevant to the latest version of Zappa, please open a new Issue.