Blueprints and lambda function inconsistency

[michaeldimchuk]

There's an inconsistency around how lambda functions are deployed when defined in app.py vs how they're deployed as part of a blueprint. When a lambda function is declared in a blueprint, the the AWS handler in sam.json is defined to be the path to the lambda function directly, so something like chalicelib.blueprints.first_blueprint.my_lambda.

Problem with this, is that the initialization logic in app.py is completely skipped when the function is invoked, so any middleware registered there is not applied to the lambda function defined in the blueprint. In addition to this, because blueprints defer middleware registration until app.register_blueprint is called, defining middleware in the blueprint containing the lambda function doesn't work either.

There's no issues with REST APIs because the AWS Handler is always app.app, but not the case with lambda functions. I tested these scenario with a bunch of log statements throughout both my code and the chalice framework, and always end up having a PureLambdaWrapper when defining the lambda function through a blueprint but MiddlewareHandler when defining it in app.py because without registration logic in app.py being executed, the middleware list is always empty in the blueprint.

Here's more or less the structure I was using:

app.py:

app = Chalice(app_name="asdf')
app.register_middleware(error_handler, "pure_lambda")

app.register_blueprint(first_blueprint.first_routes)
app.register_blueprint(second_blueprint.second_routes)

chalicelib/blueprints/first_blueprint.py:

first_routes = Blueprint(__name__)

@first_routes.lambda_function(name="my-lambda-function")
def my_lambda_function(event: dict, context) -> dict:
    return {"hello": "there"}

@first_routes.route("/cool/api/one", methods=['POST'], cors=True)
def cool_api_one() -> dict:
  return {"another": "hello"}

chalicelib/blueprints/second_blueprint.py:

second_routes = Blueprint(__name__)

@second_routes.route("/cool/api/two", methods=['POST'], cors=True)
def cool_api_two() -> dict:
  return {"yet another": "hello"}

Also not exactly related, but chalice deploy doesn't update the AWS Handler when switching between a version of the code not defining a lambda function through a blueprint and another which does. Only way I managed to fix that was by deleting my old lambda function or editing it with the new handler, but either option would create downtime. I can create another issue if this is actually a bug and is easier to deal with separately.

[jamesls]

Sorry for the delay on this. I've been thinking through how to solve this, and I think we'd always have to have the entry points be something in the app module so we always run the initialization/registration logic. I'm trying out a few ideas ranging from dynamically creating entry points on blueprint registration, having a blueprint "router" similar to how API routes work, etc. Open to suggestions.

[michaeldimchuk]

I didn't dig too deeply into the codebase, but using Chalice.__call__ as a global entry point for any and all events seemed like the cleanest solution. This would make it fairly easy to split up handler logic for various event types and properly initialize blueprints.

Only concern with that solution though would be that lambda functions aren't invoked directly anymore, so if any users had decorators defined on top of @app.lambda_function, they would no longer be invoked. One could argue though that that wasn't guaranteed behavior since HTTP routes already don't support that.

[msull]

I've run into this recently as well. Here's my current workaround -- it's not ideal but allows me to continue to define the event handlers within their own individual service modules and sets my code up so that once this issue is resolved I can easily convert to blueprint decorators. Additionally it requires no changes to app.py (or with my setup chalicelib/main.py) to bring in a new event handler which is important to me because I re-use app.py / main.py for several micro services.

inside one of my services, e.g. chalicelib/services/user_uploads.py I'll have an event handler defined something like this:

# @user_uploads_blueprint.on_sqs_message(queue='my-queue', batch_size=1)
def parse_received_file(event: SQSEvent):
    record: SQSRecord
    for record in event:
        logger.debug("Received message with contents: %s", record.body)


parse_received_file.event = "on_sqs_message"
parse_received_file.event_config = {
    "queue": settings.queue_user_uploads_handler,
    "batch_size": 1,
}

I have a file chalicelib/event_hooks.py -- all event handlers defined as such need to be imported here and added to the event_handlers list

from app import app

from .services.user_uploads import parse_received_file

event_handlers = [parse_received_file]  # import and add to this list all event handlers

# dynamically register and decorate all event handlers
# do not modify below
__all__ = tuple(x.__name__ for x in event_handlers)

for fn in event_handlers:
    # wrap function with decorator call using supplied config
    # overwrite definition of the function name in locals() so that imports of the
    # event functions from this file import the decorated version
    locals()[fn.__name__] = getattr(app, fn.event)(**fn.event_config)(fn)

And finally at the end of my app.py:

# import the main app
from chalicelib.main import app, settings  # noqa isort:skip

# import all event_hooks so they function properly; elminate once blueprint events work
if not settings.execute_db_migration:
    from chalicelib.event_hooks import *  # noqa isort:skip

This has everything working properly for me, all my event handlers work as expected (entrypoints on the lambda shows app.<function_name>)

[bradsgreen]

I'm very interested in a solution here as well. Just looking to do something as simple as .log and it does not work from a scheduled event:

@demo_product.schedule('rate(5 minutes)')
def append_to_test_file_every_five_mins(event):
    event_vals = event.to_dict()
    print(f'{event_vals["detail-type"]}:OK')
    demo_product.log.debug(event_vals)

yields:

Traceback (most recent call last):
  File "/var/task/chalice/app.py", line 1564, in __call__
    return self.handler(event_obj)
  File "/var/task/chalicelib/demo.py", line 16, in append_to_test_file_every_five_mins
    demo_product.log.debug(event_vals)
  File "/var/task/chalice/app.py", line 1932, in log
    raise RuntimeError(
RuntimeError: Can only access Blueprint.log if it's registered to an app.

[bradsgreen]

I'm very interested in a solution here as well. Just looking to do something as simple as .log and it does not work from a scheduled event:

@demo_product.schedule('rate(5 minutes)')
def append_to_test_file_every_five_mins(event):
    event_vals = event.to_dict()
    print(f'{event_vals["detail-type"]}:OK')
    demo_product.log.debug(event_vals)

yields:

Traceback (most recent call last):
  File "/var/task/chalice/app.py", line 1564, in __call__
    return self.handler(event_obj)
  File "/var/task/chalicelib/demo.py", line 16, in append_to_test_file_every_five_mins
    demo_product.log.debug(event_vals)
  File "/var/task/chalice/app.py", line 1932, in log
    raise RuntimeError(
RuntimeError: Can only access Blueprint.log if it's registered to an app.

I've worked around this in a fairly horrible fashion:

import logging
import sys


def get_logger_for_blueprint(name):
    # TODO: This may not be needed anymore when https://github.com/aws/chalice/issues/1566 is fixed
    log = logging.getLogger(name)
    log.setLevel(logging.DEBUG)
    handler = logging.StreamHandler(sys.stdout)
    # Timestamp is handled by lambda itself so the
    # default FORMAT_STRING doesn't need to include it.h
    FORMAT_STRING = '%(name)s - %(levelname)s - %(message)s'
    formatter = logging.Formatter(FORMAT_STRING)
    handler.setFormatter(formatter)
    log.propagate = False
    log.addHandler(handler)

    return log

then from each blueprint I call:

logger = chalice_logging_workaround.get_logger_for_blueprint(__name__)

But this is a highly unsatisfactory workaround and doesn't play well with some other logging that I want to do, like using the Honeybadeger wrapper. Would appreciate a proper fix.

[agherasim]

@jamesls is there any update on this issue? There are multiple use cases where a pure lambda function might require a DB connection that could be initialized in the app module, but then it's not accessible.

[juls858]

I am also running into this issue. Is there an ETA on a fix?

[aalvrz]

Experiencing a possibly related issue where pure lambda functions registered through a blueprint are not deployed when using Terraform packaging. However registering the pure lambda using @app.lambda_function seems to work 😕

[MicheDev3]

Any news on a fix? I am having the same problem where events defined using blueprints are not able to access the app class (attribute blueprint._current_app is None), the only thing that, it looks like, works are functions decorated with blueprint.route. One of the worst thing is that the error can be discoverd only deploying the chalice application, everything was working fine locally (I wrote integrations tests using chalice.test.Client app).

[baotran2207]

+1 for this feature

[msull]

I used the workaround described previously with success for some time but have found greater joy in my life by switching to AWS SAM. With the the AWS-provided REST API powertool (https://awslabs.github.io/aws-lambda-powertools-python/2.0.0/core/event_handler/api_gateway/) you can have nearly the same flask / chalice experience for API dev and significantly better experience for overall serverless application development, including a huge community of high quality examples to borrow from https://serverlessland.com/patterns?framework=SAM&language=Python.

[darlev91]

+1 for this feature