Getting error while trying to log "WatchtowerWarning: Received message after logging system shutdown"

[dhiyan-nirva-ai]

"watchtower_init_.py:381: WatchtowerWarning: Received message after logging system shutdown warnings.warn("Received message after logging system shutdown", WatchtowerWarning)"

Getting the above error while logging ,

[EddyIonescu]

Ran into this too, supplying use_queues=False to the CloudWatchLogHandler constructor function seemed to fix it. Might not be closing properly whenever you stop/start the server (like Django or Flask) you're using: https://github.com/kislyuk/watchtower/issues/38

[ymguerra]

Hi, I am also having this issue, in my case I use Django + gunicorn (gevent), if I set use_queues=False it works. I think is somehow related to Gunicorn + gevent because we share the same configuration with others servers (celery and websocket) and all the logs are send from those servers but not from our Web server, any idea?

[o-nikolas]

I think is somehow related to Gunicorn + gevent

Linking #89 for context

[Insighttful]

Hitting this issue using Uvicorn 0.24 and Fastapi 0.105.

[medley56]

I think I've fixed this issue by flushing all CloudWatchLogHandlers before exiting any process. I'm guessing the issue is that the logging system shuts down without waiting for the cloudwatch logs to finish transmitting.

The following code doesn't work if you have handlers attached to child loggers. This only flushes the handlers attached to the root logger.

def flush_cloudwatch_logs():
    """Force flush of all cloudwatch logging handlers. For example at the end of a process just before it is killed.

    Returns
    -------
    None
    """
    root_logger = logging.getLogger()
    for handler in root_logger.handlers:
        if isinstance(handler, watchtower.CloudWatchLogHandler):
            handler.flush()