`ContextDoesNotExistErrorContext`: Filter in Python native logger with FastAPI

[JanmanX]

I want to add context data to my logs through a Filter using FastAPI.

I setup my application with:

@asynccontextmanager
async def lifespan(app: FastAPI):
    # Setup logging
    logger = logging.getLogger()
    myformatter = logging.Formatter("MY HANDLER: %(request_id)s - %(message)s")
    myhandler = logging.StreamHandler()
    myhandler.setFormatter(myformatter)
    myhandler.addFilter(ContextFilter())
    logger.addHandler(myhandler)
    
    yield


# Adds the request_id
async def context_dependency():
    initial_data = {
        'request_id': str(uuid.uuid4()),
    }

    with request_cycle_context(initial_data):
        yield

     
app = FastAPI(
    dependencies=[Depends(context_dependency)],
    lifespan=lifespan,
)

The Filter:

class ContextFilter(logging.Filter):
    def filter(self, record):
        record.request_id = context.get("request_id", "<not_set>")
        return True

However, I get the ContextDoesNotExistError:

  File "/usr/local/lib/python3.10/logging/__init__.py", line 821, in filter
    result = f.filter(record)
  File "/usr/src/app/./app/application/logging/context_filter.py", line 12, in filter
    record.request_id = context.get("request_id", "")
  File "/usr/local/lib/python3.10/_collections_abc.py", line 824, in get
    return self[key]
  File "/usr/local/lib/python3.10/collections/__init__.py", line 1102, in __getitem__
    if key in self.data:
  File "/usr/local/lib/python3.10/site-packages/starlette_context/ctx.py", line 35, in data
    raise ContextDoesNotExistError
starlette_context.errors.ContextDoesNotExistError: You didn't use the required middleware or you're trying to access `context` object outside of the request-response cycle.

I could try to use the Middleware, but my understanding was the using the context_dependency() would yield the same result.

I have tried to look at the example with context logging in this starlette-context project, but it uses a different logger (structlog), which I want to avoid if possible, for simplicity sake.

What am I doing wrong?

[tomwojcik]

The issue is that dependencies and middleware serve different purposes in FastAPI. Dependencies only exist during route handler execution and within the dependency's scope. Middlewares wraps the entire request-response cycle.

request_cycle_context only creates a context (async store), but when used only in a dependency, this context isn't available during logging calls that happen outside your route handlers (middleware, background tasks, error handlers).

In your ContextFilter, you may want to check if context.exists(). This way some logs may have the request id.

  class ContextFilter(logging.Filter):
      def filter(self, record):
          if context.exists():
              record.request_id = context.get("request_id", "<not_set>")
          else:
              record.request_id = "<no_context>"
          return True

But the middleware approach is recommended for consistent context throughout the request lifecycle.