pydantic
Update Django setup doc to deal with local settings
Description
In the django doc (https://logfire.pydantic.dev/docs/integrations/django/) it is specified to # Add the following lines at the end of the file, which is very useful.
But on some django projects, they use a local settings pattern. E.G: you may have a settings.py and a local_settings.py file instead of, say, env vars, for machine-specific settings.
Because the local_settings.py file may be the one imported first by wsgi.py, if you put:
logfire.configure()
logfire.instrument_django()
At the end of settings.py you will get no error, but also no telemetry. There is no way to debug this unless you have a hunch about how the internals of LogFire automatic instrumentation work.
There are as many settings patterns are there are projects, some have "prod/dev" settings files, some have cascading setting files with "common.py" files imported among others, some are dynamic with if/else imports, some modify wsgi.py... After all, it's just all Python.
So the doc needs to give a rule to know in which file you should put the configure() call otherwise plenty of people will feel like LogFire doesn't work and get zero feedback about why.
Python, Logfire & OS Versions, related packages (not required)
logfire="2.4.1"
platform="Linux-6.8.0-49-generic-x86_64-with-glibc2.39"
python="3.12.3 (main, Nov 6 2024, 18:32:19) [GCC 13.2.0]"
[related_packages]
requests="2.32.3"
requests="2.32.3"
pydantic="2.10.0"
fastapi="0.115.5"
protobuf="5.28.3"
protobuf="5.28.3"
rich="13.7.1"
rich="13.9.4"
rich="13.9.4"
executing="2.1.0"
executing="2.1.0"
opentelemetry-api="1.28.2"
opentelemetry-api="1.28.2"
opentelemetry-exporter-otlp-proto-common="1.28.2"
opentelemetry-exporter-otlp-proto-common="1.28.2"
opentelemetry-exporter-otlp-proto-http="1.28.2"
opentelemetry-exporter-otlp-proto-http="1.28.2"
opentelemetry-instrumentation="0.49b2"
opentelemetry-instrumentation="0.49b2"
opentelemetry-instrumentation-asgi="0.49b2"
opentelemetry-instrumentation-django="0.49b2"
opentelemetry-instrumentation-fastapi="0.49b2"
opentelemetry-instrumentation-wsgi="0.49b2"
opentelemetry-proto="1.28.2"
opentelemetry-proto="1.28.2"
opentelemetry-sdk="1.28.2"
opentelemetry-sdk="1.28.2"
opentelemetry-semantic-conventions="0.49b2"
opentelemetry-semantic-conventions="0.49b2"
opentelemetry-util-http="0.49b2"
opentelemetry-util-http="0.49b2"
It's hard to say what a good rule would be. We used to suggest manage.py, but then settings.py seemed better. OTEL seems to not even try suggesting a location at all: https://opentelemetry-python-contrib.readthedocs.io/en/latest/instrumentation/django/django.html
If you look inside, it turns out that the way this works is to add the string 'opentelemetry.instrumentation.django.middleware.otel_middleware._DjangoMiddleware' to the beginning of the settings.MIDDLEWARE list. And then any arguments to logfire.instrument_django() / DjangoInstrumentor().instrument() are set as attributes on the _DjangoMiddleware class 🤮
I would say something like :
"Those lines must be the last thing to execute in your settings. On a regular Django project, this means at the end of settings.py"
WARNING: If you use an exotic configuration setup with several settings files divided into local/prod/dev, make sure you put those lines where they will be imported and executed last, or you won't see any telemetry and yet, no error."
Having the call logfire configure in settings.py is cumbersome for mypy and pytest, which import the django settings file. It would be good to have an alternative recommendation.
I was experimenting adding it to asgi.py but it did not work for me
It shouldn't be a problem for mypy and pytest to call logfire.configure. Also are you saying mypy executes code?
Yeah. settings.py is executed by the mypy django plugin - https://github.com/typeddjango/django-stubs/issues/458
I know that this is not your issue, but it does seem pretty challenging to fix - so I am wondering if there is an alternative location to configure logfire in.
I think i came up with a solution that works:
Setup a middleware to configure logfire
# filename: `logfire.middleware.py`
def logfire_middleware(get_response):
"""
Configure logfire once for the entire Django app.
This middleware is executed for each request and response.
"""
logfire.configure(service_name="api", distributed_tracing=True)
logfire.instrument_django(capture_headers=True)
def middleware(request):
# Code to be executed for each request before
# the view (and later middleware) are called.
response = get_response(request)
# Code to be executed for each request/response after
# the view is called.
return response
return middleware
Add the middleware to settings
#file: settings.py
...
MIDDLEWARE = [
...
"package.logfire_middleware.logfire_middleware"
]
...
instrument_django modifies MIDDLEWARE by inserting at the beginning of the list. This will modify the list as it's being iterated over. I'm surprised if this works at all, but either way I don't think it's reliable.
right! that is not great then.
I have added it to gunicorn.conf.py - this seems to work