Instrument logs in Celery tasks

[jeremydvoss]

Before opening a feature request against this repo, consider whether the feature should/could be implemented in the other OpenTelemetry client libraries. If so, please open an issue on opentelemetry-specification first.

Steps to reproduce:

1. Instrument logging
2. Log outside of celery task
3. Log inside of celery task
4. Trigger celery task
5. Notice that the log from inside the task is not captured

Example:

from fastapi import FastAPI
from celery import Celery
import logging


logging.basicConfig(level=logging.INFO)
app = FastAPI()
celery = Celery('tasks', broker='redis://localhost:6379/0')
@app.get('/')
def root():
    #This log is captured
    logging.info(f'Begin Task')
    task = count_to_10000.delay()
    return {'task_id': task.id}
@celery.task
def count_to_10000():
    for i in range(1, 10001):
        if i % 1000 == 0:
            #This log not captured
            logging.info(f'Count: {i}')
        else:
            pass
    return 'Counting complete!'

Is your feature request related to a problem? Yes. Celery tasks can seemingly use the same longer as code outside a celery task. Yet, the logs from within the task do not appear to be captured.

Describe the solution you'd like The logging instrumentation (or perhaps the celery instrumentation) is updated to somehow capture logs from inside celery tasks.

Describe alternatives you've considered I have not found alternatives.

Additional context Datadog seems to have the same issue