Add support for OTEL_LOG_LEVEL env variable

[codeboten]

The spec describes environment variables that should be supported to configure the log level, this feature request is to add support in the current implementation.

https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/sdk-environment-variables.md

[ffe4]

I'll take this up.

[codeboten]

Thanks!

[ffe4]

There is ongoing discussion about what the values for this should look like, or whether this variable should even exist right now in opentelemetry/opentelemetry-specification#920.

[github-actions[bot]]

This issue was marked stale due to lack of activity. It will be closed in 30 days.

[JPLachance]

Hello,

I recently realized that the open telemetry instrumentation is logging formatted JSON objects instead of JSON lines.

In the context of:

• an application running in Kubernetes
• a log collector like Filebeat that expects JSON lines

We get broken logs in Elasticsearch, because Filebeat is indexing each line of the JSON document separately (because each line ends with a \n).

Is there a way to tell opentelemetry-python to log in JSON lines?

The issue comes from https://github.com/open-telemetry/opentelemetry-python/blob/44e9b5531d91c8c52f58cd4ea3930cda5c1ed662/opentelemetry-sdk/src/opentelemetry/sdk/_logs/init.py#L101. The indent parameter of json.dumps must not be passed when running inside a Kubernetes pod with a log collector that expects JSON lines.

It would be ideal to get the control over both the logging level and the logging format.

For the logging indent, adding a new OTEL_LOG_INDENT environment variable that we can set to 0 would work.

Thank you!

[srikanthccv]

It would be ideal to get the control over both the logging level and the logging format.

I think you are mixing different things here. The OTEL_LOG_LEVEL is for setting the log level for the loggers used in internally by SDK not for your application logs.

For the logging indent, adding a new OTEL_LOG_INDENT environment variable that we can set to 0 would work.

Are you using the ConsoleLogExporter? That's primarily intended for debugging purposes. I don't see an issue in enabling you to configuring indent through func args but not sure if we can support the env. I am also wondering if you are actually looking for exporter that writes to file instead of an network destination (OTLP gRPC exporter).

[owais]

@JPLachance Could you please open a new issue for what you're reporting? Looks like we are talking about different things here :)

[JPLachance]

Hello,

We use the OTLPSpanExporter, with a fallback to the ConsoleSpanExporter when we cannot properly configure the OTLPSpanExporter.

That fallback made me fell into a rabbit hole. My bad 🤦🏼

[ffe4]

As there was a question on the linked PR #1069 I will post a quick update here.

I wanted to wait for the open spec issue opentelemetry-specification#920 regarding standardising the logging variables to be resolved before implementing yet another languages specific solution. There haven't really been any updates on that issue and at this point some languages have had their own solution for long enough that I think we can just implement our own as well.

As I don't have time right now, feel free to reassign if someone wants to drive this issue. If we decide on implementing an otel-python specific solution it should be easy enough to implement.

[anooprh]

Is the original issue of supporting OTEL_LOG_LEVEL env variable still open ?

Looks like https://github.com/open-telemetry/opentelemetry-python/pull/1069 was never merged. I can take this up as I'm looking to get started on this project.

[srikanthccv]

Yes, this is still open. The author of #1069 didn't have the time to re-work the PR after many changes got made to codebase.

[anooprh]

Given that there has been no updates on standardization of the logging levels in specifications, https://github.com/open-telemetry/opentelemetry-specification/issues/920 since @ffe4 worked on this last, I still don't see a clear path forward here.

Given that in the other languages (js and java from the thread ^) implemented what is most sensible as per the respective language, and python sdk is still not respecting the OTEL_LOG_LEVEL variable, I'm inclined to recreate whatever @ffe4 had in https://github.com/open-telemetry/opentelemetry-python/pull/1069 but on the updated version.

We will likely have to make changes to this once again when there is a consensus reached on https://github.com/open-telemetry/opentelemetry-specification/issues/920 on language independent/numeric logging levels.

cc : @srikanthccv

[srikanthccv]

I think we can start by supporting DEBUG, INFO and ERROR since they are more universal and expand later if the spec adds more. I looked at the prev PR, and I don't think we should set up the level only for SDK. This level should be used in other components such as propagator, exporters, and instrumentation.