Allow decision logs to be written to persistence_directory instead of getting discarded

[anderseknert]

There are currently a few ways to configure OPA so that decision logs exceeding some set threshold (in bytes or decisions per second) get discarded. While possibly useful in some scenarios there are many organizations where this would be unacceptable. For that reason it would be useful if there was an option to rather persist excess decision logs to the configure persistence_directory until things have cooled down, the remote endpoint is reachable, and so on.

Note that the main use case here is to avoid throwing away logs in case of sudden bursts or remote connectivity problems. This ticket does not mean to solve persistent disk storage of decision logs for auditing or archival purposes.

[tsandall]

Related: #2605

[stale[bot]]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.

[NickLarsenNZ]

I was hoping I could ship logs via OTLP (OpenTelemetry), and then let the opentelemetry-collector be configured for the next step (eg: redaction, sending to an external sink, etc...).

I see OPA supports OpenTelemetry tracing, so maybe not too much difficulty to add this?

It also saves OPA from having to cater to every desire of end users/operators.

[srenatus]

I see OPA supports OpenTelemetry tracing, so maybe not too much difficulty to add this?

It's used for traces only right now. Does the Go SDK already have OTel logs support?

[NickLarsenNZ]

Looks like it's still in development 😞

[stale[bot]]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days. Although currently inactive, the issue could still be considered and actively worked on in the future. More details about the use-case this issue attempts to address, the value provided by completing it or possible solutions to resolve it would help to prioritize the issue.