feat(jans-auth-server): Add structured logging with OpenTelemetry integration for enhanced observability

[vivekkumarrathour]

Description

Enhance the jans-auth-server logging capabilities by implementing structured logging with OpenTelemetry integration. This will enable better observability, debugging, and monitoring of authentication flows in production environments.

Development target issue

Implement comprehensive structured logging throughout jans-auth-server with OpenTelemetry (OTel) support to enable distributed tracing and better observability.

Tasks

• [ ] Replace current logging framework with structured logging (JSON format)
• [ ] Add correlation IDs to track requests across microservices
• [ ] Implement OpenTelemetry SDK integration for traces and spans
• [ ] Add configurable log levels per component (auth, token, userinfo, etc.)
• [ ] Create span annotations for key authentication events (login attempts, token generation, consent flows)
• [ ] Add metrics export capabilities (Prometheus format)
• [ ] Implement log sampling for high-volume production environments
• [ ] Add documentation for configuring log exporters (OTLP, Jaeger, Zipkin)
• [ ] Create example configurations for common observability platforms (Datadog, New Relic, Elastic APM)
• [ ] Add performance tests to ensure minimal overhead

Benefits

• Better Debugging: Structured logs with correlation IDs make it easier to trace authentication flows
• Production Monitoring: Real-time insights into system behavior and performance
• Compliance: Detailed audit trails for security and compliance requirements
• Performance Analysis: Identify bottlenecks in authentication flows
• Integration: Works with modern observability platforms out of the box