Capture request headers in the audit topic

[yeikel]

Issue submitter TODO list

• [x] I've searched for an already existing issues here
• [x] I'm running a supported version of the application which is listed here and the feature is not present there

Is your proposal related to a problem?

I expose the UI behind a proxy that receives and forwards the request. This proxy includes additional and meaningful headers in the request that are currently not captured.

That causes the following data to be persisted in the audit logs that is not enough to trace/audit the usage :

{
	"timestamp": "2025-04-15T20:05:09.872497460Z",
	"username": "Unknown",
	"clusterName": "kafka-cluster",
	"resources": [],
	"operation": "getTopics",
	"result": {
		"success": true
	}
}

Describe the feature you're interested in

1. Add a new entry in the configuration to specify the list of headers to log. For example, a tentative name could be audit-request-headers-properties

For example:

kafka:
  clusters:
    - name: local
      audit:
        topic-audit-enabled: true
        console-audit-enabled: true
        # The other properties were omitted for brevity
        # A list of request headers that should be persisted if present in the request
        audit-request-headers-properties:
          - User-Agent
          - my-custom-header    

2. Read and persist the headers in the audit log under a new configuration property such as requestHeaders

{
	"timestamp": "2025-04-15T20:05:09.872497460Z",
	"username": "Unknown",
	"clusterName": "kafka-cluster",
	"resources": [],
        "requestHeaders": {
         "User-Agent" : "PostmanRuntime/7.30.0",
         "my-custom-header" : "the custom value"
}, 
	"operation": "getTopics",
	"result": {
		"success": true
	}
}

Describe alternatives you've considered

I considered to this audit at my proxy but it will split the audit to two different places

Version you're running

b71a75371c3a75a4ed4df6772744f53e0f0f86af

Additional context

No response

[yeikel]

@Haarolean What are your thoughts about this one? I'd be interested to implement it if accepted

[germanosin]

@yeikel Thanks again for your contribution! Appreciate the feature description. Just one small addition from my side — let's include these headers only if they’re explicitly enabled in the config and specified as a list.

[yeikel]

@yeikel Thanks again for your contribution! Appreciate the feature description. Just one small addition from my side — let's include these headers only if they’re explicitly enabled in the config and specified as a list.

Thank you for the feedback @germanosin

I updated the issue to reflect these changes. Could you please re-review and share any other feedback?

Also, please confirm if I can pick this up to send a PR :)

[Haarolean]

@Haarolean What are your thoughts about this one? I'd be interested to implement it if accepted

sure thing!

[bhasinsaurav]

@Haarolean

I would love to work on this issue , if this is not resolved yet?

[Haarolean]

@Haarolean

I would love to work on this issue , if this is not resolved yet?

Hi @bhasinsaurav, feel free to tackle it