Improve trace logs

[AmaliMatharaarachchi]

Describe your problem(s)

There's room for improvement in the following areas.

1. Add Interceptor trace logs

• Interceptor lua scripts should contain before and after message transformation logs. message body, headers, trailers, metadata etc should be included to trace/debug logs for before and after message transformation in request and response methods.

2. Add enforcer trace logs add trace logs for the message body, headers, trailers, metadata, etc after applying all enforcer filters.

3. Improve / verify ext_authz filter and router filters trace logs

• Adding trace logs in router filter in envoy (contribute to envoy, if existing trace logs are lacking), So we can see what backend request would look like after all transformations coming from ext_authz(enforcer) and lua(interceptors).

Describe your solution

Existing HTTP request flow; user request --1--> envoy --2--> ext_authz filter --3--> enforcer --4--> ext_authz filter--5--> lua --6--> router filter --7--> upstream --8--> user response

Tracing the above flow will be as follows, 1 and 8 = observe actual client request and response ----> using envoy access logs 2 and 5 = observe transformations from ext_authz filter ---> using envoy trace logs of ext_authz filter (might need to contribute to envoy if existing trace logs are inadequate) 3 and 4 = observe transformation from enforcer filters ---> using enforcer trace logs (add trace logs before applying any filter and after applying all filters) 5 and 6 = observe transformation from interceptors ---> using interceptor lua script's trace log (add trace logs before and after the external service call) 7 = actual request sent to the upstream --> using envoy trace logs of router filter (might need to contribute to envoy if existing trace logs are inadequate)

How will you implement it

---

Optional Fields

Related Issues:

Suggested Labels:

Suggested Assignees:

[Amila-Rukshan]

Findings

As I have checked we already have the envoy level logs see the request / response changes

logs	configuration	status
router <-> enforcer (ext auth filter )	--component-log-level ext_authz:trace in router trailing args	ok
router <-> interceptors (lua filter)	enabling [wireLogs] in log_config.toml	ok
router <-> client/downstream	--component-log-level http:debug in router trailing args	ok
router -> upstream request	--component-log-level router:debug in router trailing args	ok
upstream -> router response	--component-log-level router:debug in router trailing args	need envoy changes

To get the logs mentioned in the last row of the above table, we need following changes in envoy repo https://github.com/envoyproxy/envoy/pull/21922

With the above changes we get the following log output:

[2022-06-28 12:00:32.036][43452151][debug][router] [source/common/router/upstream_request.cc:151] [C0][S4494666626217692216] upstream response headers:
':status', '200'
'x-powered-by', 'Express'
'content-type', 'text/plain'
'trailer', 'Content-MD5'
'trialer', 'Time-Taken'
'date', 'Tue, 28 Jun 2022 06:30:32 GMT'
'connection', 'keep-alive'
'keep-alive', 'timeout=5'
'transfer-encoding', 'chunked'

[2022-06-28 12:00:32.036][43452151][debug][router] [source/common/router/router.cc:1351] [C0][S4494666626217692216] upstream headers complete: end_stream=false
[2022-06-28 12:00:32.038][43452151][debug][router] [source/common/router/upstream_request.cc:204] [C0][S4494666626217692216] upstream response trailers:
'content-md5', '7895bf4b8828b55ceaf47747b4bca667'