envoy
envoy copied to clipboard
envoyproxy
Ext proc http basic functionality support
Risk Level: low Testing: n/a Docs Changes: n/a Release Notes: inline Fixes:
Description: This is to address the 3rd step of issue: https://github.com/envoyproxy/envoy/issues/35488, i.e, integrate the ext_proc HTTP client to ext_proc filter. With this PR, the basic functionalities to have Envoy ext_proc filter talk to a HTTP server using HTTP messages are accomplished.
This is the follow up of PR: https://github.com/envoyproxy/envoy/pull/35676
This PR adds the support of sending ext_proc call out messages to the HTTP side stream server. The proto messages: ProcessingRequest/ProcessingResponse are transcoded into JSON text and encoded in the body of a HTTP message.
The processing flow works this way:
- When Envoy needs to send call out messages to the side stream server, either header, body , or trailer, Envoy construct a ProcessingRequest proto message.
- This ProcessingRequest message is transcoded into a JSON text. This JSON text is then added as a body to a HTTP "POST" message. This HTTP message is sent to the side stream HTTP server.
- The HTTP server receives the "POST" message. Transcoding the body, i.e, a JSON text, into a ProcessingRequest message.
- The HTTP server then performs normal header/body/trailer mutation as existing, and put them into a ProcessingResponse proto message.
- The HTTP server then transcode the ProcessingResponse into JSON text, and sends back a 200 response with this JSON text as body.
- After receives the 200 response, Envoy then convert the body from JSON text back to a ProcessingResponse proto message. Then continue processing this ProcessingResponse.
Overall, the ext_proc gRPC and HTTP share identical state machine and logic. The only difference is in the message sending mechanism. One using gRPC, and the ProcessingRequest/ProcessingResponse messages are sent as proto messages. The other sends messages using HTTP, with the ProcessingRequest/ProcessingResponse proto messages are transcoded into JSON text, and encoded as the body in the HTTP messages.
@yanjunxiang-google thoughts on how the receiving server correlates HTTP requests for a given dataplane streams? There are two things here and I think we need to do both, to avoid making users solve a distributed system coordination problem:
- Use HTTP session affinity. I'm not sure if we have a strong session affinity mechanism here that doesn't rely on cookies (and hence doesn't work for ext_proc initiated HTTP) though.
- Attach a UUID (e.g. the x-envoy-request-id) as a distinguished field to the JSON to allow the server to correlate.
@yanjunxiang-google thoughts on how the receiving server correlates HTTP requests for a given dataplane streams? There are two things here and I think we need to do both, to avoid making users solve a distributed system coordination problem:
- Use HTTP session affinity. I'm not sure if we have a strong session affinity mechanism here that doesn't rely on cookies (and hence doesn't work for ext_proc initiated HTTP) though.
- Attach a UUID (e.g. the x-envoy-request-id) as a distinguished field to the JSON to allow the server to correlate.
Here is related design and internal discussion https://docs.google.com/document/d/1UDPm1J4txb2P2oCnl98qClD2FMZWu3SdnRnoHJZxGBU/edit?disco=AAABM_WD00c
@htuch Regarding point 1 above, Should we consider suggesting gRPC as the recommend protocol for strong affinity.
@yanjunxiang-google thoughts on how the receiving server correlates HTTP requests for a given dataplane streams? There are two things here and I think we need to do both, to avoid making users solve a distributed system coordination problem:
- Use HTTP session affinity. I'm not sure if we have a strong session affinity mechanism here that doesn't rely on cookies (and hence doesn't work for ext_proc initiated HTTP) though.
- Attach a UUID (e.g. the x-envoy-request-id) as a distinguished field to the JSON to allow the server to correlate.
Good suggestions!
I put this as the step 7 of the issue: https://github.com/envoyproxy/envoy/issues/35488, and plan to add this in my next PR. BTW, we will only remove the [#not-implemented-hide:] from the API : https://github.com/envoyproxy/envoy/blob/84c8da7540e4e5064f6a6dffc66825668f6220e4/api/envoy/extensions/filters/http/ext_proc/v3/ext_proc.proto#L134 after all the steps listed in the issue https://github.com/envoyproxy/envoy/issues/35488 completed.
Please continue review this PR and let me know the comments.
@yanjunxiang-google thoughts on how the receiving server correlates HTTP requests for a given dataplane streams? There are two things here and I think we need to do both, to avoid making users solve a distributed system coordination problem:
- Use HTTP session affinity. I'm not sure if we have a strong session affinity mechanism here that doesn't rely on cookies (and hence doesn't work for ext_proc initiated HTTP) though.
- Attach a UUID (e.g. the x-envoy-request-id) as a distinguished field to the JSON to allow the server to correlate.
Here is related design and internal discussion https://docs.google.com/document/d/1UDPm1J4txb2P2oCnl98qClD2FMZWu3SdnRnoHJZxGBU/edit?disco=AAABM_WD00c
@htuch Regarding point 1 above, Should we consider suggesting gRPC as the recommend protocol for strong affinity.
Thanks for the info!
To add the support of 2) is straightforward. It looks to me ext_authz only have 2) at this moment.
I think we should recommend gRPC for affinity, but I know inevitably we will end up with folks who need to solve this problem the moment we support > 1 event at a time on the HTTP variant. I think it's pretty rare to solve this via some distributed state sharing mechanism on the service callout, so you will need affinity.
At a minimum, please do (2) and document as appropriately the limitations for users of the HTTP variant, leave a bug open for (1), thanks.
I think we should recommend gRPC for affinity, but I know inevitably we will end up with folks who need to solve this problem the moment we support > 1 event at a time on the HTTP variant. I think it's pretty rare to solve this via some distributed state sharing mechanism on the service callout, so you will need affinity.
At a minimum, please do (2) and document as appropriately the limitations for users of the HTTP variant, leave a bug open for (1), thanks.
The x-request-id is added in the change now.
@yanjunxiang-google Could you take a look at CI failures? The asan failures look legit.
/wait
@yanjunxiang-google Could you take a look at CI failures? The asan failures look legit.
/wait
Yes, working on it now.
CC @envoyproxy/api-shepherds: Your approval is needed for changes made to (api/envoy/|docs/root/api-docs/).
envoyproxy/api-shepherds assignee is @wbpcode
CC @envoyproxy/api-watchers: FYI only for changes made to (api/envoy/|docs/root/api-docs/).
:cat:
Caused by: https://github.com/envoyproxy/envoy/pull/35740 was synchronize by yanjunxiang-google.
![repokitteh-read-only[bot] avatar](https://avatars.githubusercontent.com/in/16729?v=4 "Published by a pub.dev verified publisher"){kind=small}
@tyxia I am scaling back to only support forwarding headers in this PR, and leaving body and trailers forwarding as future tasks. This is agreed in our offline discussion before.
Docs for this Pull Request will be rendered here:
https://storage.googleapis.com/envoy-pr/35740/docs/index.html
The docs are (re-)rendered each time the CI envoy-presubmit (precheck docs) job completes.
:cat:
Caused by: a https://github.com/envoyproxy/envoy/pull/35740#issuecomment-2363186220 was created by @wbpcode.
Please check the list int the API's comments. rst's format is different with .md
Needs main merge. I'm having a little trouble following whether we are not blocked on reviewers or on author activity.
/wait
Needs main merge. I'm having a little trouble following whether we are not blocked on reviewers or on author activity.
/wait
This PR is currently blocked by another PR I am working on: https://github.com/envoyproxy/envoy/pull/36228
Docs for this Pull Request will be rendered here:
https://storage.googleapis.com/envoy-pr/35740/docs/index.html
The docs are (re-)rendered each time the CI envoy-presubmit (precheck docs) job completes.
:cat:
Caused by: a https://github.com/envoyproxy/envoy/pull/35740#issuecomment-2379364576 was created by @yanjunxiang-google.