envoy
envoy copied to clipboard
envoyproxy
Lua HttpCall response cookies are missing
Title: HttpCall response cookies are missing
Description:
After recieiving the Lua httpCall response only the last "Set-Cookie" header is exists in the response headers table.
Code:
local headers = {} headers[":method"] = method headers[":path"] = path headers[":authority"] = request_handle:headers():get("Host") local rheaders, rbody = request_handle:httpCall("cluster_external_api",headers, body, timeout) for key, value in pairs(rheaders) do print(key, value) end
Sample of tcpdump response:
HTTP/1.1 200 OK Content-Type: text/html; charset=utf-8 Transfer-Encoding: chunked Connection: keep-alive Set-Cookie: __uzma=de72db3c-d421-4bd6-98f5-a7b351e0c2db; HttpOnly; path=/; Expires=Wed, 11-Jan-23 13:47:21 GMT ; Max-Age=15724800; SameSite=Lax Set-Cookie: __uzmb=1657720041; HttpOnly; path=/; Expires=Wed, 11-Jan-23 13:47:21 GMT ; Max-Age=15724800; SameSite=Lax Set-Cookie: __uzme=9373; HttpOnly; path=/; Expires=Wed, 11-Jan-23 13:47:21 GMT ; Max-Age=15724800; SameSite=Lax Set-Cookie: __uzmc=719701042365; HttpOnly; path=/; Expires=Wed, 11-Jan-23 13:47:21 GMT ; Max-Age=15724800; SameSite=Lax Set-Cookie: __uzmd=1657720041; HttpOnly; path=/; Expires=Wed, 11-Jan-23 13:47:21 GMT ; Max-Age=15724800; SameSite=Lax x-rdwr-oop-request-status: allowed ShieldSquare-Response: 0 Server: RemoveIdentity
Log of printing the headers inside the for loop:
shieldsquare-response 0 server RemoveIdentity x-envoy-upstream-service-time 235 :status 200 content-type text/html; charset=utf-8 x-rdwr-oop-request-status allowed transfer-encoding chunked connection keep-alive set-cookie __uzmd=1657720041; HttpOnly; path=/; Expires=Wed, 11-Jan-23 13:47:21 GMT ; Max-Age=15724800; SameSite=Lax
You can see in the tcpdump I get more than one cookie to store in the browser but the Lua function only reads the lsat one
*Envoy version: *
envoy version: d362e791eb9e4efa8d87f6d878740e72dc8330ac/1.18.2/clean-getenvoy-76c310e-envoy/RELEASE/BoringSSL
Note: The request is being sent correctry and the reponse is OK There is an issue just with the cookies of the external API reponse
Please assist me to correct the issue
Regards, Daniel Y.
I checked code of lua filter. The response headers of httpCall will be stored to key/value table. Multiple same keys is not supported.
But I think we should try to solve this problem.
@wbpcode OK thanks! you think you can insert this fix to the next 1.23 series ?
@daniel-yaba new features are unlikely to be added to an existing release branch unless there is a compelling reason to do so
not sure if this is a bugfix or feature, either way i think its more likely a target for the next release
@phlax can you point me to the file of the httpCall function to check if I can help with this one ? I thin it is a bugfix becase rfc of http should have the capability so parse more that one cookie in the response
better ask @wbpcode re where to fix
re backporting - if we can fix, and it is a fix, and someone is willing to backport, then its defo possible
Check code here. The response headers will be convert to Lua table. You may need con-cat all the value of same key and then push it to the state.
https://github.com/envoyproxy/envoy/blob/b00201133123a88e5c73adfdf067dc1003566a66/source/extensions/filters/http/lua/lua_filter.cc#L316-L332
@wbpcode Sorry I didn't have the time to deal with it I will appriciate if you will supply a fix for this one
@wbpcode Sorry I didn't have the time to deal with it I will appriciate if you will supply a fix for this one
i will try fix it.
if we can resolve this within the next day or so (and @wbpcode thinks its a good idea) we can backport to 1.23 in the next release
Hi, @StarryVae is there any progress? Or any help is needed? 😸
Sorry for late, if it is urgent, @wbpcode please just fix it, thanks.
I am trying to fix this problem. However, I am not sure could we fold the set-cookie headers correctly.
If we cannot fold the set-cookie correctly and want keep all the values, then may be we cannot push headers to Lua state as string table. 🤔
cc @mattklein123
It looks like the get functions handle multiple values: https://github.com/envoyproxy/envoy/blob/c58b4eaab93cdc13a786c246dd75cdb26b7a7ce0/source/extensions/filters/http/lua/wrappers.cc#L43-L65
So I think for the iterator we just also need to support the all values version?
It looks like the get functions handle multiple values: https://github.com/envoyproxy/envoy/blob/c58b4eaab93cdc13a786c246dd75cdb26b7a7ce0/source/extensions/filters/http/lua/wrappers.cc#L43-L65
So I think for the iterator we just also need to support the all values version?
yeah, our final target is to support iterater all the values of response headers of httpCall.
But it's a little different with the get functions. There's no headers wrapper for response headers of httpCall.
The httpCall in lua will return a normal lua table as the headers. The key of table should be unique and value of table pair is single string.
I had tried to fold multiple headers to this single string, but I am not sure how to handle the 'set-cookie' correctly.
Or we can add a new optional parameter in the lua httpCall. If this parameter is set then the httpCall will retuen a table with value type of table? Then we can set multiple values for same header key.
And I just find a new bug. 🤣 The get functions not handled the cookie header correctly. The , will be used as separator.
Yeah if we want to handle folding it will likely need to be special cased. I'm not sure the best thing to do here. If we don't care about folding, comma delimited is probably fine and could be used for the HTTP call result as well. Or your idea to return a table of tables would be fine also, though that would need to be a new option for back compat as you point out.