tempesta
tempesta copied to clipboard
tempesta-tech
Chunked transfer encoding: PURGE+GET request connection is not closed
Scope
For PURGE requests with the X-Tempesta-Cache: get header set:
- Connection is not closed when chunked transfer encoding is used by backend
- When keepalive is used with chunked transfer encoding,
Content-Lengthvalue is concatenated with the next header:0via: 1.1 tempesta_fw (Tempesta FW pre-0.7.0)
WordPress https://github.com/tempesta-tech/tempesta-test/issues/290 home page purging example:
# curl --verbose --request PURGE --max-time 16 172.17.0.1 -H 'X-Tempesta-Cache: get'
* Trying 172.17.0.1:80...
* Connected to 172.17.0.1 (172.17.0.1) port 80 (#0)
> PURGE / HTTP/1.1
> Host: 172.17.0.1
> User-Agent: curl/7.83.1
> Accept: */*
> X-Tempesta-Cache: get
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Fri, 16 Sep 2022 10:25:42 GMT
< X-Powered-By: PHP/7.4.30
< Link: <http://127.0.0.1/index.php?rest_route=/>; rel="https://api.w.org/"
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=UTF-8
< Content-Length: 0via: 1.1 tempesta_fw (Tempesta FW pre-0.7.0)
< Server: Tempesta FW/pre-0.7.0
<
* Operation timed out after 16000 milliseconds with 0 bytes received
* Closing connection 0
curl: (28) Operation timed out after 16000 milliseconds with 0 bytes received
Raw response from WordPress:
HTTP/1.1 200 OK
Date: Fri, 16 Sep 2022 10:25:42 GMT
Server: Apache/2.4.54 (Debian)
X-Powered-By: PHP/7.4.30
Link: <http://127.0.0.1/index.php?rest_route=/>; rel="https://api.w.org/"
Vary: Accept-Encoding
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
\r\n
142b0\r\n
<!DOCTYPE html>
...
</html>
\r\n
0\r\n
\r\n
Testing
Test to reproduce: cache.test_purge.TestPurgeGetWithTransferEncoding
Why Tempesta should close the connection? There is no Connection header in the curl request - does RFC requires to close connection by default?
If connection is not closed, then chunked body is expected by a client, but body is cutted off by Tempesta.
And even if Connection: close is set by a client, header is replaced with Connection: keep-alive by Tempesta.
+Content-Length header has broken value instead of 0.
Session with the Connection: close set by a client:
# curl --verbose --max-time 16 172.17.0.1 -X PURGE -H 'X-Tempesta-Cache: get' -H 'Connection: close'
* Trying 172.17.0.1:80...
* Connected to 172.17.0.1 (172.17.0.1) port 80 (#0)
> PURGE / HTTP/1.1
> Host: 172.17.0.1
> User-Agent: curl/7.83.1
> Accept: */*
> X-Tempesta-Cache: get
> Connection: close
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Fri, 16 Sep 2022 13:09:36 GMT
< X-Powered-By: PHP/7.4.30
< Link: <http://127.0.0.1/index.php?rest_route=/>; rel="https://api.w.org/"
< Vary: Accept-Encoding
< Connection: close
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=UTF-8
< Content-Length: 0via: 1.1 tempesta_fw (Tempesta FW pre-0.7.0)
< Server: Tempesta FW/pre-0.7.0
<
* transfer closed with outstanding read data remaining
* Closing connection 0
curl: (18) transfer closed with outstanding read data remaining
Test added to check that Connection: close header is not passed to backend: cache.test_purge.TestPurgeGetWithTransferEncoding
It seems the problem is that the backend sends Transfer-Encoding: chunked and when Tempesta FW cuts off the response for X-Tempesta-Cache: get if doesn't cut off Transfer-Encoding.