vegeta
vegeta copied to clipboard
Vegeta sends RST_STREAM with 'cancel' error flag on a closed stream
Version and Runtime
Vegeta 12.8.3 Golang: go1.15.2 linux/amd64
Version: 12.8.3
Commit: d9b795aec8585a0fb435072f68d842d596c332de
Runtime: go1.14 linux/amd64
Date: 2020-03-25T11:03:54Z"
Expected Behaviour
After the server sends DATA with END_STREAM flag Vegeta should not send anything more in the same stream.
Actual Behaviour
Vegeta sends an HTTP2 request to the server. The HEADERS frame contains END_STREAM flag so after processing the request the server sets the stream to HALF_CLOSED_REMOTE state (as per https://tools.ietf.org/html/rfc7540#section-5.1). Then the server sends the response and the DATA frame contains the END_STREAM flag to fully close it. So far so good. But in some cases Vegeta also sends a new frame for the same stream with error code CANCEL (https://tools.ietf.org/html/rfc7540#section-7). Spec complaint server may complain here with connection error (https://tools.ietf.org/html/rfc7540#section-5.4.1)
Steps to Reproduce
I've found this issue while trying to load test Apache Tomcat web server.
- Download Tomcat from https://tomcat.apache.org/download-90.cgi
- Unzip/Untar it
- Edit /path/to/tomcat/conf/server.conf and remove the comments around this connector: https://github.com/apache/tomcat/blob/27c4109a7a686b71454205faab909e5e1f041b18/conf/server.xml#L102-L113
- Generate self-signed certificate and update the paths for
certificateKeyFile
andcertificateFile
attributes. ThecertificateChainFile
attribute could be removed. - Start Tomcat with:
cd /path/to/tomcat/
,./bin/startup.sh
- Check
./logs/catalina.out
that there are no errors - Run Vegeta against Tomcat:
echo -e '{"method": "GET", "url": "https://localhost:8443/examples/index.html"}' | vegeta attack -format=json -http2 -rate=0 -max-workers=100 -insecure -duration=5s | vegeta encode > /tmp/http2.json; and vegeta report -type=json /tmp/http2.json | jq .
Additional Context
There is a discussion about this at https://lists.apache.org/thread.html/r6f9ce1521346d5ced72c82a4b5d882a624514c9a7b308ad54f064ba1%40%3Cusers.tomcat.apache.org%3E
The issue seems related to https://github.com/golang/go/issues/27208
Since https://github.com/golang/net/commit/1c5f79cfb1642860bbe00b6cfce66700c01e04f6 if there is a problem with reading the response body the http client will send RST_STREAM with Cancel
error. But since the server has closed the stream the processing of the RST_STREAM frame will lead to more errors.