vegeta icon indicating copy to clipboard operation
vegeta copied to clipboard

Vegeta sends RST_STREAM with 'cancel' error flag on a closed stream

Open martin-g opened this issue 3 years ago • 1 comments

Version and Runtime

Vegeta 12.8.3 Golang: go1.15.2 linux/amd64

Version: 12.8.3
Commit: d9b795aec8585a0fb435072f68d842d596c332de
Runtime: go1.14 linux/amd64
Date: 2020-03-25T11:03:54Z"

Expected Behaviour

After the server sends DATA with END_STREAM flag Vegeta should not send anything more in the same stream.

Actual Behaviour

Vegeta sends an HTTP2 request to the server. The HEADERS frame contains END_STREAM flag so after processing the request the server sets the stream to HALF_CLOSED_REMOTE state (as per https://tools.ietf.org/html/rfc7540#section-5.1). Then the server sends the response and the DATA frame contains the END_STREAM flag to fully close it. So far so good. But in some cases Vegeta also sends a new frame for the same stream with error code CANCEL (https://tools.ietf.org/html/rfc7540#section-7). Spec complaint server may complain here with connection error (https://tools.ietf.org/html/rfc7540#section-5.4.1)

Steps to Reproduce

I've found this issue while trying to load test Apache Tomcat web server.

  1. Download Tomcat from https://tomcat.apache.org/download-90.cgi
  2. Unzip/Untar it
  3. Edit /path/to/tomcat/conf/server.conf and remove the comments around this connector: https://github.com/apache/tomcat/blob/27c4109a7a686b71454205faab909e5e1f041b18/conf/server.xml#L102-L113
  4. Generate self-signed certificate and update the paths for certificateKeyFile and certificateFile attributes. The certificateChainFile attribute could be removed.
  5. Start Tomcat with: cd /path/to/tomcat/, ./bin/startup.sh
  6. Check ./logs/catalina.out that there are no errors
  7. Run Vegeta against Tomcat:
     echo -e '{"method": "GET", "url": "https://localhost:8443/examples/index.html"}' | vegeta attack -format=json -http2 -rate=0 -max-workers=100 -insecure -duration=5s | vegeta encode > /tmp/http2.json; and vegeta report -type=json /tmp/http2.json | jq . 

Additional Context

There is a discussion about this at https://lists.apache.org/thread.html/r6f9ce1521346d5ced72c82a4b5d882a624514c9a7b308ad54f064ba1%40%3Cusers.tomcat.apache.org%3E

martin-g avatar Sep 21 '20 19:09 martin-g

The issue seems related to https://github.com/golang/go/issues/27208 Since https://github.com/golang/net/commit/1c5f79cfb1642860bbe00b6cfce66700c01e04f6 if there is a problem with reading the response body the http client will send RST_STREAM with Cancel error. But since the server has closed the stream the processing of the RST_STREAM frame will lead to more errors.

martin-g avatar Sep 22 '20 14:09 martin-g