martian
martian copied to clipboard
Published
20 hours ago •
google
google
HTTP/2 issues with curl - peer does not support HTTP/2 properly
Hello, I'm currently attempting to build an HTTP2 logging proxy using Martian. The test snippet below works fine with Firefox; however, when posting data with curl via the proxy, I encounter errors. The following figures shows the results when going via the HTTP2 proxy (code provided).
GET works fine:
$ curl -v -x 127.0.0.1:8080 -k https://example.com
* Expire in 0 ms for 6 (transfer 0x55f31d912fb0)
* Trying 127.0.0.1...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55f31d912fb0)
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to example.com:443
> CONNECT example.com:443 HTTP/1.1
...snip...
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55f31d912fb0)
> GET / HTTP/2
> Host: example.com
> User-Agent: curl/7.64.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< age: 303681
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Tue, 01 Mar 2022 05:56:07 GMT
< etag: "3147526947+ident"
< expires: Tue, 08 Mar 2022 05:56:07 GMT
< last-modified: Thu, 17 Oct 2019 07:18:26 GMT
< server: ECS (sab/572B)
< vary: Accept-Encoding
< x-cache: HIT
< content-length: 1256
<
<!doctype html>
<html>
<head>
<title>Example Domain</title>
...snip...
With the following log:
##Headers##
:method: GET
:path: /
:scheme: https
:authority: example.com
user-agent: curl/7.64.0
accept: */*
##Headers##
:status: 200
age: 303681
cache-control: max-age=604800
content-type: text/html; charset=UTF-8
date: Tue, 01 Mar 2022 05:56:07 GMT
etag: "3147526947+ident"
expires: Tue, 08 Mar 2022 05:56:07 GMT
last-modified: Thu, 17 Oct 2019 07:18:26 GMT
server: ECS (sab/572B)
vary: Accept-Encoding
x-cache: HIT
content-length: 1256
##Message##
<!doctype html>
<html>
<head>
...snip...
However, POST requests fail:
$ curl -v -x 127.0.0.1:8080 -k https://example.com -d 'foo=bar'
* Expire in 0 ms for 6 (transfer 0x55995f82efb0)
* Trying 127.0.0.1...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55995f82efb0)
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to example.com:443
> CONNECT example.com:443 HTTP/1.1
...snip...
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55995f82efb0)
> POST / HTTP/2
> Host: example.com
> User-Agent: curl/7.64.0
> Accept: */*
> Content-Length: 7
> Content-Type: application/x-www-form-urlencoded
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* We are completely uploaded and fine
* http2 error: Remote peer returned unexpected data while we expected SETTINGS frame. Perhaps, peer does not support HTTP/2 properly.
* Connection #0 to host 127.0.0.1 left intact
curl: (16) Error in the HTTP2 framing layer
Can you please advise? Is this an issue with my implementation or an underlying problem with Martian? The same results occur regardless of whether i use my logging processor or set the processors to nil.
Proxy code to replicate the issue below:
package main
import (
"crypto/tls"
"fmt"
"log"
"net"
"net/http"
"net/url"
"time"
"github.com/google/martian/v3"
"github.com/google/martian/v3/h2"
"github.com/google/martian/v3/mitm"
"golang.org/x/net/http2"
"golang.org/x/net/http2/hpack"
)
type loggingProcessor struct {
sink h2.Processor
}
func (p *loggingProcessor) Header(
headers []hpack.HeaderField,
streamEnded bool,
priority http2.PriorityParam,
) error {
println("##Headers##")
for _, header := range headers {
println(header.Name + ": " + header.Value)
}
return p.sink.Header(headers, streamEnded, priority)
}
func (p *loggingProcessor) Data(data []byte, streamEnded bool) error {
println("##Message##")
println(string(data))
return p.sink.Data(data, streamEnded)
}
func (p *loggingProcessor) Priority(priority http2.PriorityParam) error {
return p.sink.Priority(priority)
}
func (p *loggingProcessor) RSTStream(errCode http2.ErrCode) error {
return p.sink.RSTStream(errCode)
}
func (p *loggingProcessor) PushPromise(promiseID uint32, headers []hpack.HeaderField) error {
return p.sink.PushPromise(promiseID, headers)
}
func main() {
fmt.Println("Http2 Proxy Test")
p := martian.NewProxy()
x509c, priv, _ := mitm.NewAuthority("martian.proxy", "Martian Authority", 30*24*time.Hour)
mc, err := mitm.NewConfig(x509c, priv)
if err != nil {
fmt.Printf("creating mitm config: %v", err)
return
}
mc.SetValidity(time.Hour)
mc.SetOrganization("Martian Proxy")
mc.SetH2Config(&h2.Config{
AllowedHostsFilter: func(_ string) bool { return true },
StreamProcessorFactories: []h2.StreamProcessorFactory{
func(url *url.URL, sinks *h2.Processors) (h2.Processor, h2.Processor) {
procCtoS := &loggingProcessor{
sink: sinks.ForDirection(h2.ClientToServer),
}
procStoC := &loggingProcessor{
sink: sinks.ForDirection(h2.ServerToClient),
}
return procCtoS, procStoC
//return nil, nil
},
},
EnableDebugLogs: true,
})
p.SetMITM(mc)
tr := &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true,
},
}
p.SetRoundTripper(tr)
l, err := net.Listen("tcp", "0.0.0.0:8080")
if err != nil {
log.Fatal(err)
}
if err != nil {
fmt.Printf("creating proxy: %v", err)
return
}
done := make(chan bool)
go func(done chan bool) {
p.Serve(l)
<-done
}(done)
fmt.Printf("serving on 0.0.0.0:8080\n")
<-done
}
