aws-sdk-go-v2 icon indicating copy to clipboard operation
aws-sdk-go-v2 copied to clipboard

Panic when ungzip with http timeout

Open zhengkai opened this issue 2 years ago • 0 comments

Describe the bug

When I develop kinesis-related programs locally, there are a lot of shards, and insufficient local bandwidth will cause timeouts, and panics are very likely to occur at this time. (index out of range or slice bounds out of range)

Should prevent decompressing http.Body when it times out

I have this problem in the kinesis client, but I think the problem is common

Expected Behavior

reutrn error, not panic

Current Behavior

in [email protected]/aws/transport/http/timeout_read_closer.go, there is a new goroutine to r.reader.Read(b), so I can't recover panic

func (r *timeoutReadCloser) Read(b []byte) (int, error) {
	timer := time.NewTimer(r.duration)
	c := make(chan readResult, 1)

	go func() {
		n, err := r.reader.Read(b)
		timer.Stop()
		c <- readResult{n: n, err: err}
	}()

panic example 1/2:

panic: runtime error: slice bounds out of range [21391:0]

goroutine 10107 [running]:
compress/flate.(*decompressor).Read(0xc00073c600, {0xc0085c6000, 0x197e00, 0x406045?})
    /usr/local/go/src/compress/flate/inflate.go:338 +0x20c
compress/gzip.(*Reader).Read(0xc000f96840, {0xc0085c6000, 0x197e00, 0x197e00})
    /usr/local/go/src/compress/gzip/gunzip.go:252 +0xbb
net/http.(*gzipReader).Read(0xc000a15ba0, {0xc0085c6000, 0x197e00, 0x197e00})
    /usr/local/go/src/net/http/transport.go:2838 +0x18a
github.com/aws/aws-sdk-go-v2/aws/transport/http.(*timeoutReadCloser).Read.func1()
    /go/pkg/mod/github.com/aws/[email protected]/aws/transport/http/timeout_read_closer.go:48 +0x47
created by github.com/aws/aws-sdk-go-v2/aws/transport/http.(*timeoutReadCloser).Read
    /go/pkg/mod/github.com/aws/[email protected]/aws/transport/http/timeout_read_closer.go:47 +0x11e

panic example 2/2:

panic: runtime error: index out of range [4096] with length 4096

goroutine 10827 [running]:
bufio.(*Reader).ReadByte(0xc001659ec0)
    /usr/local/go/src/bufio/bufio.go:267 +0xad
compress/flate.(*decompressor).huffSym(0xc000fb8000, 0xc000fb8028)
    /usr/local/go/src/compress/flate/inflate.go:719 +0x102
compress/flate.(*decompressor).huffmanBlock(0x0?)
    /usr/local/go/src/compress/flate/inflate.go:494 +0x45
compress/flate.(*decompressor).Read(0xc000fb8000, {0xc001350000, 0x2000, 0xc00509bd40?})
    /usr/local/go/src/compress/flate/inflate.go:347 +0x7e
compress/gzip.(*Reader).Read(0xc0000fbb80, {0xc001350000, 0x2000, 0x2000})
    /usr/local/go/src/compress/gzip/gunzip.go:252 +0xbb
net/http.(*gzipReader).Read(0xc001bd67e0, {0xc001350000, 0x2000, 0x2000})
    /usr/local/go/src/net/http/transport.go:2838 +0x18a
github.com/aws/aws-sdk-go-v2/aws/transport/http.(*timeoutReadCloser).Read.func1()
    /go/pkg/mod/github.com/aws/[email protected]/aws/transport/http/timeout_read_closer.go:48 +0x47
created by github.com/aws/aws-sdk-go-v2/aws/transport/http.(*timeoutReadCloser).Read
    /go/pkg/mod/github.com/aws/[email protected]/aws/transport/http/timeout_read_closer.go:47 +0x11e

Reproduction Steps

Read shards flood with limited bandwidth, let the client read irregular corrupted data

Possible Solution

No response

Additional Information/Context

How I can stop client use timeoutReadCloser ?or disable AddResponseReadTimeoutMiddleware ( all of them are in aws/transport/http)

AWS Go SDK V2 Module Versions Used

github.com/aws/aws-sdk-go-v2 v1.16.11
github.com/aws/aws-sdk-go-v2/config v1.15.15
github.com/aws/aws-sdk-go-v2/credentials v1.12.10
github.com/aws/aws-sdk-go-v2/service/dynamodb v1.15.12
github.com/aws/aws-sdk-go-v2/service/kinesis v1.15.13
github.com/aws/aws-sdk-go-v2/service/opensearch v1.10.4
github.com/aws/smithy-go v1.12.1
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.4 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.9 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.18 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.12 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.16 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.7.11 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.9 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.11.13 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.16.10 // indirect

Compiler and Version used

go version go1.19.1 linux/amd64

Operating System and version

Ubuntu 22.04.1 LTS

zhengkai avatar Sep 19 '22 10:09 zhengkai