aws-sdk-go-v2
aws-sdk-go-v2 copied to clipboard
Panic when ungzip with http timeout
Describe the bug
When I develop kinesis-related programs locally, there are a lot of shards, and insufficient local bandwidth will cause timeouts, and panics are very likely to occur at this time. (index out of range or slice bounds out of range)
Should prevent decompressing http.Body when it times out
I have this problem in the kinesis client, but I think the problem is common
Expected Behavior
reutrn error, not panic
Current Behavior
in [email protected]/aws/transport/http/timeout_read_closer.go, there is a new goroutine to r.reader.Read(b), so I can't recover panic
func (r *timeoutReadCloser) Read(b []byte) (int, error) {
timer := time.NewTimer(r.duration)
c := make(chan readResult, 1)
go func() {
n, err := r.reader.Read(b)
timer.Stop()
c <- readResult{n: n, err: err}
}()
panic example 1/2:
panic: runtime error: slice bounds out of range [21391:0]
goroutine 10107 [running]:
compress/flate.(*decompressor).Read(0xc00073c600, {0xc0085c6000, 0x197e00, 0x406045?})
/usr/local/go/src/compress/flate/inflate.go:338 +0x20c
compress/gzip.(*Reader).Read(0xc000f96840, {0xc0085c6000, 0x197e00, 0x197e00})
/usr/local/go/src/compress/gzip/gunzip.go:252 +0xbb
net/http.(*gzipReader).Read(0xc000a15ba0, {0xc0085c6000, 0x197e00, 0x197e00})
/usr/local/go/src/net/http/transport.go:2838 +0x18a
github.com/aws/aws-sdk-go-v2/aws/transport/http.(*timeoutReadCloser).Read.func1()
/go/pkg/mod/github.com/aws/[email protected]/aws/transport/http/timeout_read_closer.go:48 +0x47
created by github.com/aws/aws-sdk-go-v2/aws/transport/http.(*timeoutReadCloser).Read
/go/pkg/mod/github.com/aws/[email protected]/aws/transport/http/timeout_read_closer.go:47 +0x11e
panic example 2/2:
panic: runtime error: index out of range [4096] with length 4096
goroutine 10827 [running]:
bufio.(*Reader).ReadByte(0xc001659ec0)
/usr/local/go/src/bufio/bufio.go:267 +0xad
compress/flate.(*decompressor).huffSym(0xc000fb8000, 0xc000fb8028)
/usr/local/go/src/compress/flate/inflate.go:719 +0x102
compress/flate.(*decompressor).huffmanBlock(0x0?)
/usr/local/go/src/compress/flate/inflate.go:494 +0x45
compress/flate.(*decompressor).Read(0xc000fb8000, {0xc001350000, 0x2000, 0xc00509bd40?})
/usr/local/go/src/compress/flate/inflate.go:347 +0x7e
compress/gzip.(*Reader).Read(0xc0000fbb80, {0xc001350000, 0x2000, 0x2000})
/usr/local/go/src/compress/gzip/gunzip.go:252 +0xbb
net/http.(*gzipReader).Read(0xc001bd67e0, {0xc001350000, 0x2000, 0x2000})
/usr/local/go/src/net/http/transport.go:2838 +0x18a
github.com/aws/aws-sdk-go-v2/aws/transport/http.(*timeoutReadCloser).Read.func1()
/go/pkg/mod/github.com/aws/[email protected]/aws/transport/http/timeout_read_closer.go:48 +0x47
created by github.com/aws/aws-sdk-go-v2/aws/transport/http.(*timeoutReadCloser).Read
/go/pkg/mod/github.com/aws/[email protected]/aws/transport/http/timeout_read_closer.go:47 +0x11e
Reproduction Steps
Read shards flood with limited bandwidth, let the client read irregular corrupted data
Possible Solution
No response
Additional Information/Context
How I can stop client use timeoutReadCloser ?or disable AddResponseReadTimeoutMiddleware ( all of them are in aws/transport/http)
AWS Go SDK V2 Module Versions Used
github.com/aws/aws-sdk-go-v2 v1.16.11
github.com/aws/aws-sdk-go-v2/config v1.15.15
github.com/aws/aws-sdk-go-v2/credentials v1.12.10
github.com/aws/aws-sdk-go-v2/service/dynamodb v1.15.12
github.com/aws/aws-sdk-go-v2/service/kinesis v1.15.13
github.com/aws/aws-sdk-go-v2/service/opensearch v1.10.4
github.com/aws/smithy-go v1.12.1
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.4 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.9 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.18 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.12 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.16 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.7.11 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.9 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.11.13 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.16.10 // indirect
Compiler and Version used
go version go1.19.1 linux/amd64
Operating System and version
Ubuntu 22.04.1 LTS