aws-sdk-go-v2
aws-sdk-go-v2 copied to clipboard
Panic when ungzip with http timeout
Describe the bug
When I develop kinesis-related programs locally, there are a lot of shards, and insufficient local bandwidth will cause timeouts, and panics are very likely to occur at this time. (index out of range
or slice bounds out of range
)
Should prevent decompressing http.Body
when it times out
I have this problem in the kinesis client, but I think the problem is common
Expected Behavior
reutrn error
, not panic
Current Behavior
in [email protected]/aws/transport/http/timeout_read_closer.go
, there is a new goroutine to r.reader.Read(b)
, so I can't recover panic
func (r *timeoutReadCloser) Read(b []byte) (int, error) {
timer := time.NewTimer(r.duration)
c := make(chan readResult, 1)
go func() {
n, err := r.reader.Read(b)
timer.Stop()
c <- readResult{n: n, err: err}
}()
panic example 1/2:
panic: runtime error: slice bounds out of range [21391:0]
goroutine 10107 [running]:
compress/flate.(*decompressor).Read(0xc00073c600, {0xc0085c6000, 0x197e00, 0x406045?})
/usr/local/go/src/compress/flate/inflate.go:338 +0x20c
compress/gzip.(*Reader).Read(0xc000f96840, {0xc0085c6000, 0x197e00, 0x197e00})
/usr/local/go/src/compress/gzip/gunzip.go:252 +0xbb
net/http.(*gzipReader).Read(0xc000a15ba0, {0xc0085c6000, 0x197e00, 0x197e00})
/usr/local/go/src/net/http/transport.go:2838 +0x18a
github.com/aws/aws-sdk-go-v2/aws/transport/http.(*timeoutReadCloser).Read.func1()
/go/pkg/mod/github.com/aws/[email protected]/aws/transport/http/timeout_read_closer.go:48 +0x47
created by github.com/aws/aws-sdk-go-v2/aws/transport/http.(*timeoutReadCloser).Read
/go/pkg/mod/github.com/aws/[email protected]/aws/transport/http/timeout_read_closer.go:47 +0x11e
panic example 2/2:
panic: runtime error: index out of range [4096] with length 4096
goroutine 10827 [running]:
bufio.(*Reader).ReadByte(0xc001659ec0)
/usr/local/go/src/bufio/bufio.go:267 +0xad
compress/flate.(*decompressor).huffSym(0xc000fb8000, 0xc000fb8028)
/usr/local/go/src/compress/flate/inflate.go:719 +0x102
compress/flate.(*decompressor).huffmanBlock(0x0?)
/usr/local/go/src/compress/flate/inflate.go:494 +0x45
compress/flate.(*decompressor).Read(0xc000fb8000, {0xc001350000, 0x2000, 0xc00509bd40?})
/usr/local/go/src/compress/flate/inflate.go:347 +0x7e
compress/gzip.(*Reader).Read(0xc0000fbb80, {0xc001350000, 0x2000, 0x2000})
/usr/local/go/src/compress/gzip/gunzip.go:252 +0xbb
net/http.(*gzipReader).Read(0xc001bd67e0, {0xc001350000, 0x2000, 0x2000})
/usr/local/go/src/net/http/transport.go:2838 +0x18a
github.com/aws/aws-sdk-go-v2/aws/transport/http.(*timeoutReadCloser).Read.func1()
/go/pkg/mod/github.com/aws/[email protected]/aws/transport/http/timeout_read_closer.go:48 +0x47
created by github.com/aws/aws-sdk-go-v2/aws/transport/http.(*timeoutReadCloser).Read
/go/pkg/mod/github.com/aws/[email protected]/aws/transport/http/timeout_read_closer.go:47 +0x11e
Reproduction Steps
Read shards flood with limited bandwidth, let the client read irregular corrupted data
Possible Solution
No response
Additional Information/Context
How I can stop client use timeoutReadCloser
?or disable AddResponseReadTimeoutMiddleware
( all of them are in aws/transport/http
)
AWS Go SDK V2 Module Versions Used
github.com/aws/aws-sdk-go-v2 v1.16.11
github.com/aws/aws-sdk-go-v2/config v1.15.15
github.com/aws/aws-sdk-go-v2/credentials v1.12.10
github.com/aws/aws-sdk-go-v2/service/dynamodb v1.15.12
github.com/aws/aws-sdk-go-v2/service/kinesis v1.15.13
github.com/aws/aws-sdk-go-v2/service/opensearch v1.10.4
github.com/aws/smithy-go v1.12.1
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.4 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.9 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.18 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.12 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.16 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.7.11 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.9 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.11.13 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.16.10 // indirect
Compiler and Version used
go version go1.19.1 linux/amd64
Operating System and version
Ubuntu 22.04.1 LTS