ydb-go-sdk icon indicating copy to clipboard operation
ydb-go-sdk copied to clipboard

bug: leak goroutines while network problems

Open orloffv opened this issue 9 months ago • 1 comments

Screenshot 2024-05-07 at 13 35 44

YDB GO SDK version: v3.65.3

A lot of goroutines with stacktrace:

goroutine profile: total 26837
8922 @ 0x440bce 0x452965 0x94c52c 0x4785a1
# labels: {"background":"reconnector-loop"}
#	0x94c52b	google.golang.org/grpc.newClientStreamWithParams.func4+0x8b	external/gazelle~~go_deps~org_golang_google_grpc/stream.go:392

8921 @ 0x440bce 0x40a71f 0x40a352 0xe4a9b7 0x4785a1
# labels: {"background":"reconnector-loop"}
#	0xe4a9b6	github.com/ydb-platform/ydb-go-sdk/v3/internal/background.(*Worker).starterLoop+0x96	external/gazelle~~go_deps~com_github_ydb_platform_ydb_go_sdk_v3/internal/background/worker.go:132

8920 @ 0x440bce 0x452965 0xeab6e5 0xeab3dd 0xdb5c9d 0xe4ad2d 0x4785a1
# labels: {"background":"commit pusher", "base-context":"topic-stream-reader"}
#	0xeab6e4	github.com/ydb-platform/ydb-go-sdk/v3/internal/topic/topicreaderinternal.(*committer).waitSendTrigger+0xa4	external/gazelle~~go_deps~com_github_ydb_platform_ydb_go_sdk_v3/internal/topic/topicreaderinternal/committer.go:159
#	0xeab3dc	github.com/ydb-platform/ydb-go-sdk/v3/internal/topic/topicreaderinternal.(*committer).pushCommitsLoop+0x3c	external/gazelle~~go_deps~com_github_ydb_platform_ydb_go_sdk_v3/internal/topic/topicreaderinternal/committer.go:124
#	0xdb5c9c	runtime/pprof.Do+0x9c												GOROOT/src/runtime/pprof/runtime.go:51
#	0xe4ad2c	github.com/ydb-platform/ydb-go-sdk/v3/internal/background.(*Worker).starterLoop.func1+0x1ec			external/gazelle~~go_deps~com_github_ydb_platform_ydb_go_sdk_v3/internal/background/worker.go:138

goroutine_stacked.txt allocs.txt goroutine.txt

mb it's related to https://github.com/ydb-platform/ydb-go-sdk/issues/1044

orloffv avatar May 07 '24 11:05 orloffv

According to call stack worker.starterLoop() should be called when we call Topic().StartReader(), but we do this only once in main()

goroutine 121 [chan receive, 1083 minutes]:
github.com/ydb-platform/ydb-go-sdk/v3/internal/background.(*Worker).starterLoop(0xc0008697c0)
        external/gazelle~~go_deps~com_github_ydb_platform_ydb_go_sdk_v3/internal/background/worker.go:132 +0x97
created by github.com/ydb-platform/ydb-go-sdk/v3/internal/background.(*Worker).init.func1 in goroutine 1
        external/gazelle~~go_deps~com_github_ydb_platform_ydb_go_sdk_v3/internal/background/worker.go:125 +0x19b

goroutine 247497 [chan receive, 154 minutes]:
github.com/ydb-platform/ydb-go-sdk/v3/internal/background.(*Worker).starterLoop(0xc00391aff8)
        external/gazelle~~go_deps~com_github_ydb_platform_ydb_go_sdk_v3/internal/background/worker.go:132 +0x97
created by github.com/ydb-platform/ydb-go-sdk/v3/internal/background.(*Worker).init.func1 in goroutine 247495
        external/gazelle~~go_deps~com_github_ydb_platform_ydb_go_sdk_v3/internal/background/worker.go:125 +0x19b


goroutine 247492 [chan receive, 154 minutes]:
github.com/ydb-platform/ydb-go-sdk/v3/internal/background.(*Worker).starterLoop(0xc00391ad58)
        external/gazelle~~go_deps~com_github_ydb_platform_ydb_go_sdk_v3/internal/background/worker.go:132 +0x97
created by github.com/ydb-platform/ydb-go-sdk/v3/internal/background.(*Worker).init.func1 in goroutine 247490
        external/gazelle~~go_deps~com_github_ydb_platform_ydb_go_sdk_v3/internal/background/worker.go:125 +0x19b

holycheater avatar May 07 '24 11:05 holycheater