goroutine leaks in TestPeerTaskManager_StartFilePeerTask

[timmyyuan]

Bug report:

    leaks.go:82: found unexpected goroutines:
        [Goroutine 10 in state select, with google.golang.org/grpc.(*ccBalancerWrapper).watcher on top of the stack:
        goroutine 10 [select]:
        google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc0000ae0a0)
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:71 +0x130
        created by google.golang.org/grpc.newCCBalancerWrapper
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:62 +0x2e8

         Goroutine 50 in state select, with d7y.io/dragonfly/v2/pkg/rpc.(*Connection).startGC on top of the stack:
        goroutine 50 [select]:
        d7y.io/dragonfly/v2/pkg/rpc.(*Connection).startGC(0xc0002f6000)
                /home/yuanting/work/dev/goprojects/dragonfly/pkg/rpc/client_util.go:45 +0x397
        created by d7y.io/dragonfly/v2/pkg/rpc.NewConnection
                /home/yuanting/work/dev/goprojects/dragonfly/pkg/rpc/client.go:164 +0x285

         Goroutine 11 in state chan receive, with google.golang.org/grpc.(*addrConn).resetTransport on top of the stack:
        goroutine 11 [chan receive]:
        google.golang.org/grpc.(*addrConn).resetTransport(0xc0003a8000)
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/clientconn.go:1219 +0x74c
        created by google.golang.org/grpc.(*addrConn).connect
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/clientconn.go:849 +0x105

         Goroutine 29 in state select, with google.golang.org/grpc/internal/transport.(*http2Client).keepalive on top of the stack:
        goroutine 29 [select]:
        google.golang.org/grpc/internal/transport.(*http2Client).keepalive(0xc0000e21e0)
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1525 +0x1a9
        created by google.golang.org/grpc/internal/transport.newHTTP2Client
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/internal/transport/http2_client.go:341 +0x22fc

         Goroutine 31 in state select, with google.golang.org/grpc/internal/transport.(*controlBuffer).get on top of the stack:
        goroutine 31 [select]:
        google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc000100140, 0x1)
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:406 +0x1a9
        google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0000a25a0)
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:533 +0x125
        google.golang.org/grpc/internal/transport.newHTTP2Client.func3()
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/internal/transport/http2_client.go:396 +0x106
        created by google.golang.org/grpc/internal/transport.newHTTP2Client
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/internal/transport/http2_client.go:394 +0x2ccb

         Goroutine 51 in state select, with google.golang.org/grpc/internal/transport.(*controlBuffer).get on top of the stack:
        goroutine 51 [select]:
        google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0000ae1e0, 0x1)
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:406 +0x1a9
        google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0003d6300)
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:533 +0x125
        google.golang.org/grpc/internal/transport.NewServerTransport.func2()
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/internal/transport/http2_server.go:304 +0x1cd
        created by google.golang.org/grpc/internal/transport.NewServerTransport
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/internal/transport/http2_server.go:301 +0x2128

         Goroutine 52 in state select, with google.golang.org/grpc/internal/transport.(*http2Server).keepalive on top of the stack:
        goroutine 52 [select]:
        google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc00047c000)
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/internal/transport/http2_server.go:1061 +0x34f
        created by google.golang.org/grpc/internal/transport.NewServerTransport
                /home/yuanting/work/dev/goprojects/dragonfly/vendor/google.golang.org/grpc/internal/transport/http2_server.go:313 +0x21ab

        ]

Expected behavior:

No goroutine leaks after the test passed

How to reproduce it:

Introduce goleak.VerifyNone(t) (See https://github.com/uber-go/goleak) in the beginning of the function TestPeerTaskManager_StartFilePeerTask and run:

go test -v -count 1 -gcflags "all=-l" -race -short -run TestPeerTaskManager_StartFilePeerTask  d7y.io/dragonfly/v2/client/daemon/peer

Environment:

• Dragonfly version: master branch abfa0a3fb0753cfad3b121e53149ab5ff871a379
• OS: ubuntu 20.04
• Kernel (e.g. uname -a): Linux DESKTOP-L5197OT 4.19.128-microsoft-standard #1 SMP Tue Jun 23 12:58:10 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
• Others:

[jim3ma]

All goroutines is belong to grpc connections. I did not think this is goroutine leak.

[timmyyuan]

@jim3ma Please note the goroutine in the backtrace:

         Goroutine 50 in state select, with d7y.io/dragonfly/v2/pkg/rpc.(*Connection).startGC on top of the stack:
        goroutine 50 [select]:
        d7y.io/dragonfly/v2/pkg/rpc.(*Connection).startGC(0xc0002f6000)
                /home/yuanting/work/dev/goprojects/dragonfly/pkg/rpc/client_util.go:45 +0x397
        created by d7y.io/dragonfly/v2/pkg/rpc.NewConnection
                /home/yuanting/work/dev/goprojects/dragonfly/pkg/rpc/client.go:164 +0x285

I think there may lack of a cancel or timeout mechanism for startGC in the tests

[gaius-qi]

The v2.0.8 version has been fixed.