hertz icon indicating copy to clipboard operation
hertz copied to clipboard

[Bug] accept tcp host:port use of closed network connection on server shutdown

Open roma-glushko opened this issue 1 year ago • 13 comments

Describe the bug

I'm using Hertz with the standard transport like this:

  • https://github.com/EinStack/glide/blob/develop/pkg/api/http/config.go#L43-L69

On shutdown though, Hertz constantly returns me this error:

2024-02-11T12:41:25.093+0200    ERROR   HERTZ: Error=accept tcp 127.0.0.1:9099: use of closed network connection
2024-02-11T12:41:25.093+0200    ERROR   api/servers.go:46       error on running HTTP server    {"error": "accept tcp 127.0.0.1:9099: use of closed network connection"}
glide/pkg/api.(*ServerManager).Start.func1
        /Users/roman/Projects/ideas/Glide/pkg/api/servers.go:46
^C^C2024-02-11T12:41:30.093+0200        INFO    HERTZ: Execute OnShutdownHooks finish
make: *** [run] Error 1

Does anyone know what I'm doing wrong here? 🤔

Expected behavior

I expect Hertz server to shutdown without errors.

Screenshots

If applicable, add screenshots to help explain your problem.

Hertz version:

v0.7.3

Environment:

GO111MODULE='on'
GOARCH='amd64'
GOBIN=''
GOCACHE='/Users/usr/Library/Caches/go-build'
GOENV='/Users/usr/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/usr/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/usr/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/Cellar/go/1.21.5/libexec'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/Cellar/go/1.21.5/libexec/pkg/tool/darwin_amd64'
GOVCS=''
GOVERSION='go1.21.5'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/Users/usr/Project/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/s1/ts3nxvl965lfts126qzdcw300000gn/T/go-build662925335=/tmp/go-build -gno-record-gcc-switches -fno-common'

roma-glushko avatar Feb 11 '24 10:02 roma-glushko

I have a feeling it's similar to this issue.

li-jin-gou avatar Feb 11 '24 11:02 li-jin-gou

@li-jin-gou hey, you seem to reference this issue probably by mistake 😌 Could please review that link?

roma-glushko avatar Feb 11 '24 11:02 roma-glushko

https://github.com/cloudwego/hertz/issues/878

li-jin-gou avatar Feb 11 '24 11:02 li-jin-gou

The fastest solution is to use netpoll transport.

li-jin-gou avatar Feb 11 '24 11:02 li-jin-gou

@li-jin-gou thank you for the reference! I will take a look at the chat history.

I was using netpoll, but the thing is that Glide needs to support:

  • windows
  • uploading of voice stream which looks like a file uploading (still not sure what the final implementation will look like)

These two cases don't look like the ideal for netpoll usage right?

roma-glushko avatar Feb 11 '24 11:02 roma-glushko

@li-jin-gou thank you for the reference! I will take a look at the chat history.

I was using netpoll, but the thing is that Glide needs to support:

  • windows
  • uploading of voice stream which looks like a file uploading (still not sure what the final implementation will look like)

These two cases don't look like the ideal for netpoll usage right?

Yes

li-jin-gou avatar Feb 11 '24 12:02 li-jin-gou

@li-jin-gou if we batched the voice streams and send them as websocket messages, would this be still not the ideal case for netpoll?

roma-glushko avatar Feb 11 '24 12:02 roma-glushko

If the QPS is not high, netpoll is fine. refer to https://www.cloudwego.io/docs/hertz/tutorials/basic-feature/network-lib/#choosing-appropriate-network-library

li-jin-gou avatar Feb 11 '24 12:02 li-jin-gou

@li-jin-gou I actually have seen that one. But the doc is more focused on the request size rather than throughput. Is the assumption here that with somewhat low throughput it's unlikely that the request size would cause any memory issues?

roma-glushko avatar Feb 11 '24 12:02 roma-glushko

I actually have seen that one. But the doc is more focused on the request size rather than throughput. Is the assumption here that with somewhat low throughput it's unlikely that the request size would cause any memory issues?

Yes

li-jin-gou avatar Feb 12 '24 08:02 li-jin-gou

Refer to https://github.com/cloudwego/hertz/issues/878 and duplicate issue, I'll close it for now.

li-jin-gou avatar Feb 18 '24 08:02 li-jin-gou

This error is returned if an accepting listener is closed. It is a normal returning process, and it is safe to ignore. Try this code below to check:

func main() {
	mux := http.NewServeMux()
	mux.HandleFunc("/ping", func(writer http.ResponseWriter, request *http.Request) {
		writer.WriteHeader(200)
		writer.Write([]byte("pong!"))
	})
	server := &http.Server{Addr: "127.0.0.1:8888", Handler: mux}

	go func() {
		time.Sleep(5 * time.Second)
		server.Close()
	}()

	err := server.ListenAndServe()
	if err != nil {
		fmt.Println(err.Error())
	}
}

I think it is an optimized point that we ignore this Server closed error when closing the server. I will put it into the Good first issues pool. Please feel free to open a pr to do so if you want to.

welkeyever avatar Mar 04 '24 05:03 welkeyever

@welkeyever I would like to try to optimize this problem, plz assign to me

YarBor avatar Apr 06 '24 11:04 YarBor