p4runtime-go-client icon indicating copy to clipboard operation
p4runtime-go-client copied to clipboard

Client reconnection to grpc server

Open rickycraft opened this issue 2 years ago • 3 comments

I'd love to help with the reconnection to the switch. How were you planning to do it? https://github.com/antoninbas/p4runtime-go-client/blob/f5609a358ca5d980b28c9e021e230cc54fb8b01b/pkg/client/client.go#L78

I think that the client should just notify that a disconnection has happened and expose a method to reconnect. This is my attempt but is not the most elegant and convinient solution client.go

rickycraft avatar Apr 28 '22 10:04 rickycraft

I was planning on logging a message and keep retrying until success or until the stopCh channel is closed.

The gRPC library for golang takes care of restoring the underlying connection in case of an issue. So one could just keep retrying (i.e., calling StreamChannel until success). This can be done with some sort of an exponential backoff. The WaitForReady call option (https://pkg.go.dev/google.golang.org/grpc#WaitForReady) can also be used, but I would expect the client to log an error from time to time, so the exponential backoff solution is probably best.

I do believe that it is best to handle reconnections automatically here. I don't see a reason not to. Of course we can also add an option to the Run method if we want to support both behaviors.

antoninbas avatar Apr 28 '22 18:04 antoninbas

I was more talking about if the swich gets rebooted for example. In that case the connection has to be manually restarted. In that case the WaitForReady doesn't help. Maybe WaitForStateChange can be used to stop the loop and handle the reconnection.

The first step for me is to remove log.Fatalf and ad a logging message. Right now is quite unpleasent having the whole application crash if a switch stops. https://github.com/antoninbas/p4runtime-go-client/blob/f5609a358ca5d980b28c9e021e230cc54fb8b01b/pkg/client/client.go#L82

rickycraft avatar May 04 '22 12:05 rickycraft

Something like this. In wich i cancel the context when i want to stop the client. I preferred using context but can be adapted to use the stopCh.

go func() {
    if !cc.WaitForStateChange(ctx, connectivity.Ready) {
	    return
    }
    log.Println("Switch disconnected")
    for {
	    time.Sleep(time.Second * 5)
	    if cc.GetState() == connectivity.Ready {
		    log.Println("Reconnected")
		    // stopping running goroutine
		    cancel()
		    // restarting switch
		    go c.Run(ct, cc, arbitrationCh, messageCh)
		    return
	    }
	    cc.Connect()
	    log.Println("Trying to reconnect")
    }
}()

https://github.com/alsadiamir/ulissep4controller/blob/0df951e76898b0903e0f68ee0fa947639e464e82/controller/pkg/client/client.go#L80-98

rickycraft avatar May 05 '22 16:05 rickycraft