grpc
Updating credentials of a running server
Please see the FAQ in our main README.md before submitting your issue.
Use case(s) - what problem will this feature solve?
When using server certificates, while the server is running new certificates are created. The new certificate must be updated in the server's options. Currently this is not possible without restarting the server and disconnecting the clients.
Proposed Solution
If the options can be updated while the server is running this problem can be resolved.
Alternatives Considered
I didn't find any alternatives. Do you have any suggestions?
Additional Context
Hi @TalLerner, a possible solution is to implement your own TransportCredentials that delegates to TLS credentials created using one of the available constructors. Your custom TransportCredentials can provide the option to switch the delegate during runtime. An example of such a TransportCredentials implementation is as follows:
type DynamicCreds struct {
delegate credentials.TransportCredentials
rwMutex sync.RWMutex
}
func (d *DynamicCreds) ClientHandshake(ctx context.Context, host string, conn net.Conn) (net.Conn, credentials.AuthInfo, error) {
d.rwMutex.RLock()
defer d.rwMutex.RUnlock()
return d.delegate.ClientHandshake(ctx, host, conn)
}
func (d *DynamicCreds) ServerHandshake(conn net.Conn) (net.Conn, credentials.AuthInfo, error) {
d.rwMutex.RLock()
defer d.rwMutex.RUnlock()
return d.delegate.ServerHandshake(conn)
}
func (d *DynamicCreds) Info() credentials.ProtocolInfo {
d.rwMutex.RLock()
defer d.rwMutex.RUnlock()
return d.delegate.Info()
}
func (d *DynamicCreds) Clone() credentials.TransportCredentials {
d.rwMutex.RLock()
defer d.rwMutex.RUnlock()
return NewDynamicCreds(d.delegate.Clone())
}
func (d *DynamicCreds) OverrideServerName(name string) error {
d.rwMutex.RLock()
defer d.rwMutex.RUnlock()
return d.delegate.OverrideServerName(name)
}
func (d *DynamicCreds) UpdateDelegate(newCreds credentials.TransportCredentials) {
d.rwMutex.Lock()
defer d.rwMutex.Unlock()
if newCreds == d {
fmt.Printf("Can't point to self!")
return
}
d.delegate = newCreds
}
func NewDynamicCreds(delegate credentials.TransportCredentials) *DynamicCreds {
return &DynamicCreds{
delegate: delegate,
rwMutex: sync.RWMutex{},
}
}
You can then create DynamicCreds and use them while starting your server as follows:
serverCertFile := data.Path("x509/server_cert.pem")
serverKeyFile := data.Path("x509/server_key.pem")
serverCreds, err := credentials.NewServerTLSFromFile(serverCertFile, serverKeyFile)
if err != nil {
log.Fatalf("Failed to generate credentials: %v", err)
}
dynCreds := NewDynamicCreds(serverCreds)
opts = []grpc.ServerOption{grpc.Creds(dynCreds)}
grpcServer := grpc.NewServer(opts...)
When you want to change the delegate, you can call dynCreds.UpdateDelegate() while passing in the new credentials. This way you gain the ability to change only the transport credentials without updating the server options.
I tried this out in https://github.com/arjan-bal/routeguide/commit/b7b06088a321b7d46ee0defde2895e0cef6b3a1f which has a server that switches it's TLS certs every 5 seconds.
Let me know if this works for you.
Another option suggested by @atollena is to create a tls.Config with empty Certificates, write a closure that gets the latest certificates and assign it to the GetCertificate field of the tls.Config. The tls library will call your closure during every handshake to fetch the certificates.
Use this tls.Config to create gRPC transport credentials by calling the constructor.
This issue is labeled as requiring an update from the reporter, and no update has been received after 6 days. If no update is provided in the next 7 days, this issue will be automatically closed.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Thank you, Arjan, for your reply! I am in the process of applying and testing your proposal. I will update you on the results soon.
Kind Regards, Tal Lerner
From: Arjan Singh Bal @.> Date: Wednesday, 8 May 2024 at 20:41 To: grpc/grpc-go @.> Cc: Tal Lerner @.>, Mention @.> Subject: Re: [grpc/grpc-go] Updating credentials of a running server (Issue #7209)
Hi @TalLernerhttps://github.com/TalLerner, a possible solution is to implement your own TransportCredentialshttps://pkg.go.dev/google.golang.org/grpc/credentials#TransportCredentials that delegates to TLS credentials created using one of the available constructorshttps://pkg.go.dev/google.golang.org/grpc/credentials#TransportCredentials. Your custom TransportCredentials can provide the option to switch the delegate during runtime. An example of such a TransportCredentials implementation is as follows:
type DynamicCreds struct {
delegate credentials.TransportCredentials
rwMutex sync.RWMutex
}
func (d *DynamicCreds) ClientHandshake(ctx context.Context, host string, conn net.Conn) (net.Conn, credentials.AuthInfo, error) {
d.rwMutex.RLock()
defer d.rwMutex.RUnlock()
return d.delegate.ClientHandshake(ctx, host, conn)
}
func (d *DynamicCreds) ServerHandshake(conn net.Conn) (net.Conn, credentials.AuthInfo, error) {
d.rwMutex.RLock()
defer d.rwMutex.RUnlock()
return d.delegate.ServerHandshake(conn)
}
func (d *DynamicCreds) Info() credentials.ProtocolInfo {
d.rwMutex.RLock()
defer d.rwMutex.RUnlock()
return d.delegate.Info()
}
func (d *DynamicCreds) Clone() credentials.TransportCredentials {
d.rwMutex.RLock()
defer d.rwMutex.RUnlock()
return NewDynamicCreds(d.delegate.Clone())
}
func (d *DynamicCreds) OverrideServerName(name string) error {
d.rwMutex.RLock()
defer d.rwMutex.RUnlock()
return d.delegate.OverrideServerName(name)
}
func (d *DynamicCreds) UpdateDelegate(newCreds credentials.TransportCredentials) {
d.rwMutex.Lock()
defer d.rwMutex.Unlock()
if newCreds == d {
fmt.Printf("Can't point to self!")
return
}
d.delegate = newCreds
}
func NewDynamicCreds(delegate credentials.TransportCredentials) *DynamicCreds {
return &DynamicCreds{
delegate: delegate,
rwMutex: sync.RWMutex{},
}
}
You can then create DynamicCreds and use them while starting your server as follows:
serverCertFile := data.Path("x509/server_cert.pem")
serverKeyFile := data.Path("x509/server_key.pem")
serverCreds, err := credentials.NewServerTLSFromFile(serverCertFile, serverKeyFile)
if err != nil {
log.Fatalf("Failed to generate credentials: %v", err)
}
dynCreds := NewDynamicCreds(serverCreds)
opts = []grpc.ServerOption{grpc.Creds(dynCreds)}
grpcServer := grpc.NewServer(opts...)
When you want to change the delegate, you can call dynCreds.UpdateDelegate() while passing in the new credentials. This way you gain the ability to change only the transport credentials without updating the server options.
I tried this out in @.***https://github.com/arjan-bal/routeguide/commit/b7b06088a321b7d46ee0defde2895e0cef6b3a1f which has a server that switches it's TLS certs every 5 seconds.
Let me know if this works for you.
— Reply to this email directly, view it on GitHubhttps://github.com/grpc/grpc-go/issues/7209#issuecomment-2101085166, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUS3O5TQMHPKNQKQCDXU363ZBJPS5AVCNFSM6AAAAABHLRBKOCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBRGA4DKMJWGY. You are receiving this because you were mentioned.Message ID: @.***>
This issue is labeled as requiring an update from the reporter, and no update has been received after 6 days. If no update is provided in the next 7 days, this issue will be automatically closed.
This issue is labeled as requiring an update from the reporter, and no update has been received after 6 days. If no update is provided in the next 7 days, this issue will be automatically closed.
I just wanted to update you that the solution you suggested was implemented and tested and seems to work great.
Thank you!
From: github-actions[bot] @.> Date: Monday, 3 June 2024 at 13:46 To: grpc/grpc-go @.> Cc: Tal Lerner @.>, Assign @.> Subject: Re: [grpc/grpc-go] Updating credentials of a running server (Issue #7209)
This issue is labeled as requiring an update from the reporter, and no update has been received after 6 days. If no update is provided in the next 7 days, this issue will be automatically closed.
— Reply to this email directly, view it on GitHubhttps://github.com/grpc/grpc-go/issues/7209#issuecomment-2144877084, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUS3O5Q3JZXFPQW2YIACVTLZFRCO7AVCNFSM6AAAAABHLRBKOCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBUHA3TOMBYGQ. You are receiving this because you were assigned.Message ID: @.***>