Akka.Remote / Akka.IO: certificate rotation handling

[Aaronontheweb]

Is your feature request related to a problem? Please describe.

We've had some requests for supporting certificate rotation handling inside Akka.Remote - and, well, to put it more aptly: we need to be able to handle multiple certificates with overlapping expiration dates so they can be safely rotated in production without downtime.

It looks to me like the .NET base classes for certificate validation should be able to support this:

• https://learn.microsoft.com/en-us/dotnet/core/extensions/sslstream-best-practices#specify-a-server-certificate
• https://learn.microsoft.com/en-us/dotnet/api/system.net.security.sslserverauthenticationoptions.servercertificateselectioncallback?view=net-9.0#system-net-security-sslserverauthenticationoptions-servercertificateselectioncallback

Which means that this is mostly a "how do we expose this configuration to end-users and select the correct certificate?" exercise rather than a "is this even theoretically possible" exercise.

Describe the solution you'd like

After working on https://github.com/akkadotnet/akka.net/pull/7637 and doing some low-level TCP bits there, I think there are two ways of going about this:

1. Write a general purpose TlsConfigurationSetup that either DotNetty, Akka.IO.Tcp, or our new Quic-based transport can select certificates from. Basically the idea is that we're going to expose the certificate selection configuration in an unopinionated way and then just consume it from inside the appropriate areas in Akka.NET - rather than have a bespoke setup for each type.
2. This will probably replace the akka.remote.dot-netty.tcp.ssl configuration settings block when it's used.

Describe alternatives you've considered

We could try making this all doable with HOCON - passing in a certificate list or whatever. I'd be open to that if it's easier for end-users. We don't want to make users have to reason about ALL of the certificate selection details themselves if it can be helped.

[Aaronontheweb]

I'm open to backporting this to v1.5 too btw - but I wanted it on the list as a v1.6-facing item primarily.