Document rtcd server limitations when deployed on Kubernetes

[coltoneshaw]

trafficstars

When deploying calls on a k8 infrastructure we typically require a public IP for hosts to connect to, or keeping all the traffic internal, so users can directly access private IPs.

We have two paths forward at the moment.

1. They can reduce the IP range of the k8 cluster to a very limited amount and do port mapping.
2. Open public IPs directly to the RTCD nodes for clients to connect to.

Option 1

If you want to deploy calls with three RTCD nodes you could do the below.

1. Limit the total available IP addresses to match the number of nodes. So with three nodes this might be a /29, since Azure takes five IPs, leaving you with three usable.

2. Create NAT rules on your ALB/ELB/DNAT/etc that maps external ports into every available internal IP

   udp external_ip:8448 -> ip_1:8443
   tcp external_ip:8448 -> ip_1:8443
   udp external_ip:8449 -> ip_2:8443
   tcp external_ip:8449 -> ip_2:8443
   udp external_ip:8450 -> ip_3:8443
   tcp external_ip:8450 -> ip_3:8443
   

3. Setup the port override on the rtcd environment variable like below. - https://github.com/mattermost/rtcd/blob/cacb9692afb51974848205eddfeb568473d529d8/config/config.sample.toml#L71C29-L71C70

   RTCD_RTC_ICEHOSTPORTOVERRIDE=ip_1/8448,ip_2/8449,ip_3/8450
   

4. Set the RTCD_RTC_ICEHOSTOVERRIDE as the external IP

   RTCD_RTC_ICEHOSTOVERRIDE=external_ip
   

Option 2

You will have to create three unique deployments of your RTCD service, and set the correct host override per deployment. Note this requires one external IP address per rtcd

You'll set the below environment variable three different times with a different IP per

RTCD_RTC_ICEHOSTOVERRIDE=external_ip

See the original post here.

[streamer45]

@coltoneshaw Option 2 can be simpler than that. Through STUN rtcd can figure out its own public IP address so there's no override to manually set. Also, no need to have separate deployments. It can be the same deployment with one node per rtcd instance.

[cwarnermm]

@streamer45 - Are you open to drafting a docs PR to address this issue?

[streamer45]

Created https://mattermost.atlassian.net/browse/MM-58583