Mysterious CLR/Org.BouncyCastle.Crypto.Tls.TlsFatalAlert on AutoUpdateAsync

[MBAnslow]

I managed to get the entertainment demo working at https://github.com/michielpost/Q42.HueApi/tree/master/src/HueApi.Entertainment.ConsoleSample.

However, this myteriously stopped working on:

await client.AutoUpdateAsync(stream, new System.Threading.CancellationToken(), 50, onlySendDirtyStates: false);

in SetupAndReturnGroup.

This throws the following exception:

Exception has occurred: CLR/Org.BouncyCastle.Crypto.Tls.TlsFatalAlert
Exception thrown: 'Org.BouncyCastle.Crypto.Tls.TlsFatalAlert' in System.Private.CoreLib.dll: 'internal_error(80)'
 Inner exceptions found, see $exception in variables window for more details.
 Innermost exception 	 System.Net.Sockets.SocketException : Connection refused
   at System.Net.Sockets.Socket.Send(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
   at HueApi.Entertainment.Connection.UdpTransport.Send(Byte[] buf, Int32 off, Int32 len)
   at Org.BouncyCastle.Crypto.Tls.DtlsRecordLayer.SendDatagram(DatagramTransport sender, Byte[] buf, Int32 off, Int32 len)
   at Org.BouncyCastle.Crypto.Tls.DtlsRecordLayer.SendRecord(Byte contentType, Byte[] buf, Int32 off, Int32 len)
   at Org.BouncyCastle.Crypto.Tls.DtlsRecordLayer.Send(Byte[] buf, Int32 off, Int32 len)
   at Org.BouncyCastle.Crypto.Tls.DtlsTransport.Send(Byte[] buf, Int32 off, Int32 len)

I don't believe I changed anything in my code. I even regenerated the hue credentials (app_key, app_id, client_key) and setup my entertainment group again.

Have you experienced this problem before?