nats.net icon indicating copy to clipboard operation
nats.net copied to clipboard

NullReference while connecting to remote server via VPN

Open Sergey-Terekhin opened this issue 3 years ago • 5 comments

Hello!

I have problem: with establishing connection with NATS server via VPN:

  • client and server are located on different machines connected via internet and VPN channel
  • I can access to server's web interface via browser and connect to 4222 port via browser (receive invalid portocol operation response, but it's OK)
  • when I try to establish connection from code, I receive NullReferenceException here:
   at NATS.Client.Connection.Control..ctor(String s) in Connection.cs:line 3780
   at NATS.Client.Connection.readOp() in Connection.cs:line 799
   at NATS.Client.Connection.processExpectedInfo(Srv s) in Connection.cs:line 627

Nats client version: 0.11.0 Nats server: 2.1.9

I can see that TCP connection is established, but data can not be read from it...

Update: I seems to be a bug in the client v.0.11.0. I downgraded to the previous version (0.10.1) and was able to connect to the server

Sergey-Terekhin avatar Dec 09 '20 14:12 Sergey-Terekhin

@Sergey-Terekhin , thanks for raising this issue. I did verify that locally the 0.11.0 client can connect to a NATS server v2.1.9. It's looking like the stream reader returned null, which indicates the connection was terminated before the stream reader could read the line. Are you going through a proxy or load balancer as well?

Also, the stack looks different from the source. We don't have a source file named Connection.cs (but do have Conn.cs). Are you working with a modified client?

https://github.com/nats-io/nats.net/blob/0.11.0/src/NATS.Client/Conn.cs#L3780

In 0.11.0 the Control constructor is found in Conn.cs line 312. ReadOp can be found in Conn.cs Line 1222

ColinSullivan1 avatar Dec 09 '20 17:12 ColinSullivan1

There is an bug here where we need to gracefully throw a connection exception when the stream reader returns a null (indicating EOS). When this happens, we might also want to investigate a delay and retry of the read as no IOException has occurred.

ColinSullivan1 avatar Dec 10 '20 00:12 ColinSullivan1

Custom exception and possible retries would be great - but previous package v. 0.10.1 works fine in same scenario

We don't have a source file named Connection.cs (but do have Conn.cs)

I showed decompiled stack - may be Resharper changed class name somehow - I don't know. But I definitely used unmodified client from NuGet :)

Update 1: The main difference I see between 0.10.1 and 0.11.0 is the way to create TcpClient. May be clients created with AddressFamily.InterNetworkV6 option work incorrectly in our network - I need to investigate

Update 2: I've just replaced code to create TcpClient to the old one from 0.10.1 and it started to work again. My Nats configuration is:

"nats-client": {
    "name": "notifications-service-russian-railways",
    "servers": [ "nats://10.23.10.88:4222" ]
  },

Not working code (Conn.cs line 380):

client = new TcpClient(AddressFamily.InterNetworkV6);
client.Client.DualMode = true;

Working code (Conn.cs line 377, 400-407 + UriExtensions.cs):

client = createTcpClient(s.url);
...
private static TcpClient createTcpClient(Uri uri)
{
    var interNetworkAddressFamily = uri.GetInterNetworkAddressFamily();

    return interNetworkAddressFamily != null
         ? new TcpClient(interNetworkAddressFamily.Value)
         : new TcpClient();
}

May be old code with parsing server URL should be returned

Sergey-Terekhin avatar Dec 10 '20 05:12 Sergey-Terekhin

@Sergey-Terekhin Would you mind to try this patch? I haven't tested it on k8s where we have issues with 0.11.0, but on my machine nats.net works just fine again when the ipv6 stack is disabled.

jasper-d avatar Feb 10 '21 02:02 jasper-d

I believe this is resolved in #432.

ColinSullivan1 avatar Feb 25 '21 19:02 ColinSullivan1

Closed as resolved in #432

scottf avatar Nov 16 '22 18:11 scottf