lorawan-server icon indicating copy to clipboard operation
lorawan-server copied to clipboard

Lot of too_early, downlinks_lost, ack_lost

Open ahiregange opened this issue 4 years ago • 4 comments

Hello,

I have successfully been using the server v0.6.7 with multiple units now for a while, thanks to your help. Also have it successfully passing on uplinks via a Connector to our cloud.

Now suddenly for last few days though we made no changes to our devices nor to server, we are getting a huge number of red/yellow boxes on the dashboard (too_early, downlinks_lost, ack_lost). Earlier it used to be mostly blue with the rate yellow box due to retry. Capture1

I have gone through previous issues posted with similar observations, read the Troubleshooting guide and changed Gatewate settings accordingly. Also went through the gateway<->server protocol documentation you have provided a link to.

  1. But I have not been able to find out what can be done to fix each of the above situations and what each situation should be handled. For example, is it simply a question of improving the data link between gateway and server?

  2. I am using a Laird Rg1xx gateway. Attaching two screenshots: one of gateway settings including Push Timeout increased to 500milliseconds as per your suggestion for a similar issue. Will it help to increase / decrease Keep Alive time or Stat Interval? Laird-Rg1xx

Thanks! Arun

ahiregange avatar May 21 '20 12:05 ahiregange

The xx_lost are due to "too_early", which is a weird, but not an uncommon issue. It comes from the gateway-- do you have some gateway log, or some other information that could help debugging why too early?

gotthardp avatar May 21 '20 14:05 gotthardp

Hi Petr,

Unfortunately I did not collect gateway logs at that time. :(

And right now after resetting the gateway & internet modem the situation is back to being very clean, mostly blue, just the odd yellow box for retry.

I will monitor and send you a log as soon as I get the same situation again.

Thanks, Arun

ahiregange avatar May 22 '20 01:05 ahiregange

@ahiregange It is a certain indication for some gateways that the network connection between the gateway and the server became extremely bad. Too Early actually means Too Late - the server sends the reply to an uplink, but by the time it reaches the gateway the time is already in the past, hence downlinks lost to follow. Normally this is seen over cellular networks, the first signal is the number of Lost_ack messages.

  1. Check the network connection between the server and the gateway both ways.
  2. Ensure that the gateway and the server are time-synchronized!!!: The gateway and the server must run NTP or have another source of time synchronization, like GPS. It is a requirement of the LoRaWAN.

altishchenko avatar May 23 '20 18:05 altishchenko

Hi Alexander,

Thanks for your input, especially the point about time-synchronization via NTP or GPS. I will look into this.

Thanks, Arun

ahiregange avatar May 24 '20 07:05 ahiregange