packet_forwarder icon indicating copy to clipboard operation
packet_forwarder copied to clipboard

UPSTREAM stops after aprox. 4min

Open jerano opened this issue 5 years ago • 7 comments

Cloned repo and built with "build-pi.sh"-script yesterday. After replacing the repo for WiringPi which was offline, everything installed seemingly correctly.

Process starts successfully, but only runs correctly for about 4 minutes. After that, nothing is sent UPstream, but HEARTBEATS are still running back and forth.

In a Wireshark capture I can see UPSTREAM packets and the ACK's returned. The last UPSTREAM packets seems to not be ACK'ed, and after that nothing more is sent UPSTREAM.

What could be wrong?

jerano avatar Dec 05 '19 13:12 jerano

Could you provide more information like what upstream server, what server type, some logging?

kersing avatar Dec 09 '19 20:12 kersing

Upstream server is European TTN. Will attach packet capture/log as soon as I get to it.

jerano avatar Dec 09 '19 20:12 jerano

Attached you can find the logfile from the packet forwarder, and an ascii-export of the last packets. In the log, the last successful transmission occurs at 14:16:18, after that just [stats] of rcvd radio packets and [down] PULL_ACKs rcvd. (I assume...)

ttn-gateway.log capture.txt

jerano avatar Dec 11 '19 20:12 jerano

Still same problem in v3.0.26 What information do you need from me? Debugging?

To the packet forwarder, it's like the eu-server stops sending ACK's.

From this: 20:23:50 INFO: [down] for server router.eu.thethings.network PULL_ACK received in 39 ms 20:23:56 INFO: [down] for server router.eu.thethings.network PULL_ACK received in 39 ms 20:23:57 INFO: [stats] received packet with valid CRC from mote: 06AA3BD4 (fcnt=4628) 20:23:57 INFO: [up] PUSH_ACK for server router.eu.thethings.network received in 41 ms 20:23:58 INFO: [stats] received packet with valid CRC from mote: 06C7EA38 (fcnt=3878) 20:23:58 INFO: [up] PUSH_ACK for server router.eu.thethings.network received in 41 ms 20:24:00 INFO: [stats] received packet with valid CRC from mote: 06CC3551 (fcnt=2645) 20:24:00 INFO: [up] PUSH_ACK for server router.eu.thethings.network received in 41 ms

We go end up with this: 20:21:15 INFO: [stats] received packet with valid CRC from mote: 062938BE (fcnt=64) 20:21:15 INFO: [stats] received packet with valid CRC from mote: 077CFEB7 (fcnt=59895) 20:21:20 INFO: [stats] received packet with valid CRC from mote: 0751AAEC (fcnt=983) 20:21:22 INFO: [stats] received packet with valid CRC from mote: 073BD5B8 (fcnt=635) 20:21:24 INFO: [stats] received packet with valid CRC from mote: 07D7D4B8 (fcnt=4545) 20:21:24 INFO: [stats] received packet with valid CRC from mote: 07382D89 (fcnt=4628) 20:21:25 INFO: [stats] received packet with valid CRC from mote: 0655FD9C (fcnt=13666) 20:21:30 INFO: [stats] received packet with valid CRC from mote: 07047CD2 (fcnt=4546)

(Ignore the time stamps, working log is from when I restarted the service.)

jerano avatar Jan 15 '20 19:01 jerano

Please check the firewall/session tracking/logging of your router to see if that shows any possible cause.

kersing avatar Jan 15 '20 21:01 kersing

but how different can it work, compared to poly_pkt_fw (v2.1.0) with regards to sending/receiving udp-packets?

Right now mp... was running for more than an hour, then it stopped working when I was looking at it. But it just stops...

Will try some debug_pkt_fwd

jerano avatar Jan 15 '20 22:01 jerano

Only debugging option I can get to produce any output is "debug_log" and that only gives me the received packet in JSON and this: src/mp_pkt_fwd.c:2098:thread_valid(): Time ref: invalid, XTAL correct: invalid (1.000000000000000)

jerano avatar Jan 15 '20 22:01 jerano