routinator icon indicating copy to clipboard operation
routinator copied to clipboard

RTR connections statistics wrong

Open bluikko opened this issue 3 years ago • 8 comments

On Routinator 0.9.0 Docker image the RTR connections statistics are not displayed right:

  • "Total Connections" shows no number at all.
  • "Current Connections" shows value 0 while looking at the server there is definitely 1 open connection to port 3323 and BIRD shows that the RTR connection is up.
  • If one additional connection to 3323 is opened as a test then "Current Connections" increases to value 1. "Total Connections" stays empty.
  • "Bytes Read"/"Bytes Written" seem to work correctly if data is sent/received to a testing connection - but maybe not increase right for the existing "real" BIRD connection.
  • "HTTP Connections" seem to work correctly.

In #465 there is a screenshot that shows "RTR Connections" working right - in here it looks like: image The screenshot is taken while a testing connection (a second connection in addition to 1 open connection from BIRD) was open. Once that is closed the "Current Connections" drops to 0 even though BIRD has 1 open connection:

tcp        0      0 192.0.2.1:43344       192.0.2.2:3323      ESTABLISHED 22945/bird

There are no errors in the browser JavaScript console. The tini and routinator process uptime is 2 weeks.

bluikko avatar Jul 13 '21 07:07 bluikko

Thanks for the detailed report. With regards to the Total Connections being empty: we removed the metric because it was a bit meaningless but we forgot to remove the item from the UI. I have a pull request open to fix this. I will run some tests for the Current Connections metric.

AlexanderBand avatar Jul 13 '21 07:07 AlexanderBand

I can't seem to reproduce this. I fired up an RTRclient instance on a test machine and Routinator correctly shows one connection. Connecting a second RTRclient from another machine makes Routinator show two.
Screenshot 2021-07-13 at 10 47 44

AlexanderBand avatar Jul 13 '21 10:07 AlexanderBand

I am pretty sure it initially did show the single RTR connection just fine. I was looking at the stats when this was deployed and I think I'd noticed if it was 0. I suspect some time later that connection was "lost"...

I will keep an eye on this especially when restarting next time (might not happen until update to next version, unless doing so helps debugging somehow).

bluikko avatar Jul 13 '21 10:07 bluikko

I have observed the issue in two cases (with 0.11.3). In both cases the RTR connection probably was not terminated cleanly as the routers involved could not longer reach the routinator instance. The counter was correctly decremented. But when the connection was restored, the counter did not increment again.

birkschroeder avatar Sep 29 '22 11:09 birkschroeder

@birkschroeder Oh, this is helpful! Thank you!

partim avatar Oct 03 '22 09:10 partim

I think I fixed the issue in #811 and thus in 0.12.0-rc1, but since I was never able to reproduce it, I am not entirely sure. I will close the issue now, but please do re-open it if the problem persists.

partim avatar Nov 02 '22 14:11 partim

@partim I am facing this issue on the Routinator v0.12.1.

situation:

  • Routinator is having 2 RTR Clients(BIRD v2.08) but displays 0 current connections on bashboard.
  • The clients are connecting using IPv4 private IPs.

image

If you would like me to publish a new issue, please let me know. thank you.

mf-y-nishimura avatar Oct 22 '23 07:10 mf-y-nishimura

No need for a new issue, I’ll just re-open this one.

I’m starting to think Bird is the common denominator of all the reports. We will create a test setup and see if we can replicate the issue.

partim avatar Oct 23 '23 09:10 partim