smokeping_prober
smokeping_prober copied to clipboard
Adding support for tcp on specific port?
Would it be possible to add support for sending TCP Syn packages on a specific port (per host) ? I need to check network link to certain hosts - where I only have a TCP port open - and if I even have UDP - the network guys often dismiss any package loss with "its just udp - its what we drop first" - but when I tell them I get package loss on the TCP port the service runs on - they take it seriously and fix it :)
I do this with a async perl module currently - works perfectly (but does not do prometheus output :)
I would send a tcp syn and just check for a tcp ack coming back or not - and even if I send 100 of them, since I never respond to the initial tcp-ack - the other end never goes to the application as the kernel handles the 3-way handshake and we don't finish it.
If I were to extend this module - would you accept a PR on this?
I had considered extending tcp-shaker ( https://github.com/tevino/tcp-shaker/issues/20 ) - but this exporter is exactly what I need - I just need it to support tcp - so I can get packageloss for tcp endpoints as well :) It would need to support sending and checking tcp packages asynchronously, which I assume it does not currently.
Sending a lot of half-open connections seems like a bad idea, as it will pile up on the target kernel's connection tracking.
This prober is only really concerned with ICMP and UDP ping probes. If you want to do TCP testing, you might be better off with the blackbox_exporter. The blackbox_exporter already has good support for TCP probes.
If your network team likes to drop UDP preferentially, you already have worse problems. Your network team doesn't understand networking. Your DNS requests are going to suffer badly. :smile:
Its actually a very commonly used method - by f.ex. haproxy etc.. see https://github.com/tevino/tcp-shaker that does "something along these lines". you send a syn, get a syn-ack and send a RST (to close it)..
and since everyone is (and should be) using syncookies - it has neglible cost on the receiving end - even if a RST was not send (as otherwise other end would be easy target for simple DOS).
also many places i ONLY have a tcp opening for a specific port - so I can't test on anything except the same port the service is on.
And I didn't say the network teams response was a good one - but I've seen this often in many larger orgs - they only respect findings done with tcp on actual service port (and especially ignore icmp - as routers typicly drops those if it goes above certain load)
Ahh, yes, I didn't realize you were also wanting to RST as well. That's a lot more reasonable.
SYN, SYN-ACK, RST could be a useful probe.
Maybe this could be implemented in the upstream ping library first? That would make it much easier to integrate here.
Another useful item for you, if you have the node_exporter deployed, there are some useful TCP loss metrics. node_netstat_Tcp_RetransSegs
for general TCP packet loss, and node_netstat_TcpExt_TCPSynRetrans
for TCP syn loss.