TCP port scans failing since upgrading to 1.19.0

[derekoharrow]

⚠️ Please verify that this bug has NOT been raised before.

• [X] I checked and didn't find similar issue

🛡️ Security Policy

• [X] I agree to have read this project Security Policy

Description

Just upgraded to 1.19.0 (from beta) and now the monitors I have that use TCP port scans are failing regularly.

Graphs below show 2 separate monitors, both of which use port scans. The failures start at the moment I updated the docker container.

👟 Reproduction steps

Just running UK v1.19.0

👀 Expected behavior

TCP Port monitor work as expected

😓 Actual Behavior

Says Request Timeout

🐻 Uptime-Kuma Version

1.19.0

💻 Operating System and Arch

Ubuntu 20.04

🌐 Browser

Chrome

🐋 Docker Version

Docker version 20.10.18, build b40c2f6

🟩 NodeJS Version

No response

📝 Relevant log output

~$ docker logs -f uptime-kuma
==> Performing startup jobs and maintenance tasks
==> Starting application with user 1000 group 117
Welcome to Uptime Kuma
Your Node.js version: 16
2022-12-23T15:54:21Z [SERVER] INFO: Welcome to Uptime Kuma
2022-12-23T15:54:21Z [SERVER] INFO: Node Env: production
2022-12-23T15:54:21Z [SERVER] INFO: Importing Node libraries
2022-12-23T15:54:21Z [SERVER] INFO: Importing 3rd-party libraries
2022-12-23T15:54:21Z [SERVER] INFO: Creating express and socket.io instance
2022-12-23T15:54:21Z [SERVER] INFO: Server Type: HTTP
2022-12-23T15:54:21Z [SERVER] INFO: Importing this project modules
2022-12-23T15:54:22Z [NOTIFICATION] INFO: Prepare Notification Providers
2022-12-23T15:54:22Z [SERVER] INFO: Version: 1.19.0
2022-12-23T15:54:22Z [DB] INFO: Data Dir: ./data/
2022-12-23T15:54:22Z [SERVER] INFO: Connecting to the Database
2022-12-23T15:54:22Z [DB] INFO: SQLite config:
[ { journal_mode: 'wal' } ]
[ { cache_size: -12000 } ]
2022-12-23T15:54:22Z [DB] INFO: SQLite Version: 3.39.4
2022-12-23T15:54:22Z [SERVER] INFO: Connected
2022-12-23T15:54:22Z [DB] INFO: Your database version: 10
2022-12-23T15:54:22Z [DB] INFO: Latest database version: 10
2022-12-23T15:54:22Z [DB] INFO: Database patch not needed
2022-12-23T15:54:22Z [DB] INFO: Database Patch 2.0 Process
2022-12-23T15:54:22Z [DB] INFO: patch-grpc-monitor.sql is not patched
2022-12-23T15:54:22Z [DB] INFO: Backing up the database
2022-12-23T15:56:03Z [DB] INFO: patch-grpc-monitor.sql is patching
2022-12-23T15:56:10Z [DB] INFO: patch-grpc-monitor.sql was patched successfully
2022-12-23T15:56:10Z [DB] INFO: patch-maintenance-table2.sql is not patched
2022-12-23T15:56:10Z [DB] INFO: patch-maintenance-table2.sql is patching
2022-12-23T15:56:11Z [DB] INFO: patch-maintenance-table2.sql was patched successfully
2022-12-23T15:56:11Z [DB] INFO: Database Patched Successfully
2022-12-23T15:56:11Z [SERVER] INFO: Load JWT secret from database.
2022-12-23T15:56:11Z [SERVER] INFO: Adding route
2022-12-23T15:56:11Z [SERVER] INFO: Adding socket handler
2022-12-23T15:56:11Z [SERVER] INFO: Init the server
2022-12-23T15:56:11Z [SERVER] INFO: Listening on 3001
2022-12-23T15:56:20Z [MONITOR] WARN: Monitor #8 'MQTT': Pending: Request timeout | Max retries: 4 | Retry: 1 | Retry Interval: 30 seconds | Type: port
2022-12-23T15:56:20Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 1 | Retry Interval: 30 seconds | Type: port
2022-12-23T15:56:37Z [MONITOR] WARN: Monitor #40 'Eufy Security WS': Pending: Request timeout | Max retries: 3 | Retry: 1 | Retry Interval: 300 seconds | Type: port
2022-12-23T15:56:40Z [MONITOR] WARN: Monitor #49 'Transmission (RPC)': Pending: Request timeout | Max retries: 3 | Retry: 1 | Retry Interval: 300 seconds | Type: port
2022-12-23T15:56:49Z [MONITOR] WARN: Monitor #26 'Node Red (GUI)': Pending: timeout of 24000ms exceeded | Max retries: 3 | Retry: 1 | Retry Interval: 30 seconds | Type: http
2022-12-23T15:56:50Z [MONITOR] WARN: Monitor #27 'Home Assistant (GUI)': Pending: timeout of 24000ms exceeded | Max retries: 3 | Retry: 1 | Retry Interval: 30 seconds | Type: http
2022-12-23T15:56:51Z [MONITOR] WARN: Monitor #31 'Google.co.uk (HTTPS)': Pending: timeout of 24000ms exceeded | Max retries: 3 | Retry: 1 | Retry Interval: 30 seconds | Type: http
2022-12-23T15:56:55Z [MONITOR] WARN: Monitor #8 'MQTT': Pending: Request timeout | Max retries: 4 | Retry: 2 | Retry Interval: 30 seconds | Type: port
2022-12-23T15:56:55Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 2 | Retry Interval: 30 seconds | Type: port
2022-12-23T15:57:03Z [MONITOR] WARN: Monitor #58 'Zigbee2MQTT (MQTT)': Pending: Timeout | Max retries: 3 | Retry: 1 | Retry Interval: 30 seconds | Type: mqtt
2022-12-23T15:57:19Z [MONITOR] WARN: Monitor #84 'NGINX ping.oharrow.uk': Pending: timeout of 24000ms exceeded | Max retries: 3 | Retry: 1 | Retry Interval: 30 seconds | Type: http
2022-12-23T15:57:30Z [MONITOR] WARN: Monitor #8 'MQTT': Pending: Request timeout | Max retries: 4 | Retry: 3 | Retry Interval: 30 seconds | Type: port
2022-12-23T15:57:30Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 3 | Retry Interval: 30 seconds | Type: port
2022-12-23T15:58:07Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 4 | Retry Interval: 30 seconds | Type: port
2022-12-23T15:58:12Z [AUTH] INFO: Disabled Auth: auto login to admin
2022-12-23T15:58:22Z [MONITOR] INFO: Get Monitor: 11 User ID: 1
2022-12-23T15:58:41Z [MONITOR] WARN: Monitor #8 'MQTT': Pending: Request timeout | Max retries: 4 | Retry: 1 | Retry Interval: 30 seconds | Type: port
2022-12-23T15:58:42Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 5 | Retry Interval: 30 seconds | Type: port
2022-12-23T15:59:16Z [MONITOR] WARN: Monitor #8 'MQTT': Pending: Request timeout | Max retries: 4 | Retry: 2 | Retry Interval: 30 seconds | Type: port
2022-12-23T15:59:23Z [MONITOR] WARN: Monitor #9 'MariaDB': Failing: Request timeout | Interval: 30 seconds | Type: port | Down Count: 0 | Resend Interval: 5
2022-12-23T16:00:32Z [AUTH] INFO: Disabled Auth: auto login to admin
2022-12-23T16:00:41Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 1 | Retry Interval: 30 seconds | Type: port
2022-12-23T16:01:16Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 2 | Retry Interval: 30 seconds | Type: port
2022-12-23T16:01:34Z [MONITOR] WARN: Monitor #8 'MQTT': Pending: Request timeout | Max retries: 4 | Retry: 1 | Retry Interval: 30 seconds | Type: port
2022-12-23T16:01:42Z [MONITOR] WARN: Monitor #40 'Eufy Security WS': Pending: Request timeout | Max retries: 3 | Retry: 2 | Retry Interval: 300 seconds | Type: port
2022-12-23T16:01:45Z [MONITOR] WARN: Monitor #49 'Transmission (RPC)': Pending: Request timeout | Max retries: 3 | Retry: 2 | Retry Interval: 300 seconds | Type: port
2022-12-23T16:01:50Z [AUTH] INFO: Disabled Auth: auto login to admin
2022-12-23T16:01:51Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 3 | Retry Interval: 30 seconds | Type: port
2022-12-23T16:02:09Z [MONITOR] WARN: Monitor #8 'MQTT': Pending: Request timeout | Max retries: 4 | Retry: 2 | Retry Interval: 30 seconds | Type: port
2022-12-23T16:02:17Z [MONITOR] WARN: Monitor #84 'NGINX ping.oharrow.uk': Pending: timeout of 24000ms exceeded | Max retries: 3 | Retry: 1 | Retry Interval: 30 seconds | Type: http
2022-12-23T16:02:26Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 4 | Retry Interval: 30 seconds | Type: port
2022-12-23T16:02:46Z [MONITOR] WARN: Monitor #8 'MQTT': Pending: Request timeout | Max retries: 4 | Retry: 3 | Retry Interval: 30 seconds | Type: port
2022-12-23T16:03:01Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 5 | Retry Interval: 30 seconds | Type: port
2022-12-23T16:03:11Z [MONITOR] WARN: Monitor #84 'NGINX ping.oharrow.uk': Pending: timeout of 24000ms exceeded | Max retries: 3 | Retry: 2 | Retry Interval: 30 seconds | Type: http
2022-12-23T16:04:10Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 1 | Retry Interval: 30 seconds | Type: port
2022-12-23T16:04:29Z [MONITOR] WARN: Monitor #8 'MQTT': Pending: Request timeout | Max retries: 4 | Retry: 1 | Retry Interval: 30 seconds | Type: port
2022-12-23T16:05:54Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 1 | Retry Interval: 30 seconds | Type: port
2022-12-23T16:06:29Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 2 | Retry Interval: 30 seconds | Type: port
2022-12-23T16:06:47Z [MONITOR] WARN: Monitor #40 'Eufy Security WS': Pending: Request timeout | Max retries: 3 | Retry: 3 | Retry Interval: 300 seconds | Type: port
2022-12-23T16:06:47Z [MONITOR] WARN: Monitor #8 'MQTT': Pending: Request timeout | Max retries: 4 | Retry: 1 | Retry Interval: 30 seconds | Type: port
2022-12-23T16:06:50Z [MONITOR] WARN: Monitor #49 'Transmission (RPC)': Pending: Request timeout | Max retries: 3 | Retry: 3 | Retry Interval: 300 seconds | Type: port
2022-12-23T16:07:04Z [MONITOR] WARN: Monitor #9 'MariaDB': Pending: Request timeout | Max retries: 5 | Retry: 3 | Retry Interval: 30 seconds | Type: port

[louislam]

Have you try to restart the container? Because tcp monitor is no code change. beta.2 and 1.19.0 are nearly identical.

[derekoharrow]

Yep - I restarted and re-enabled the TCP monitors which I had paused because of this and straight-away they start timing out.

Absolutely nothing has changed on my end other than updating uptime-kuma, and it does seem to be affecting all TCP monitors.

[derekoharrow]

I've upgraded to 1.19.2 today - problem still remains.

[louislam]

I cannot reproduce the issue. Could you backup the volume and downgrade to 1.8.5, see if the problem whether is gone.

Also can you reproduce on https://demo.uptime.kuma.pet:27000/

[derekoharrow]

Found an obscure DNS issue at my end that was causing this. Thanks for looking anyway!

[Swizz]

Hi @derekoharrow.

I am sorry to bring this issue up, but we are facing the exact same problem since we upgraded to 1.20.1 Only on the TCP probes and randomly on working services (double checked with old fashioned monitoring).

Can you elaborate on how you fixed it ?

Thank you

[fl0schi]

Hey, funny - I'm facing a similar issue at the moment. Just migrated the container to another host machine. Everything is working fine, also some checks to the host machine itself (like tcp/53 port check, HTTP curls, ..) but the SSH check to the host machine + DNS Query is not working since I migrated. DNS looks fine, as other check are working to the host machine.

2023-03-13T21:26:42+01:00 [MONITOR] WARN: Monitor #15 'SSH': Failing: Request timeout | Interval: 60 seconds | Type: port | Down Count: 0 | Resend Interval: 0
2023-03-13T21:27:47+01:00 [MONITOR] WARN: Monitor #15 'SSH': Failing: Request timeout | Interval: 60 seconds | Type: port | Down Count: 0 | Resend Interval: 0
2023-03-13T21:28:31+01:00 [MONITOR] WARN: Monitor #56 'DDNS A': Failing: queryA ETIMEOUT a.ddns.domain.tld | Interval: 60 seconds | Type: dns | Down Count: 0 | Resend Interval: 0
2023-03-13T21:28:33+01:00 [MONITOR] WARN: Monitor #57 'DDNS B': Failing: queryA ETIMEOUT b.ddns.domain.tld | Interval: 60 seconds | Type: dns | Down Count: 0 | Resend Interval: 0

EDIT: Okay, not DNS ... just crazy firewalling issue. :-)