unreasonable ttl will cause all DCS connection raise [Errno 22] Invalid argument

[waynerv]

What happened?

can not establish new connection to DCS:

2024-06-18 11:35:58,695 ERROR: Request to server https://246.96.0.1:443 failed: MaxRetryError("HTTPSConnectionPool(host='246.96.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/qfusion-admin/pods?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d6564f4e0>: Failed to establish a new connection: [Errno 22] Invalid argument',))",)

patroni will use ((ttl - (loop_wait + retry_timeout)) / 3) to set TCP_KEEPINTVL socket option, and max allowed value in Linux is 32767, invalid value will cause a OSError [Errno 22] Invalid argument.

How can we reproduce it (as minimally and precisely as possible)?

set ttl to 99999

What did you expect to happen?

Setting ttl to an unreasonable value is the user's fault, but since it's difficult to link the ttl of the leader lock with the TCP_KEEPINTVL parameter of the underlying socket in the operating system, rejecting invalid parameters or including helpful hints in logs would be very helpful.

Patroni/PostgreSQL/DCS version

• Patroni version: 3.1.2
• PostgreSQL version: 14
• DCS (and its version): Kubernetes

Patroni configuration file

use default loop_wai, retry_timeout & ttl to bootstrap

patronictl show-config

after setting ttl to 99999 patronictl show-config will fail

2024-06-18 11:34:23,226 - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=0, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f08b51694e0>: Failed to establish a new connection: [Errno 22] Invalid argument',)': /api/v1/namespaces/qfusion-admin/pods?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e
2024-06-18 11:34:23,226 - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=0, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f08b5169a20>: Failed to establish a new connection: [Errno 22] Invalid argument',)': /api/v1/namespaces/qfusion-admin/configmaps?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e
2024-06-18 11:34:23,227 - ERROR - Request to server https://246.96.0.1:443 failed: MaxRetryError("HTTPSConnectionPool(host='246.96.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/qfusion-admin/pods?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f08b51692e8>: Failed to establish a new connection: [Errno 22] Invalid argument',))",)
2024-06-18 11:34:23,228 - ERROR - Request to server https://246.96.0.1:443 failed: MaxRetryError("HTTPSConnectionPool(host='246.96.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/qfusion-admin/configmaps?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f08b51696d8>: Failed to establish a new connection: [Errno 22] Invalid argument',))",)
2024-06-18 11:34:24,229 - ERROR - ObjectCache.run K8sConnectionFailed('No more API server nodes in the cluster',)
2024-06-18 11:34:24,231 - ERROR - ObjectCache.run K8sConnectionFailed('No more API server nodes in the cluster',)
2024-06-18 11:34:24,233 - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=0, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f08b5189390>: Failed to establish a new connection: [Errno 22] Invalid argument',)': /api/v1/namespaces/qfusion-admin/configmaps?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e
2024-06-18 11:34:24,234 - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=0, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f08b51898d0>: Failed to establish a new connection: [Errno 22] Invalid argument',)': /api/v1/namespaces/qfusion-admin/pods?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e

Patroni log files

2024-06-18 11:35:56,686 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=0, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d6411e9e8>: Failed to establish a new connection: [Errno 22] Invalid argument',)': /api/v1/namespaces/qfusion-admin/pods?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e
2024-06-18 11:35:56,686 ERROR: Request to server https://246.96.0.1:443 failed: MaxRetryError("HTTPSConnectionPool(host='246.96.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/qfusion-admin/configmaps?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d641a5828>: Failed to establish a new connection: [Errno 22] Invalid argument',))",)
2024-06-18 11:35:56,686 ERROR: Request to server https://246.96.0.1:443 failed: MaxRetryError("HTTPSConnectionPool(host='246.96.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/qfusion-admin/pods?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d6564f240>: Failed to establish a new connection: [Errno 22] Invalid argument',))",)
2024-06-18 11:35:57,687 ERROR: ObjectCache.run K8sConnectionFailed('No more API server nodes in the cluster',)
2024-06-18 11:35:57,688 ERROR: ObjectCache.run K8sConnectionFailed('No more API server nodes in the cluster',)
2024-06-18 11:35:57,689 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=0, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d64056be0>: Failed to establish a new connection: [Errno 22] Invalid argument',)': /api/v1/namespaces/qfusion-admin/pods?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e
2024-06-18 11:35:57,690 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=0, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d640567b8>: Failed to establish a new connection: [Errno 22] Invalid argument',)': /api/v1/namespaces/qfusion-admin/configmaps?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e
2024-06-18 11:35:57,690 ERROR: Request to server https://246.96.0.1:443 failed: MaxRetryError("HTTPSConnectionPool(host='246.96.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/qfusion-admin/pods?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d640772e8>: Failed to establish a new connection: [Errno 22] Invalid argument',))",)
2024-06-18 11:35:57,690 ERROR: Request to server https://246.96.0.1:443 failed: MaxRetryError("HTTPSConnectionPool(host='246.96.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/qfusion-admin/configmaps?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d64077518>: Failed to establish a new connection: [Errno 22] Invalid argument',))",)
2024-06-18 11:35:58,691 ERROR: ObjectCache.run K8sConnectionFailed('No more API server nodes in the cluster',)
2024-06-18 11:35:58,693 ERROR: ObjectCache.run K8sConnectionFailed('No more API server nodes in the cluster',)
2024-06-18 11:35:58,694 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=0, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d64056be0>: Failed to establish a new connection: [Errno 22] Invalid argument',)': /api/v1/namespaces/qfusion-admin/configmaps?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e
2024-06-18 11:35:58,694 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=0, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d6405ddd8>: Failed to establish a new connection: [Errno 22] Invalid argument',)': /api/v1/namespaces/qfusion-admin/pods?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e
2024-06-18 11:35:58,695 ERROR: Request to server https://246.96.0.1:443 failed: MaxRetryError("HTTPSConnectionPool(host='246.96.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/qfusion-admin/configmaps?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d64056b38>: Failed to establish a new connection: [Errno 22] Invalid argument',))",)
2024-06-18 11:35:58,695 ERROR: Request to server https://246.96.0.1:443 failed: MaxRetryError("HTTPSConnectionPool(host='246.96.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/qfusion-admin/pods?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d6564f4e0>: Failed to establish a new connection: [Errno 22] Invalid argument',))",)
2024-06-18 11:35:59,696 ERROR: ObjectCache.run K8sConnectionFailed('No more API server nodes in the cluster',)
2024-06-18 11:35:59,697 ERROR: ObjectCache.run K8sConnectionFailed('No more API server nodes in the cluster',)
2024-06-18 11:35:59,698 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=0, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d6411e940>: Failed to establish a new connection: [Errno 22] Invalid argument',)': /api/v1/namespaces/qfusion-admin/configmaps?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e
2024-06-18 11:35:59,698 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=0, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d6411ecf8>: Failed to establish a new connection: [Errno 22] Invalid argument',)': /api/v1/namespaces/qfusion-admin/pods?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e
2024-06-18 11:35:59,699 ERROR: Request to server https://246.96.0.1:443 failed: MaxRetryError("HTTPSConnectionPool(host='246.96.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/qfusion-admin/configmaps?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d642f3518>: Failed to establish a new connection: [Errno 22] Invalid argument',))",)
2024-06-18 11:35:59,699 ERROR: Request to server https://246.96.0.1:443 failed: MaxRetryError("HTTPSConnectionPool(host='246.96.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/qfusion-admin/pods?labelSelector=vendor%3Dqfusion%2Cqfusion-pgha-scope%3Dpostgres-2744242e (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f7d642cee80>: Failed to establish a new connection: [Errno 22] Invalid argument',))",)

PostgreSQL log files

-

Have you tried to use GitHub issue search?

• [X] Yes

Anything else we need to know?

No response