headscale icon indicating copy to clipboard operation
headscale copied to clipboard

Too many open connections with Postgres server

Open pallabpain opened this issue 1 year ago • 1 comments

Bug description

When postgres is used as the database for headscale, I have observed that it frequently opens hundreds of connections with the postgres server which sometimes leads to high CPU usage on the server. The postgres server is shared between databases for other services and hence such spike in usage affects the overall performance.

While setting up pgbouncer does mitigate the issue, it would be better if headscale limits the number of open connections.

Environment

  • OS: NA
  • Headscale version: v0.22.3
  • Tailscale version: NA
  • [ ] Headscale is behind a (reverse) proxy
  • [x] Headscale runs in a container

To Reproduce

  1. Run headscale as a k8s pod and use postgres as the database
  2. Register a bunch of nodes with headscale
  3. netstat and grep the connections established with your postgres server
  4. Monitor it with the watch command for some time.

pallabpain avatar Oct 30 '23 05:10 pallabpain

Most likely a side effect of that is that tailscale logins never succeed and the interface never gets populated.

SuperSandro2000 avatar Jan 23 '24 23:01 SuperSandro2000

The issue has been addressed in https://github.com/juanfont/headscale/pull/1583

pallabpain avatar Feb 12 '24 12:02 pallabpain