headscale icon indicating copy to clipboard operation
headscale copied to clipboard
verified publisher icon juanfont

[Bug] after removing node there are errors in the logs

Open mastier opened this issue 3 weeks ago • 1 comments

Is this a support request?

  • [x] This is not a support request

Is there an existing issue for this?

  • [x] I have searched the existing issues

Current Behavior

Hi,

the current versio I am running is 0.27.1. I removed one of the nodes and in the logs I can see popping this

headscale | 2025-11-30T15:44:18Z ERR failed to apply change error="generating map response for node 26: generating map response for nodeID 26: multiple errors:\n\tnode not found\n\tnode not found" change=NodeNewOrUpdate node.id=25 worker.id=1

what happened

I needed to update the domain name of the server, so reconfigured. On my android device on Tailscale client I modified the server name from https://this.somedomain.com to https://that.somedomain.com When I reconnected it prompted me to register again, so I though ok , let's remove the old registration (I cnnot see in the logs). I removed node id 26. Then I registered with the given registration string Since then I see that message in the logs

Expected Behavior

  1. I think it should not require to register again
  2. after removal there should be no error

Steps To Reproduce

  1. have setup android client with the headscale server already registered change the domain in config (remeber to change the tls cert in reverse proxy like nginx)
  2. restart server
  3. change the domain on the android
  4. if you see the requirement for registering again, remove the nodes id with headscale nodes delete -i <ID>
  5. Finish new registration
  6. Observe the logs

Environment

- OS: Ubuntu 22.04
- Headscale version: 0.27.1
- Tailscale version: 1.90.9

Runtime environment

  • [x] Headscale is behind a (reverse) proxy
  • [x] Headscale runs in a container

Debug information

ginx-1      | 139.28.84.170 - - [30/Nov/2025:15:38:28 +0000] "GET /key?v=130 HTTP/1.1" 200 176 "-" "Go-http-client/1.1"
nginx-1      | 139.28.84.170 - - [30/Nov/2025:15:38:28 +0000] "POST /ts2021 HTTP/1.1" 301 169 "-" "Go-http-client/1.1"
headscale    | 2025-11-30T15:38:30Z INF Node connected node.id=26 node.name=localhost                
headscale    | 2025-11-30T15:38:30Z INF home/runner/work/headscale/headscale/hscontrol/poll.go:383 > node has connected, mapSession: 0xc00099a300, chan: 0xc00043fdc0 node.id=26 node.name=localhost omitPee
rs=false stream=true
headscale    | 2025-11-30T15:38:41Z INF Node disconnected node.id=26 node.name=localhost
headscale    | 2025-11-30T15:38:41Z INF home/runner/work/headscale/headscale/hscontrol/poll.go:383 > node has disconnected, mapSession: 0xc00099a300, chan: 0xc00043fdc0 node.id=26 node.name=localhost omit
Peers=false stream=true
nginx-1      | 139.28.84.170 - - [30/Nov/2025:15:38:51 +0000] "POST /ts2021 HTTP/1.1" 101 12809 "-" "Go-http-client/1.1"
nginx-1      | 139.28.84.170 - - [30/Nov/2025:15:38:51 +0000] "GET /key?v=130 HTTP/1.1" 200 176 "-" "Go-http-client/1.1"
headscale    | 2025-11-30T15:38:51Z INF Node connected node.id=26 node.name=localhost
headscale    | 2025-11-30T15:38:51Z INF home/runner/work/headscale/headscale/hscontrol/poll.go:383 > node has connected, mapSession: 0xc00099a480, chan: 0xc00027bc00 node.id=26 node.name=localhost omitPee
rs=false stream=true
headscale    | 2025-11-30T15:38:51Z ERR home/runner/work/headscale/headscale/hscontrol/poll.go:401 > cannot write update to client error="client disconnected" node.id=26 node.name=localhost omitPeers=fals
e stream=true
headscale    | 2025-11-30T15:38:52Z INF Starting node registration using key: N1OmWA9z.........kzj0Xr 
nginx-1      | 139.28.84.170 - - [30/Nov/2025:15:38:52 +0000] "GET /key?v=130 HTTP/1.1" 200 176 "-" "Go-http-client/1.1"
nginx-1      | 139.28.84.170 - - [30/Nov/2025:15:38:54 +0000] "GET /register/N1OmW........1dkzj0Xr HTTP/1.1" 200 433 "-" "Mozilla/5.0 (Android 15; Mobile; rv:145.0) Gecko/145.0 Firefox/145.0"
headscale    | 2025-11-30T15:39:01Z INF Node disconnected node.id=26 node.name=localhost
headscale    | 2025-11-30T15:39:01Z INF home/runner/work/headscale/headscale/hscontrol/poll.go:383 > node has disconnected, mapSession: 0xc00099a480, chan: 0xc00027bc00 node.id=26 node.name=localhost omit
Peers=false stream=true
nginx-1      | 139.28.84.170 - - [30/Nov/2025:15:39:52 +0000] "POST /ts2021 HTTP/1.1" 101 766 "-" "Go-http-client/1.1"
nginx-1      | 139.28.84.170 - - [30/Nov/2025:15:39:53 +0000] "GET /key?v=130 HTTP/1.1" 200 176 "-" "Go-http-client/1.1"
headscale    | 2025-11-30T15:40:41Z ERR failed to apply change error="generating map response for node 26: generating map response for nodeID 26: multiple errors:\n\tnode not found\n\tnode not found\n\tno
de not found\n\tnode not found\n\tnode not found\n\tnode not found" change=Full node.id=0 worker.id=1
headscale    | 2025-11-30T15:40:44Z ERR failed to apply change error="generating map response for node 26: generating map response for nodeID 26: multiple errors:\n\tnode not found\n\tnode not found" chan
ge=NodeNewOrUpdate node.id=6 worker.id=1
headscale    | 2025-11-30T15:40:44Z ERR failed to apply change error="generating map response for node 26: generating map response for nodeID 26: multiple errors:\n\tnode not found\n\tnode not found" chan
ge=NodeNewOrUpdate node.id=6 worker.id=1

mastier avatar Nov 30 '25 18:11 mastier

Seeing the same behavior on my headscale.

imightbelosthere avatar Dec 07 '25 14:12 imightbelosthere

same, even with 0.27.2-rc1

dariosusman avatar Dec 09 '25 12:12 dariosusman