spread icon indicating copy to clipboard operation
spread copied to clipboard

Close ssh connection when reboot is stuck

Open sergiocazzolato opened this issue 7 years ago • 4 comments

This is to avoid this messages showing the output still empty when the reboot was already done:

2018-11-05 14:48:35 Rebooting on nov051744-506863 (google:debian-9-64-base:tasks/google/update-debian-9) as requested... 2018-11-05 14:48:35 Getting uptime 2018-11-05 14:48:35 Uptime calculated 2018-11-05 14:48:35 Running reboot command 2018-11-05 14:53:36 WARNING: nov051744-506863 (google:debian-9-64-base:tasks/google/update-debian-9) running late. Output still empty. 2018-11-05 14:58:36 WARNING: nov051744-506863 (google:debian-9-64-base:tasks/google/update-debian-9) running late. Output still empty.

It can be reproduced easy on debian-9 where the reboot is done very fast (~8sec).

With this chage after the reboot command is triggered, if the warning timeout is reached and the command didn't finish, the ssh connection is closed manually, otherwise the execution continues as usual. Then the uptime is compared to make sure the rebook was properly done.

This is the spread output using -vv: https://paste.ubuntu.com/p/x3cyDFZ2nb/

sergiocazzolato avatar Nov 05 '18 18:11 sergiocazzolato

@niemeyer could you please take a look to this one?

Thanks

sergiocazzolato avatar Nov 28 '18 14:11 sergiocazzolato

@niemeyer, I already added the change to detect when the reboot command is stuck, and in that case we manually close the ssh connection. I already validated this with the images debian-9-stretch and debian-9-64-base.

sergiocazzolato avatar Jan 08 '19 01:01 sergiocazzolato

Currently I see this errors running on boards like pi3, where sometimes it is getting stuck when a reboot is requested. This is the new log for my pi3: https://paste.ubuntu.com/p/YgY3HJ8b7d/

sergiocazzolato avatar Apr 04 '19 15:04 sergiocazzolato

This is an error that heppened today: https://paste.ubuntu.com/p/kzJhnP7BHj/

sergiocazzolato avatar Apr 16 '19 13:04 sergiocazzolato