arduino-cli
arduino-cli copied to clipboard
arduino
Recover from interrupted discovery/monitor installation
Describe the request
Reinstall pluggable discovery and monitor tools of the builtin package if the tool executable is not found.
Describe the current behavior
Arduino CLI automatically installs pluggable discovery and pluggable monitor tools of the builtin package when missing or updatable. These tools allow Arduino CLI to find the "ports" used for communication with Arduino boards.
It can happen that the installation is interrupted while in progress.
🙁 If a problem during the installation results in the creation of the tool's release installation folder without the tool executable, Arduino CLI can no longer do discovery. It will not automatically recover from this condition and the user must manually delete the incomplete installation in order to restore Arduino CLI to functionality.
To reproduce
$ ./arduino-cli version
arduino-cli Version: nightly-20220218 Commit: dd48868 Date: 2022-02-18T01:31:17Z
$ rm ~/.arduino15/packages/builtin/tools/serial-discovery/1.3.1/*
$ ./arduino-cli board list
Error detecting boards: Error starting board discoveries: [discovery builtin:serial-discovery process not started: fork/exec /home/per/.arduino15/packages/builtin/tools/serial-discovery/1.3.1/serial-discovery: no such file or directory]
No boards found.
Arduino CLI version
nightly-20220218
Operating system
Windows, Linux
Operating system version
Windows 10, Ubuntu 20.04
Additional context
Originally reported at https://forum.arduino.cc/t/ide-2-0-lost-all-com-ports/960650
This situation is more problematic for Arduino IDE users, since the error message about the missing tool is not visible to them and they only notice the symptom of the IDE not seeing their ports.
It might seem unlikely for this interrupted installation to occur under real world conditions, but the equivalent issue with Boards Manager producing incomplete tools installations has been reported by users periodically ever since the introduction of Boards Manager, at sufficient volume for Arduino Support to consider it worth a Help Center article:
https://support.arduino.cc/hc/en-us/articles/360018444739-Error-file-does-not-exist-no-such-file-or-directory-system-cannot-find-the-file-specified
This now results in a cryptic panic that gives no clue about the cause to mere mortals:
$ arduino-cli version
arduino-cli Version: 0.25.1-rc1 Commit: 436f0bb9 Date: 2022-07-23T14:28:16Z
$ rm ~/.arduino15/packages/builtin/tools/serial-discovery/1.3.2/*
$ ./arduino-cli board list
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4a51d9]
goroutine 35 [running]:
os.(*Process).signal(0xc000174770, {0x1034aa0, 0x15fd848})
/usr/local/go/src/os/exec_unix.go:64 +0x39
os.(*Process).Signal(...)
/usr/local/go/src/os/exec.go:138
os.(*Process).kill(...)
/usr/local/go/src/os/exec_posix.go:68
os.(*Process).Kill(...)
/usr/local/go/src/os/exec.go:123
github.com/arduino/arduino-cli/executils.(*Process).Kill(...)
/home/per/Documents/deleteme/arduino-cli/executils/process.go:120
github.com/arduino/arduino-cli/arduino/discovery.(*PluggableDiscovery).killProcess(0xc0000b5480)
/home/per/Documents/deleteme/arduino-cli/arduino/discovery/discovery.go:268 +0xc9
github.com/arduino/arduino-cli/arduino/discovery.(*PluggableDiscovery).Quit(0xc0000b5480)
/home/per/Documents/deleteme/arduino-cli/arduino/discovery/discovery.go:381 +0xde
github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).remove(0xc000070a20, {0xc000017dfe, 0x18})
/home/per/Documents/deleteme/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:82 +0xba
github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).RunAll.func1(0xc0000b5480)
/home/per/Documents/deleteme/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:131 +0x71
github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).parallelize.func1(0x1)
/home/per/Documents/deleteme/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:101 +0x68
created by github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).parallelize
/home/per/Documents/deleteme/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:99 +0x1c5
The demo was done using Linux, but the problem also occurs on Windows.
That is correct.
$ arduino-cli version
arduino-cli.exe Version: 0.25.1-rc1 Commit: 436f0bb9 Date: 2022-07-22T15:47:42Z
$ arduino-cli board list
Port Protocol Type Board Name FQBN Core
COM1 serial Serial Port Unknown
COM254 serial Serial Port (USB) Unknown
COM3 serial Serial Port Unknown
COM52 serial Serial Port (USB) Unknown
COM7 serial Serial Port (USB) Arduino Mega or Mega 2560 arduino:avr:mega arduino:avr
$ rm ~/AppData/Local/Arduino15/packages/builtin/tools/serial-discovery/1.3.2/serial-discovery.exe
$ arduino-cli board list
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x8 pc=0xdd27cd]
goroutine 40 [running]:
os.(*Process).signal(0xc000150770, {0x196b990, 0x1f386e0})
C:/Program Files/Go/src/os/exec_windows.go:49 +0x4d
os.(*Process).Signal(...)
C:/Program Files/Go/src/os/exec.go:138
os.(*Process).kill(...)
C:/Program Files/Go/src/os/exec_posix.go:68
os.(*Process).Kill(...)
C:/Program Files/Go/src/os/exec.go:123
github.com/arduino/arduino-cli/executils.(*Process).Kill(...)
E:/electronics/git-nobackup/arduino-cli/executils/process.go:120
github.com/arduino/arduino-cli/arduino/discovery.(*PluggableDiscovery).killProcess(0xc000985c00)
E:/electronics/git-nobackup/arduino-cli/arduino/discovery/discovery.go:268 +0xc9
github.com/arduino/arduino-cli/arduino/discovery.(*PluggableDiscovery).Quit(0xc000985c00)
E:/electronics/git-nobackup/arduino-cli/arduino/discovery/discovery.go:381 +0xde
github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).remove(0xc0003782e0, {0x1716256, 0x18})
E:/electronics/git-nobackup/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:82 +0xba
github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).RunAll.func1(0xc000985c00)
E:/electronics/git-nobackup/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:131 +0x71
github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).parallelize.func1(0x2e0031002e0032)
E:/electronics/git-nobackup/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:101 +0x68
created by github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).parallelize
E:/electronics/git-nobackup/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:99 +0x1c5
https://github.com/arduino/arduino-cli/pull/1814 fixed the panic I reported in my previous comment, but it did not resolve the feature request for Arduino CLI to recover from a situation where a discovery executable goes missing.
$ arduino-cli version
arduino-cli Version: nightly-20220808 Commit: d0b2556 Date: 2022-08-08T01:35:05Z
$ rm ~/.arduino15/packages/builtin/tools/serial-discovery/1.3.2/*
$ arduino-cli board list
Error detecting boards: Error starting board discoveries: [discovery builtin:serial-discovery process not started: fork/exec /home/per/.arduino15/packages/builtin/tools/serial-discovery/1.3.2/serial-discovery: no such file or directory]
No boards found.
I'm reopening this issue because, although a nice improvement in this area, https://github.com/arduino/arduino-cli/pull/2107 is not a complete solution.
The most common conditions where the user's encounter this issue are that only the executable file is missing (most often due to being quarantined by the user's security software). https://github.com/arduino/arduino-cli/pull/2107 detects the broken installation state only when the tool release installation folder is empty (per the issue's original simplified demo, which cleared the entire folder), but we are including a LICENSE.txt file in the discovery and monitor tools installations. This means that if only the essential executable goes missing, the directory is not empty and the recovery mechanism is not activated:
$ arduino-cli version
arduino-cli.exe Version: git-snapshot Commit: 2c2a5cc6 Date: 2023-04-22T13:21:59Z
$ ls "C:\\Users\\per\\AppData\\Local\\Arduino15\\packages\\builtin\\tools\\serial-discovery"
1.4.0/
$ ls "C:\\Users\\per\\AppData\\Local\\Arduino15\\packages\\builtin\\tools\\serial-discovery\\1.4.0"
LICENSE.txt serial-discovery.exe*
$ rm "C:\\Users\\per\\AppData\\Local\\Arduino15\\packages\\builtin\\tools\\serial-discovery\1.4.0\\serial-discovery.exe"
$ arduino-cli board list
Error starting discovery: discovery builtin:serial-discovery process not started: exec: "C:\\Users\\per\\AppData\\Local\\Arduino15\\packages\\builtin\\tools\\serial-discovery\\1.4.0\\serial-discovery": file does not exist
No boards found.
May I suggest installing in a temporary directory and then only after everything has been extraced copy it over to the final installation path?
If even a single file copy fails abort the whole process and remove the ones that have been copied.
Also I's suggest blocking the possibility of interrupting the execution to prevent it being interrupted during steps that would leave files behind.
Also to check if the installation is valid I'd create a checksum of the whole folder when it's first installed.
Then in IsInstalled() compare it with the current content checksum to verify its integrity, if they differ it's not installed.
May I suggest installing in a temporary directory and then only after everything has been extraced copy it over to the final installation path?
The issue here is that the copy is successful, but the antivirus will move the file into quarantine after the installation has been completed.
Also to check if the installation is valid I'd create a checksum of the whole folder when it's first installed. Then in IsInstalled() compare it with the current content checksum to verify its integrity, if they differ it's not installed.
This is an expensive operation to do at every run of the CLI....
The issue here is that the copy is successful, but the antivirus will move the file into quarantine after the installation has been completed.
We could add an after install check to verify the integrity and warn the user that installation wasn't successfull cause of the antivirus maybe.
Even using the same checksum strategy, assuming the file is removed immediately after the copy.
This is an expensive operation to do at every run of the CLI....
Fair enough, less expensive and frustrating than a bad UX for me though.
Maybe do it only when the CLI is trying to run a certain tool, and act accordingly.
Given the very simple format of the discoveries, I guess that just checking if the executable is there is enough.
Fair enough, less expensive and frustrating than a bad UX for me though.
UX wise you are adding a fixed delay to 100% of the users to solve an issue that 0.01% are experiencing...
Given the very simple format of the discoveries, I guess that just checking if the executable is there is enough.
Yeah, if the issue is present only for discoveries it would make sense to check only for those.
UX wise you are adding a fixed delay to 100% of the users to solve an issue that 0.01% are experiencing...
True that, I have no metrics either way. 😉