`DeviceBootloader::flashUserBootloader()` fails yet it returns success

[diablodale]

On my oak-d-pro-poe, I have factory 0.0.17 bootloader. I've written c++ code that uses the Bootloader apis to allow my users to update their firmware. Using depthai-core 2.25.0 which has embedded bootloader 0.0.26. The firmware update API returned true for success, yet the bootloader actually did not update to 0.0.26. It remained on 0.0.17. Likely related to https://github.com/luxonis/depthai-core/issues/988

Pseudocode

deviceBootloader = std::make_unique<dai::DeviceBootloader>(candidate, true);
if (deviceBootloader->isUserBootloaderSupported()) {   // this returns true
    std::tie(success, resultMessage) = deviceBootloader->flashUserBootloader(localFlashCallback); // this returns {true, "blahblah"}
}
if (!success)
    throw
deviceBootloader.reset();
// wait
deviceBootloader = std::make_unique<dai::DeviceBootloader>(candidate);
assert(deviceBootloader.getVersion() == {0, 0, 17});

Unplugging/plugging mains power to POE does not help. The bootloader continues to be 0.0.17.

Suspicion

I suspect that the rules of https://docs.luxonis.com/projects/api/en/latest/components/bootloader/#danger-zoneg and bug #988 are interacting to result in the failure of this bug. isUserBootloaderSupported() is coded to check against bootloader version 0.0.21. Since #988 returns the wrong version, the compare is 0.0.26 > 0.0.21 so Supported() returns true. But should it return false instead?

Also, as can be seen from the DEPTHAI debug log, the firmware update process did SOMETHING for 18 seconds and then declared it did the flash Success flashing the configuration userBlSize to '3824072' and userBlChecksum to '2142766317' Below is the log, I add comments with // at the end of some lines

[2024-04-01 01:35:47.042] [depthai] [info] Search for 1 devices with criteria DeviceInfo(name=192.168.2.23, mxid=18443010318EF50800, X_LINK_BOOTLOADER, X_LINK_TCP_IP, X_LINK_MYRIAD_X, X_LINK_SUCCESS) + hint=false
[2024-04-01 01:35:50.212] [depthai] [debug] Found name=192.168.2.23, mxid=18443010318EF50800, X_LINK_BOOTLOADER, X_LINK_TCP_IP, X_LINK_MYRIAD_X, X_LINK_SUCCESS, hint=false
[2024-04-01 01:35:50.349] [depthai] [debug] Connected bootloader version 0.0.26   // this is std::make_unique<dai::DeviceBootloader>(candidate, true);
progress_callback() 35  // this is flashUserBootloader(localFlashCallback);
progress_callback() 35  // ignore these numbers, I am scaling them for a dialog box
progress_callback() 36
progress_callback() 37
progress_callback() 39
progress_callback() 40
progress_callback() 41
progress_callback() 42
progress_callback() 43
progress_callback() 44
progress_callback() 45
progress_callback() 46
progress_callback() 47
progress_callback() 49
progress_callback() 50
progress_callback() 51
progress_callback() 52
progress_callback() 53
progress_callback() 54
progress_callback() 55
progress_callback() 57
progress_callback() 58
progress_callback() 59
progress_callback() 60
progress_callback() 61
progress_callback() 62
progress_callback() 63
progress_callback() 64
progress_callback() 65
progress_callback() 67
progress_callback() 78
Exception thrown at 0x00007FFA9CB95B0C in Max.exe: Microsoft C++ exception: std::runtime_error at memory location 0x00000000007D9AC8.
[2024-04-01 01:36:08.087] [depthai] [debug] Error while trying to read existing bootloader configuration: No config available
[2024-04-01 01:36:08.753] [depthai] [debug] Success flashing the configuration userBlSize to '3824072' and userBlChecksum to '2142766317'
// here occurred if (!success) throw
progress_callback() 81
// here occurred deviceBootloader.reset();
[2024-04-01 01:36:08.758] [depthai] [debug] DeviceBootloader about to be closed...
[2024-04-01 01:36:08.762] [depthai] [debug] XLinkResetRemoteTimeout(5)
Exception thrown at 0x00007FFA9CB95B0C in Max.exe: Microsoft C++ exception: dai::XLinkWriteError at memory location 0x000000003B97FB20.
[2024-04-01 01:36:09.748] [depthai] [debug] DeviceBootloader closed, 989

Also possible this is a lack/ambiguous depthai documentation. Maybe since bug #988 and I have bootloader 0.0.17, that I need to first get the non-flashing bootloader version (which is 0.0.17) and then open a 2nd bootloader connection with flash=yes and ignore (don't call isUserBootloaderSupported() because it is wrong) and instead do my own logic compare of the bootloader version from the 1st connection (0.0.17) against a hardcoded api version constant (0.0.21) and then use that to choose flash to FACTORY or USER. And then with 0.0.26 (maybe?) now as the factory firmware I can do the same thing all over again with two bootloader connections to now detect with my own logic that a user bootloader is supported and I can then use flashUserBootloader() in the future.

I am suspicious what happened to my device's flash since the log shows 18 seconds of activity and Success flashing the configuration userBlSize to '3824072' and userBlChecksum to '2142766317'. It seems to work ok -- I get depth, color, and imu data.

[moratom]

@zrezke would you mind checking this out alongside #988 when you'll get to it?

[diablodale]

@zrezke, I see PR #997 that will change the logic so that isUserBootloaderSupported() will use the new getFlashedVersion(). Looks straightforward. Now code that is written will fail the bool isUserBootloaderSupported() test and not call flashUserBootloader(). A flash to factory is the only possibility on such devices.

Does your team know the state of my 0.0.17 device? Since the code did call flashUserBootloader() due to the two bugs, and did something to flash as in the OP logs. Can I ignore that activity? Can I factory flash now? My goal is to get off 0.0.17 to see if PoE is more reliable on multiple restarts (other bugs I'm tracking)

[zrezke]

@diablodale Yes - flashing factory bl now is fine - other than the soft-brick that could happen anyways, if the device gets disconnected during flashing the factory BL.