invalid StereoDepth mesh files does not stop Device start; instead invalid config should stop start

[diablodale]

If mesh files are invalid, the depthai-core ignores them and continues to startup the device/pipeline which will now serve invalid data.

My expectation is that when any (including mesh files) configuration is invalid, that the startup of the device/pipeline fails by throwing an exception.

Setup

• depthai-core multi_cam_support at d9ffc562c7a9ab27ce6dccac6c9bef45edd601e7
• Microsoft Windows [Version 10.0.19044.1526]
• VS2019 v16.11.10

Repro

• config, build, and install depthai-core above with: x64, debug, shared
• calibrate your OAK-D or OAK-D-PRO depth sensor so that you generate mesh files.
• write an app that will digest StereoDepth.depth data at 400p resolution and calls `stereoDepth.loadMeshFiles("left.mesh", "right.mesh"). Yes...intentionally attempt to load mesh files with resolution not matching your StereoDepth resolution.
• Run your app

Result

Your app runs and receives incorrect depth data. The data does not have mesh corrections. If you collect logs, you will get a log entry

[14442C10C1A1C6D200] [38435.988] [StereoDepth(5)] [error] Mesh size 33048 not matching resolution 640x400 with step 16x16, expected 8528. Mesh won't be loaded

Expected

An exception to be thrown, either at the call stereoDepth.loadMeshFiles() or the constructor Device(pipelineConfig) with exception.what() = "Mesh size 33048 not matching resolution 640x400 with step 16x16, expected 8528."

Workaround

Messy and reliable. I could try to use addLogCallback(). Then on every construct Device() wait a few seconds 😬 and on another thread parse any log callbacks for the substring "Mesh won't be loaded" and then use a condition variable to signal back to my Device() construct thread to then check a variable if there was an error.

• race conditions and unknown amount of time to wait for enough log entries
• complex code
• substring matching on an error string which can change on any commit

[alex-luxonis]

Thanks for the issue report.

We will change to a have an exception thrown here. It's evaluated on device when the pipeline is passed to, by:

Device(pipelineConfig);

or

d = Device(); 
...
d.startPipeline(pipelineConfig);

[SzabolcsGergely]

@diablodale Addressed in https://github.com/luxonis/depthai-core/commit/08be0a71b315c8fe4dbb58bf14bb0fe57e826e51

[diablodale]

Unfortunately, no exception is thrown with v2.15.3

• No log warning or error. It is completely silent in v2.15.3. This is worse than previous.
• No exception is thrown
• The OAK device sends a stream of frames to the host with unknown mesh/rectification

Below is console output when I send mesh data calculated for 16,16 yet I set stereoDepth->setMeshStep(32, 32);

[2022-04-23 18:32:20.525] [debug] Resources - Archive 'depthai-device-fwp-921b8a6a29fa445ea1c250d76dbe8694b5768584.tar.xz' open: 9ms, archive read: 6448ms
Max: maxdb update complete
[2022-04-23 18:32:29.473] [debug] Device - OpenVINO version: 2021.4
[2022-04-23 18:32:29.474] [debug] Device - BoardConfig: {"gpio":[],"uart":[],"usb":{"flashBootedPid":63037,"flashBootedVid":999,"maxSpeed":4,"pid":63035,"vid":999},"watchdogInitialDelayMs":null,"watchdogTimeoutMs":null} 
libnop:
0000: b9 05 b9 05 81 e7 03 81 3b f6 81 e7 03 81 3d f6 04 be be bb 00 bb 00
[14442C10C1A1C6D200] [279.844] [system] [info] Memory Usage - DDR: 0.12 / 339.99 MiB, CMX: 2.05 / 2.50 MiB, LeonOS Heap: 7.22 / 78.17 MiB, LeonRT Heap: 2.88 / 41.44 MiB
[14442C10C1A1C6D200] [279.844] [system] [info] Temperatures - Average: 32.25 ┬░C, CSS: 33.21 ┬░C, MSS 31.77 ┬░C, UPA: 31.77 ┬░C, DSS: 32.25 ┬░C
[14442C10C1A1C6D200] [279.844] [system] [info] Cpu Usage - LeonOS 12.36%, LeonRT: 2.38%
[2022-04-23 18:32:31.079] [debug] Schema dump: <redacted>
[2022-04-23 18:32:31.079] [debug] Asset map dump: {"map":{"/node/5/meshLeft":{"alignment":64,"offset":0,"size":8528},"/node/5/meshRight":{"alignment":64,"offset":8576,"size":8528}}}
[14442C10C1A1C6D200] [279.912] [MonoCamera(3)] [info] Using board socket: 1, id: 1
[14442C10C1A1C6D200] [279.913] [MonoCamera(4)] [info] Using board socket: 2, id: 2
[14442C10C1A1C6D200] [279.914] [system] [info] ImageManip internal buffer size '80640'B, shave buffer size '34816'B
[14442C10C1A1C6D200] [279.914] [system] [info] SIPP (Signal Image Processing Pipeline) internal buffer size '16384'B
[14442C10C1A1C6D200] [279.949] [system] [info] ColorCamera allocated resources: no shaves; cmx slices: [13-15] 
MonoCamera allocated resources: no shaves; cmx slices: [13-15] 
StereoDepth allocated resources: shaves: [13-13] cmx slices: [10-12] 
ImageManip allocated resources: shaves: [15-15] no cmx slices. 

[14442C10C1A1C6D200] [279.956] [StereoDepth(5)] [info] Using focal length from calibration intrinsics '427.86948'
[14442C10C1A1C6D200] [280.845] [system] [info] Memory Usage - DDR: 151.85 / 339.99 MiB, CMX: 2.22 / 2.50 MiB, LeonOS Heap: 48.80 / 78.17 MiB, LeonRT Heap: 4.60 / 41.44 MiB