Oak-D-Pro-PoE overloaded on CSS vpu; leads to cascade failures in simple use cases

[diablodale]

I'd like to raise awareness of the substantial work limits of the Pro-PoE as compared to the USB variants. Opening here as requested by @szabi-luxonis PoE crash #415 prevents extensive data collection so below I provide 6 data points.

Setup

• depthai-core v2.15.0
• Microsoft Windows [Version 10.0.19044.1586]
• VS2019 v16.11.11
• OAK-D-Pro-PoE preproduction
• OAK-D

Repro

Create and build C++ app that...

• sends device-side aligned depth (not disparity) and color at various resolutions
• Use setIspScale() and setVideoSize() to scale and crop
• 30 fps
• subpixel true
• lr check true
• extended false
• median 5x5
• rectify edge fill = 0x00
• collect the device-side css and mss usage
• collect the fps of successfully synchronized depth+color framesets received by host (using the dict method)

Run test for various resolutions

Result

Pro-PoE appears to be CSS cpu bound. Even at 480p there is frame loss. As resolution increases, the CSS is so overloaded that it cascades problems which prevent MSS from doing work (notice mss usage is reverse expected) leading to cascades of mass failures and frame loss with zero synchronized frames received by host.

	OAK-D				|	D-Pro-Poe
	fps	CSS	MSS	Notes	|	fps	CSS	MSS	Notes
832x480p	30	40%	13%	good	|	28	87%	14%	minor lag and frame loss
1280x720p	30	42%	14%	good	|	14	100%	10%	significant lag and frame loss
1920x1080p	24	48%	21%	very minor lag	|	0	100%	8%	cascade failures, mass frame loss, unusable

Expected

OAK-D and OAK-D-Pro-Poe to have similar cpu usage to enable similar work...bound by the on-wire ethernet bandwidth limit of 1Gbps

Related

https://discord.com/channels/790680891252932659/924800168530944020/952999966316576818 https://github.com/luxonis/depthai/issues/546#issuecomment-980472064

[diablodale]

I tried https://github.com/luxonis/depthai-core/pull/428 as @szabi-luxonis suggested.... Same test. 480p hit 30fps. No change 720p. And 1080p hit 6fps...up from zero. Progress! :-)

D-Pro-PoE	fps	CSS	MSS	Notes
832x480p	30	83%	14.5%	very minor lag
1280x720p	14	100%	13%	significant lag and frame loss
1920x1080p	6	100%	9%	large lag, large frame loss

[Luxonis-Brandon]

Thanks again @diablodale ! As an update @diablodale we're discussing other ways to reduce the CSS load here. There are potentially some things that are causing unnecessary load that could be mitigated. We're in investigatory stage now but will report back as we make more progress here.

[diablodale]

Got it. A deeper meta-topic that needs time/patience to work through. 👍