depthai-core
depthai-core copied to clipboard
Oak-D-Pro-PoE overloaded on CSS vpu; leads to cascade failures in simple use cases
I'd like to raise awareness of the substantial work limits of the Pro-PoE as compared to the USB variants. Opening here as requested by @szabi-luxonis PoE crash #415 prevents extensive data collection so below I provide 6 data points.
Setup
- depthai-core v2.15.0
- Microsoft Windows [Version 10.0.19044.1586]
- VS2019 v16.11.11
- OAK-D-Pro-PoE preproduction
- OAK-D
Repro
Create and build C++ app that...
- sends device-side aligned depth (not disparity) and color at various resolutions
- Use
setIspScale()
andsetVideoSize()
to scale and crop - 30 fps
- subpixel true
- lr check true
- extended false
- median 5x5
- rectify edge fill = 0x00
- collect the device-side css and mss usage
- collect the fps of successfully synchronized depth+color framesets received by host (using the dict method)
Run test for various resolutions
Result
Pro-PoE appears to be CSS cpu bound. Even at 480p there is frame loss. As resolution increases, the CSS is so overloaded that it cascades problems which prevent MSS from doing work (notice mss usage is reverse expected) leading to cascades of mass failures and frame loss with zero synchronized frames received by host.
OAK-D | | | D-Pro-Poe | |||||||
---|---|---|---|---|---|---|---|---|---|
fps | CSS | MSS | Notes | | | fps | CSS | MSS | Notes | |
832x480p | 30 | 40% | 13% | good | | | 28 | 87% | 14% | minor lag and frame loss |
1280x720p | 30 | 42% | 14% | good | | | 14 | 100% | 10% | significant lag and frame loss |
1920x1080p | 24 | 48% | 21% | very minor lag | | | 0 | 100% | 8% | cascade failures, mass frame loss, unusable |
Expected
OAK-D and OAK-D-Pro-Poe to have similar cpu usage to enable similar work...bound by the on-wire ethernet bandwidth limit of 1Gbps
Related
https://discord.com/channels/790680891252932659/924800168530944020/952999966316576818 https://github.com/luxonis/depthai/issues/546#issuecomment-980472064
I tried https://github.com/luxonis/depthai-core/pull/428 as @szabi-luxonis suggested.... Same test. 480p hit 30fps. No change 720p. And 1080p hit 6fps...up from zero. Progress! :-)
D-Pro-PoE | fps | CSS | MSS | Notes |
---|---|---|---|---|
832x480p | 30 | 83% | 14.5% | very minor lag |
1280x720p | 14 | 100% | 13% | significant lag and frame loss |
1920x1080p | 6 | 100% | 9% | large lag, large frame loss |
Thanks again @diablodale ! As an update @diablodale we're discussing other ways to reduce the CSS load here. There are potentially some things that are causing unnecessary load that could be mitigated. We're in investigatory stage now but will report back as we make more progress here.
Got it. A deeper meta-topic that needs time/patience to work through. 👍