ISP scaling error is inconsistent/silent; sometimes outputs incorrect video

[diablodale]

trafficstars

illegal setIspScale() params sometimes throws exception, other times it only writes a silent log entry and outputs incorrect video. I request it always throw exception on all illegal params. The same behavior as an illegal setVideoSize()

Setup

• OAK-D via BW1098OBC
• Microsoft Windows [Version 10.0.19044.1415]
• depthai-core v2.13.3 + xlink move/thread fixes
• VS2019 v16.11.8

Repro

1. cmake config and build depthai-core for x64, debug, static library
2. cmake install
3. create main app that exercises setIspScale() for specific resolutions
4. point main app config to install location
5. config and build main app
6. Run main app

Case 1

rgbCamera->setResolution(THE_1080_P);
rgbCamera->setIspScale(480, 1080);

results in log entry

E: [global] [         0] [] XLinkReadMoveData:242	C[14442C10C1A1C6D200] [504.341] [ColorCamera(4)] [critical] Output video width 854 can't be bigger then output still width 832

and then throws exception

ColorCamera(4) - Color camera post processing invalid settings!

Why does it throw? width=854 is my goal. Why is "still width 832" involved?

Case 2

If I use width for the same goal...

rgbCamera->setResolution(THE_1080_P);
rgbCamera->setIspScale(854, 1920);

results in log entries

[14442C10C1A1C6D200] [345.275] [ColorCamera(4)] [error] Horizontal scaling 854/1920, simplified to 427/960 (chroma 427/1920), cannot meet HW constraints MAX_N:16, MAX_D:63. Horiz scaling disabled
[14442C10C1A1C6D200] [345.275] [ColorCamera(4)] [error] Vertical scaling 854/1920, simplified to 427/960 (chroma 427/1920), cannot meet HW constraints MAX_N:16, MAX_D:63. Vert scaling disabled

...then depthai continues with no errors or exceptions. It outputs the incorrect 1920x1080 video

Expected

The 1080P video to be scaled to 854x480 (FWVGA).

...or

Both case 1 and case 2 throw exceptions

[diablodale]

There is a pattern of failure related to Case 1 in OP. In this example, my goal is a output video of 1366x768 (WXGA) This can be achieve by scaling 5:7 and trimming less than 5 pixels in a crop.

rgbCamera->setResolution(THE_1080_P);
rgbCamera->setIspScale(5, 7);
rgbCamera->setVideoSize(1366, 768)

But it fails. It first logs...

E: [global] [         0] [] XLinkReadMoveData:242	Condition failed: (addEventWithPerf(&event, &opTime))E: [14442C10C1A1C6D200] [44.769] [ColorCamera(4)] [critical] Output video width 1366 can't be bigger then output still width 1344

Note that the number in that log 1344 is suspiciously a multiple of 64.

then depthai-core throws exception

ColorCamera(4) - Color camera post processing invalid settings!

[Luxonis-Brandon]

Thanks for the reports here! Adding @themarpe as he's been working on some refactoring here. Not sure if he or @alex-luxonis are best to comment here.

[diablodale]

haha...and I think I've found a related/series of bugs. Same goal... output video of 1366x768 (WXGA) Using an alternative of scaling 8:11 and cropping

rgbCamera->setResolution(THE_1080_P);
rgbCamera->setIspScale(8, 11);
rgbCamera->setVideoSize(1366, 768);

The data is being corrupted. Look at the rainbow...

But the same code with scaling 1280:1920 has correct coloring

If I point the camera to me and move, I can see the planes/layers. If I were to guess, the luminance is ok. But the Cr and Cb is being mis-indexed. I can see it being severly skewed.

[Luxonis-Brandon]

Thanks again! One of our goals for 2022 is to get a LOT more thorough automated testing on permutations/etc. like this as part of our short-term (build process) and long-term (stability over weeks) testing. In the meantime, this is super valuable for us that you are (finding and more importantly) reporting these.