OAK-D SR , Couldn't access depth information while running depthai_demo.py

[Srikiranrao]

While running demo code - depthai_demo.py . we couldn't access the depth information but able to access RGB view from the OAK- D SR camera.

depthai$ python3 depthai_demo.py Using depthai module from: /home/delta/.local/lib/python3.8/site-packages/depthai.cpython-38-x86_64-linux-gnu.so Depthai version installed: 2.21.2.0 Setting up demo... Available devices: [0] 1844301001CF5D1200 [X_LINK_UNBOOTED] USB Connection speed: UsbSpeed.SUPER OV9782 requires 720 or 800 resolution, defaulting to SensorResolution.THE_800_P Disabling depth... Disabling depth preview... Disabling depthRaw preview... Disabling left preview... Disabling rectifiedLeft preview... Disabling right preview... Disabling rectifiedRight preview... [1844301001CF5D1200] [1.1] [0.907] [MonoCamera(2)] [error] Camera not detected on socket: 2 [1844301001CF5D1200] [1.1] [0.907] [MonoCamera(0)] [error] Camera not detected on socket: 1

[themarpe]

Thanks for the report @Srikiranrao

We are working on an updated Demo / Viewer which will better support newer devices. CC: @zrezke

[zrezke]

Hey @Srikiranrao, You can try the new depthai-viewer which should work with Oak-D SR. To install: python3 -m pip install depthai-viewer==0.0.7 --extra-index-url https://test.pypi.org/simple Then run using:

python3 -m depthai_viewer
# OR
depthai-viewer

[Srikiranrao]

@zrezke , Thanks. After following steps. I am facing another issue as mentioned below. this occurs when I select the camera module and try to run it.

Connected cam doesn't have IMU, skipping IMU creation...
Getting cam by socket:  CameraBoardSocket.CAM_B  Cameras:  [CameraConfiguration(fps=30, resolution=<CameraSensorResolution.THE_800_P: 'THE_800_P'>, kind=<CameraSensorType.COLOR: 0>, board_socket=<CameraBoardSocket.CAM_B: 1>, stream_enabled=True, name='Left'), CameraConfiguration(fps=30, resolution=<CameraSensorResolution.THE_800_P: 'THE_800_P'>, kind=<CameraSensorType.COLOR: 0>, board_socket=<CameraBoardSocket.CAM_C: 2>, stream_enabled=True, name='Right')]
[1844301001CF5D1200] [1.2] [3.016] [DetectionNetwork(6)] [warning] Network compiled for 6 shaves, maximum available 10, compiling for 5 shaves likely will yield in better performance
Updating pipeline: successful with message: {'message': 'Pipeline started'}
[1844301001CF5D1200] [1.2] [3.112] [system] [critical] Fatal error. Please report to developers. Log: 'PlgSrcMipi' '985'
Handling action:  Action.GET_PIPELINE
Got message:  {'type': 'Subscriptions', 'data': {'Subscriptions': ['ColorImage', 'LeftMono', 'RightMono', 'DepthImage', 'PinholeCamera', 'DepthImage', 'ImuData', 'DepthImage']}}
Handling action:  Action.SET_SUBSCRIPTIONS
Subscriptions:  [<Topic.ColorImage: 0>, <Topic.LeftMono: 1>, <Topic.RightMono: 2>, <Topic.DepthImage: 3>, <Topic.PinholeCamera: 4>, <Topic.DepthImage: 3>, <Topic.ImuData: 7>, <Topic.DepthImage: 3>]
Handling action:  Action.GET_SUBSCRIPTIONS
Couldn't get device profiling data
Couldn't get device profiling data
Couldn't get device profiling data
Couldn't get device profiling data
Couldn't get device profiling data
Couldn't get device profiling data
Couldn't get device profiling data
Couldn't get device profiling data
Couldn't get device profiling data
Couldn't get device profiling data
Couldn't get device profiling data
Couldn't get device profiling data
Couldn't get device profiling data
Resetting...
Closing device...
Done
Sending message:  {"type": "Error", "data": {"action": "FullReset", "message": "Device disconnected"}}
[2023-06-06T07:08:52Z ERROR re_viewer::depthai::depthai] Error: Device disconnected
Got message:  {'type': 'DeviceProperties', 'data': {'DeviceProperties': {'id': '', 'cameras': [], 'imu': None, 'stereo_pairs': []}}}
Handling action:  Action.SELECT_DEVICE
Selecting device:  

[zrezke]

@Srikiranrao, Thanks for reporting this issue. Seems like you're experiancing the same issue as here. Maybe it would be best if you subscribe to that issue, to get updates as we try to resolve it...

[LinasKo]

I'm having the exact same issue. The depth in depthai_demo.py does not appear. When running the depthai-viewer==0.0.7, it gives the aforementioned 'PlgSrcMipi' '985' error.

When accessing through the SDK, a similar issue pops up.

Code:

from depthai_sdk import OakCamera

with OakCamera() as oak:
    stereo = oak.create_stereo()
    oak.visualize(stereo)
    oak.start(blocking=True)

Output:

[2023-06-12 14:49:43] INFO [root.__exit__:328] Closing OAK camera
Traceback (most recent call last):
  File "/home/linasko/projects/depthai/linas/depth.py", line 4, in <module>
    stereo = oak.create_stereo()
  File "/home/linasko/.cache/pypoetry/virtualenvs/depthai-linas-fCmO-WM2-py3.10/lib/python3.10/site-packages/depthai_sdk/oak_camera.py", line 234, in create_stereo
    comp = StereoComponent(self._oak.device,
  File "/home/linasko/.cache/pypoetry/virtualenvs/depthai-linas-fCmO-WM2-py3.10/lib/python3.10/site-packages/depthai_sdk/components/stereo_component.py", line 121, in __init__
    raise Exception('OAK-1 camera does not have Stereo camera pair!')
Exception: OAK-1 camera does not have Stereo camera pair!
Sentry is attempting to send 2 pending error messages
Waiting up to 2 seconds
Press Ctrl-C to quit

Any estimates on how long until the fix for the SDK and depthai_demo.py? It'd make a difference knowing if it was in days, rather than a few weeks - we're pretty heavily blocked on needing to estimate the depth - that's why we bought the device in the first place.

[themarpe]

Any estimates on how long the fix the SDK and depthai_demo.py? It'd make a difference knowing if it was in days, rather than a few weeks - we're pretty heavily blocked on needing to estimate the depth - that's why we bought the device in the first place.

@LinasKo we've fixed it just now, and will be adding to v2.22.0. You may expect updated demo and others tomorrow if all goes according to plan

[LinasKo]

Can confirm that I can access depth using my prior code snippet. Note, the following is a note to the devs - I don't recommend following blindly if you're looking for a fix.

So, there's still some tiny small issues, that you're likely aware of already:

1. v2.22.0 is not supported by the latest SDK. Installing v2.21.0 along with the SDK and later upgrading to v2.22.0 doesn't cause installation issues, but depthai_demo doesn't start. As expected, I suppose.
2. The depth image contains many regions of artifacts. But the following is pretty hard to show without a video.

Normal: With WSL:

(WSL turned on with visualiser.config.stereo.wls_filter = True)

It might be hard to see, but the noise on the screen has a specific pattern that remains regardless of what's visible on the camera.

Fully Covered:

3. I believe it's because the depth pipeline uses the images as mono, instead of color. For example, oak.create_camera("left") gives a mono image. Trying to create the cameras as color cameras made the 3D pattern better as well.

The temporary fix was (formatting may have changed line numbers): site-packages/depthai_sdk/components/camera_components.py:

1. _create_node:189: Changed pipeline.createMonoCamera() -> pipeline.createColorCamera()
2. on_init:106: self.node.setResolution(dai.ColorCameraProperties.SensorResolution.THE_800_P)

Afterwards, oak.create_camera("left") provided a color image, and depth showed better results.

Normal: WSL:

Fully covered:

Hope some this helps!

[Nate711]

I had the same issue. Could you guys update the primary tutorial to let users know that the SR doesn't work with the demo application? https://docs.luxonis.com/en/latest/pages/tutorials/first_steps/#first-steps-with-depthai Seriously unprofessional