OAK-D Pro-W

[Luxonis-Brandon]

Preorders available: OAK-D-Pro-W-Dev - setup docs

Start with the why:

We’ve now gotten a bunch of reachouts about something like OAK-D, but with 150 DFOV, IR-laser-dot-projection depth, and matching resolution/FOV/global shutter for the color camera.

This is similar to this idea, https://github.com/luxonis/depthai-hardware/issues/21, which we already had on the roadmap, but with the change that apparently a lot of applications want this and wide FOV, which is separately https://github.com/luxonis/depthai-hardware/issues/15.

Move to the how:

ArduCam already implemented https://github.com/luxonis/depthai-hardware/issues/15 , and now produces this IR-capable OV9282 150DFOV IR-capable module as a standard product for DepthAI customers, here. So far though, this has only been used in custom products built with OAK-SOM/etc.

We should work with ArduCam to (1) update this OV9282 to the new/standard ArduCam connector format here and (2) make a OV9782 variant with the same connector/format but that blocks IR (for good color representation). Likely prototyping first with the existing format/connector

​​Move to the what:

Wide FOV version of OAK-D-PRO (https://github.com/luxonis/depthai-hardware/issues/114)

OAK-D-PRO-W (W = WideFOV)

• 2x OV9282 1280x800 global shutter grayscale 150 DFOV / 127 HFOV (IR-capable)
• 1x OV9782 1280x800 global shutter color 150 DFOV / 127 HFOV (IR-blocking)
• 2x IR laser dot emitter (for wide FOV)
• 2x IR LED (for wide FOV)
• Similar (same) form-factor as https://github.com/luxonis/depthai-hardware/issues/114

Update:

The current iteration of hardware does not support the dot-emitters. The main reason is that the laser certification was taking longer than expected and the chip shortage for the projector prevented us from making enough boards. As we got a lot of requests for WFOV cameras only, we instead shifted the focus on this aspect of the design to have it out as soon as possible. The flood LED is still implemented on the design and can be used for testing in low light situations.

[ptyork]

Love this. Given the challenge of resolution/detail at a wide FOV, would it be possible to make just the RGB cam truly modular to allow use of a higher-resolution RGB sensor (e.g., IMX477) instead of the OV9782 for those less concerned with global shutter and matched resolution?

[Luxonis-Brandon]

That's an interesting point @ptyork . So it may be possible to make a M12 variant of this design. The challenge with modularity is making it still reasonable for calibration to hold/etc.

But perhaps such a design could be useful. Say an aluminum enclosure with M12 lens mounts. Thoughts?

[ptyork]

The challenge with modularity is making it still reasonable for calibration to hold/etc.

I guess my thought was that the modularity would be limited only to the RGB camera and not something that was designed to be done frequently. Perhaps simply working with Arducam to standardize the RGB modules' PCB size and mounting it with M2'ish screws instead of adhesive? If so it would seem that a rare/one-time recalibration of the single camera wouldn't be too onerous.

But perhaps such a design could be useful. Say an aluminum enclosure with M12 lens mounts. Thoughts?

Perhaps. But I really like your bespoke wide angle design concept. Presumably you'd be matching the dual emitters to the 150 degree FOV. So allowing a lens change would seem to be of limited value to the end user. I mean, I can always envision a use, but I don't think that would add value in our case.

A use-case I'm envisioning here is something like off-axis OCR. With a 1MP RGB sensor, would there be enough detail to make out the words on (e.g.) an office placard near the periphery after rectifying a 150 degree fisheye image? I'd imagine the extra resolution would be very useful for this scenario (which for us, might be quite common).

[ptyork]

Oh, and since we're talking future designs and "pro" models, how feasible would a light-sensor+actuated-IR-filter be? I.e., choose and optimize for active vs. passive depth automatically based on lighting conditions?

Maybe not on the camera modules (bulky/pricey), but integrated into the glass front of an optional "pro+" enclosure, thus making it an upgrade option for either pro model? And if not a near-term option, you could "future-proof" the design for this kind of concept by adding a QT/Qwiic i2c+power connector to the OAK's PCB. Would also allow for other "accessory" enclosures...actuated 2x zoom lens, maybe?? Dunno. Just a thought.

[Luxonis-Brandon]

Thanks for the brainstorming @ptyork . So in terms of actuated IR filter - that's an interesting idea. I will have to think more on the feasibility of the other ideas as well. Thinking...

[Luxonis-Brandon]

Initial layout of open source design. This is MUCH bigger than it will be - for prototyping/test purposes.

The final unit will be small like #114 .

[ptyork]

Very nice. Dumb question: what's that 10-pin connector on the front for? JTAG? Happy to see the BNO085 make the cut on this one.

[Luxonis-Brandon]

Not a dumb question. We'll probably change it to be 2 connectors actually. That is the connector for an external laser-projector board. We did this so that we can permute on laser dot projector orientations/etc. w/out re-spinning the main board.

[Luxonis-Brandon]

[doisyg]

Looks great! When can we preorder some ?

[Luxonis-Brandon]

Thanks @doisyg . We'll see how to do this and then get a page up.

Thanks, Brandon

[Luxonis-Brandon]

These are both initial engineering-sample/experimentation versions. The final versions will be single-board and smaller.

[ptyork]

Nice!! Very exciting!

The final versions will be single-board and smaller.

Fully single-board? IE, no separate SOM? Or as in you're integrating the projectors onto this single board?

[Luxonis-Brandon]

Thanks @ptyork ! Yes, so what we realized is our customers like having 2 disparate options:

1. Reference designs (like the above) which use the SoM and showcase how to easily/quickly custom integrate.
2. Finished product solutions that are smaller/lighter for cases when customization isn't required.

So then most (perhaps all) product releases will follow this:

1. An open-source reference design which also serves as our first prototypes (as making boards with the SoM is WAY quicker both from engineering standpoint and also fabrication (because they're simple)).
2. A finished product that is chip-down (so can't be open-sourced unfortunately) and single-board which is custom-made for the application.

Thanks again! -Brandon

[Luxonis-Brandon]

Working on the final version of this (non-dev-board version) now that we have validated the concept via the dev board:

[Luxonis-Brandon]

We're adding the capability to use USB cables that can fasten to the device.

[doisyg]

That's a great idea !

[kota-rr]

Hi, RGB camera of this module is OV9782 1280x800 global shutter color 150 DFOV / 127 HFOV (IR-blocking) which is different from IMX378 fixed-focus of https://github.com/luxonis/depthai-hardware/issues/114. What are the reasons why hese two models are using different RGB cameras? Thanks.

[Erol444]

What are the reasons why hese two models are using different RGB cameras?

Hello @kota-rr , I believe these will be used only for prototypes, and production devices will actuallyhave the 4k IMX on there with wide lens. Reason is that Arducam currently only has OV9782 color cam with 150 degree FOV.

[kota-rr]

What are the reasons why hese two models are using different RGB cameras?

Hello @kota-rr , I believe these will be used only for prototypes, and production devices will actuallyhave the 4k IMX on there with wide lens. Reason is that Arducam currently only has OV9782 color cam with 150 degree FOV.

Got it. Thanks.

[Luxonis-Brandon]

Oh and we’re planning on making it a factory-orders or permutation - whether wide IMX378 or wide OV9782 is wanted.

[doisyg]

Just received the OAK-D Pro W Dev, it already looks super nice. Where should I start ? Which branch ? I tried the oak-d-pro branch but could not launch the demo:

~/depthai$ python3 depthai_demo.py Using depthai module from: /home/gd/.local/lib/python3.8/site-packages/depthai.cpython-38-x86_64-linux-gnu.so Depthai version installed: 2.14.1.0.dev+1d7d9e21435df48a2c69ff923e9b01cd07d0f420 Available devices: [0] 14442C10811440D700 [X_LINK_BOOTLOADER] [14442C10811440D700] [21.998] [system] [warning] Calibration Data on device is empty ===== Op 1: dev.irWriteReg(0x2, 0x1) USB Connection speed: UsbSpeed.SUPER ===== Op 2: dev.irWriteReg(0x3, 0x0) ===== Op 3: dev.irWriteReg(0x4, 0x0) ===== Op 4: dev.irWriteReg(0x5, 0x0) ===== Op 5: dev.irWriteReg(0x6, 0x0) ===== Op 6: dev.irWriteReg(0x7, 0x9) ===== Op 7: dev.irWriteReg(0x8, 0x1a) ===== Op 8: dev.irWriteReg(0x9, 0x8) ===== Op 9: dev.irWriteReg(0x1, 0x24) [14442C10811440D700] [22.197] [ColorCamera(5)] [warning] Unsupported resolution set for detected camera OV9782, needs 800_P or 720_P. Defaulting to 800_P Warning: calibration data missing, using OAK-D defaults [14442C10811440D700] [22.390] [system] [critical] Fatal error. Please report to developers. Log: 'plgppenc' '594' Traceback (most recent call last): File "depthai_demo.py", line 273, in pv.prepareFrames(callback=callbacks.onNewFrame) File "/home/gd/depthai/depthai_sdk/src/depthai_sdk/managers/preview_manager.py", line 99, in prepareFrames packet = queue.tryGet() RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'color' (X_LINK_ERROR)' Killed

(sorry if this is not the proper place for that)

[SzabolcsGergely]

@doisyg please check: oak-d-pro-w branch. Added support for it. Note that at the moment stereo and RGB resolution must match, and they are hard coded to 720p for the purpose of demo.

[richard-xx]

@szabi-luxonis I run python3 depthai_demo.py --show left right depth color using the oak-d-pro-w branch and it's like DM9098pro, but I don't seem to see similar results. Does it need to be calibrated? Or do I need to be in complete darkness to see the results?

[SzabolcsGergely]

@szabi-luxonis I run python3 depthai_demo.py --show left right depth color using the oak-d-pro-w branch and it's like DM9098pro, but I don't seem to see similar results. Does it need to be calibrated? Or do I need to be in complete darkness to see the results?

It needs to be calibrated, but at the moment the demo doesn't support dewarping for left/right frames, which is needed for good depth results. cc: @alex-luxonis @VanDavv

[richard-xx]

Here is the frame I run the demo:

This is my device and when I start it up the IR LED doesn't seem to work:

[alex-luxonis]

@richard-xx I'll update the demo later today to use the latest IR control API (the low level API doesn't exactly work on OAK-D Pro-W-DEV), and send also mesh files for rectification, as otherwise the depth map won't look good.

Could you try for now this example: https://github.com/luxonis/depthai-python/blob/multi_cam_support/examples/StereoDepth/stereo_depth_video.py Add the option -lm to load mesh data.

To enable the IR drivers, add one or both of the lines below:

 with dai.Device(pipeline) as device:
+    device.setIrLaserDotProjectorBrightness(200)  # current in mA, 0..1200
+    device.setIrFloodLightBrightness(100)         # current in mA, 0..1500

[moshimeow]

What's the max framerate of these cameras? 60fps or 120fps?

[Erol444]

What's the max framerate of these cameras? 60fps or 120fps?

Hello @slitcch , these are current FPS for given resolution that depthai supports:

• IMX378 (color cam): 30 FPS at 4k, 60FPS at 1080P
• OV9282 (mono camera): 60 FPS at 720P/800P, 120FPS at 400P

[crnolan]

Is there a facility to switch the auto-focus camera to manual focus mode? I have an application that requires < 50 cm working distance but the plane of interest is largely fixed in any particular scenario.

Also, does the depth sensing work effectively at distances < 50 cm?