depthai-core icon indicating copy to clipboard operation
depthai-core copied to clipboard

Color/MonoCamera `frameEvent` output, per-frame IR control

Open alex-luxonis opened this issue 2 years ago • 2 comments

ColorCamera/MonoCamera: add frameEvent output, to be used for IR control, etc.

  • Outputs metadata-only ImgFrame message as an early indicator of an incoming frame. Fields populated for now: camera id, sequence number, timestamp
  • It's sent on the MIPI SoF (start-of-frame) event, just after the exposure of the current frame has finished and before the exposure for the next frame starts.
  • Could be used to synchronize various processes associated with per-frame control of camera capture: IR dot/flood alternate, manual exposure/focus, etc. Especially with a Script node running on device for low latency control

Add new Script API, similar in functionality to the respective host API:

  • Clock.now() -> datetime.timedelta
    • at the moment this returns a timestamp based on the device monotonic clock, not synced to host time
  • Device.setIrLaserDotProjectorBrightness(mA: float) -> bool
  • Device.setIrFloodLightBrightness(mA: float) -> bool
  • Device.getIrDrivers() -> List[Tuple[str, int, int]]

TODO: add C++ example to illustrate the usage

Misc:

  • Script: add optional name parameter to setScriptPath, might come handy if the path is long, to have a shorter identifier

alex-luxonis avatar Apr 13 '22 01:04 alex-luxonis

This is SWEET!

Luxonis-Brandon avatar Apr 13 '22 02:04 Luxonis-Brandon

@alex-luxonis Would it be better to integrate this (IR switching) directly into the Color/MonoCamera node to provide for an easier use and expose a property to enable? (the node would then check if IR driver is available, and take as much control over it as it needs to)?

themarpe avatar Apr 14 '22 11:04 themarpe

@alex-luxonis thoughts on making this a native node instead? And or integrating directly into Color/MonoCamera?

CC: @moratom on impl. afterwards

themarpe avatar Sep 19 '22 11:09 themarpe

Yes, later we should simplify the usage. frameEvent would be still useful though for custom things, e.g. by toggling a GPIO controlling an external strobe light.

alex-luxonis avatar Oct 07 '22 08:10 alex-luxonis