Capability to Add Text Overlay Prior to Video Encoding

[Luxonis-Brandon]

Start with the why:

For applications that involve encoded video as a output of the system it can be valuable to have on-video metrics stored into the encoded video itself. Examples include sports-filming (e.g. here where having the score or other metrics/etc. saved into the video is very useful for live-streaming, or playback later) and for robotics applications being able to save machine-state (variables, actuator states, etc.) onto the encoded video helps a ton in debugging after-the-fact without any guessing as to the machine-state.

And in these applications, decoding the video on the host to then add this overlay, just to have to re-encode, is extremely burdensome and often intractable.

So have the capability to write this information onto the frames prior to the encoding on DepthAI/megaAI can be very useful.

Move to the how:

We have the capability to write directly to the video frames prior to encoding, so we could expose an API endpoint from the host (or over SPI/Ethernet) that allows data to be passed to DepthAI/megaAI which is then superimposed on the frames prior to encoding.

Likely some API endpoint that allows position, text size, font, color, background box style, duration (e.g. seconds, or always on), and text payload will be beneficial so that the host can pass this back into DepthAI/megaAI to be encoded into the video.

Move to the what:

Support text overlay capability prior to video encoding on DepthAI/megaAI.

[ericristad]

ffmpeg has similar capability; use reload to change the text in real time. https://ffmpeg.org/ffmpeg-filters.html#drawtext-1

[Luxonis-Brandon]

Will be good reference - thanks @ericristad .

[Erol444]

Would this feature be exposed from the script node? This would make it very customizable and would allow users to decode the NN results and display BB/labels on the frame, then forward the image to the VideoEncoder

[Luxonis-Brandon]

Yes we should definitely do so. So @alex-luxonis knows a bit more about the capability to do overlay on frames directly in Myriad X. Will bring up internally.

[Aposhian]

Any updates on this?

[Erol444]

Hi @Aposhian, likely not for the RVC2, but I believe we do plan to add this to RVC3/4, as it will have onboard Linux which will allow better flexibility (eg. to add this as well). Thanks, Erik