[Feature-Request] {Reduce image copying}

[AiYangSky]

Start with the why:

I aim to transfer the captured images to other processes for further processing. Currently, I am utilizing shared memory to minimize system overhead; however, I have not identified a suitable interface to eliminate the overhead associated with image copying. Additionally, it has been observed that the ImgFrame::getFrame interface also involves a copying operation. Could you please consider optimizing this aspect?

Move to the what:

Could you please provide an interface that enables image data to be directly stored in the user-provided cache?

Like ImgFrame& setFrame(cv::Mat frame), but only providing a buffer instead of cv::Mat.

Move to the how:

Could we consider mapping the user-provided buffer to RawBuffer::data?

[moratom]

@AiYangSky the getFrame API has a flag copy which is set to false by default, we only make a copy in getCvFrame on the mainline.

What DepthAI version are you using? We have removed that option on v3.x branches, because it was not "safe" and a user had to manually take care that the reference to the DepthAI message outlived the cv::Mat frame.

[AiYangSky]

The version I used should be v2.28.0. Yes, I have also observed that the deepCopy parameter of the getFrame function maps the cache managed by depthai directly into the output cv::Mat. This approach indeed raises lifecycle concerns and compromises safety. Could the lifetime of this cache be transferred to user management? Such a change would provide both flexibility and enhanced security. In practical engineering scenarios, our primary focus is on the contents of cv::Mat::data, as all other aspects are predictable. In the current application, we implemented the following adjustments:

void ImgFrame::getFrame(void* buff, long size) {
    if(getWidth() <= 0 || getHeight() <= 0) {
        throw std::runtime_error("ImgFrame metadata not valid (width or height = 0)");
    }

    std::memcpy(buff, img.data.data(), std::min((long)(img.data.size()), size));
}

However, we believe further optimization is possible to avoid reliance on std::memcpy.

[moratom]

I see @AiYangSky - we briefly explored if we could bind the lifetime of the memory nicely to (als) cv::Mat, but couldn't find a good option.

We're happy to accept a PR if you manage to find something though!