Error: "VideoFrame.data: Corrupted buffer" when doing videocrop before gvainference

[mschreil]

Hi, I am doing my custom preprocessing of the model output in a probe attached to the src of the gvainference plugin. I works out well unit i try to get the input image from the pipeline. As soon as with frame.data() as mat gets called, the error Videoframe.data: Corrupted buffer is raised.
In my pipeline I am having a perspective transformation followed by a videocrop before it goes into the model.

videoconvert n-threads=4 capsfilter caps="video/x-raw,format=BGR ! perspective name=perspective0 ! videocrop right=930 bottom=932 !  gvainference name=gvainference0 model=/home/dlstreamer/models/model.xml model_proc=/home/dlstreamer/models/model_proc.json device=GPU ! 

def post_probe_callback(pad, info, data):
    with util.GST_PAD_PROBE_INFO_BUFFER(info) as buffer:
        caps = pad.get_current_caps()
        frame = VideoFrame(buffer, caps=caps)
        with frame.data() as mat: 
            pass 
            # do some postprocessing here 

this is how my model_proc.json file set up:

{
    "json_schema_version": "2.2.0",
    "input_preproc": [
        {
            "params": {
                "color_space": "BGR",
                "resize": "aspect-ratio"
            }
        }
    ],
    "output_postproc": []
}

https://github.com/dlstreamer/dlstreamer/blob/master/python/gstgva/video_frame.py#L180 It seems that the "mapped_data_size" and "requested_size" are of different size. Does anyone have an idea where this difference can come from

[mschreil]

Found a solution
it seems that the issue is coming from the videocrop and the format of the image that goes into the videocrop. I did some investigations on the actual buffer size and the requested buffer size (img_height * img_width * n_channels)

videoconvert n-threads=4 ! \
capsfilter caps="video/x-raw,format=BGR" ! \
perspective name=perspective0 ! \
videocrop name=videocrop0 right=930 bottom=932 ! \

img_shape:  (118, 470, 3)
img_requested_buffer_size:  166380
buffer_size:  166616
buffer_diff:  -236

After setting the format to BGRx or BGRA the videocrop did output the correct buffer size and requested buffer size

videoconvert n-threads=4 ! \
capsfilter caps="video/x-raw,format=BGRx" ! \
perspective name=perspective0 ! \
videocrop name=videocrop0 right=930 bottom=932 ! \

img_shape:  (118, 470, 4)
imge_bytes:  221840
buffer_bytes:  221840
image_bytes - buffer_bytes:  0

[akwrobel]

Closing as submitter has found a solution