How to get BayerRG12 into an opencv mat?

Not sure if this is an issue or just lack in documentation. I'm trying to read BayerRG12 frames and convert them into opencv mat.

# Synchronous grab
from vimba import *
import cv2

with Vimba.get_instance () as vimba:
    cams = vimba.get_all_cameras ()
    with cams [0] as cam:

        #Camera is set to BayerRG12
        frame = cam.get_frame ()
        frame = frame.as_opencv_image() #Here it will fail 
        frame = cv2.cvtColor(frame, cv2.COLOR_BAYER_RG2BGR)
        cv2.imshow("test", frame.as_opencv_image ())
        cv2.waitKey(0)

Error Message: ValueError: Current Format 'BayerRG8' is not in OPENCV_PIXEL_FORMATS

Now, opencv could, as far as I see, convert images from BayerRG12 down eg. a RGB format. However, the Vimba function as_opencv_image() won't accept it.

If I look into the error, it seems it will only support specific formats:

OPENCV_PIXEL_FORMATS = (
    PixelFormat.Mono8,
    PixelFormat.Bgr8,
    PixelFormat.Bgra8,
    PixelFormat.Mono16,
    PixelFormat.Bgr16,
    PixelFormat.Bgra16
)

I also tried to convert it first to a supported format: frame.convert_pixel_format(PixelFormat.Bgr16) and then use the convert to mat function:

#Camera is set to BayerRG12
        frame = cam.get_frame ()
        frame.convert_pixel_format(PixelFormat.Bgr16)
        frame = frame.as_opencv_image() #Here it will fail 
        frame = cv2.cvtColor(frame, cv2.COLOR_BAYER_RG2BGR)

This fails with:

  File "C:\Users\justRandom\anaconda3\envs\tf-gpu-cuda8\lib\site-packages\vimba\util\runtime_type_check.py", line 60, in wrapper
    return func(*args, **kwargs)

  File "C:\Users\justRandom\anaconda3\envs\tf-gpu-cuda8\lib\site-packages\vimba\frame.py", line 762, in convert_pixel_format
    raise ValueError('Current PixelFormat can\'t be converted into given format.')

ValueError: Current PixelFormat can't be converted into given format.

Is there another way to get the bayer frame directly to a opencv mat function and do the conversions there?

The problem with the list of OpenCV compatible formats is, that once you do transform from Vimba to OpenCV, the information which pixel layout the data is in gets lost. So it seems like the list of OpenCV compatible formats is intentionally kept to just Mono formats, which are identifiable by their axis dimensions, and the default color format BGR and BGRA. Adding Bayer images to this would lead to an image array, that has dimensions indicating Mono data, which is not the case. Similarly adding RGB as supported formats would make it hard to differentiate between image data in BGR vs RGB. By only explicitly allowing the listed formats, the user can be sure that they start from one of these formats and transform further from there if needed.

That being said, it seems like you can just use frame.as_numpy_ndarray() instead. The following worked for me. Please note, that I changed the format conversion string of the cvtColor call to match my pattern. Also if you do use the 12 bit data, the image will likely be very dark. It might even appear to be entirely black as the 12 bit image data will not utilize the entire displayed 16 bit color depth. Image data should still be present but is probably hard to visualize appropriately:

# Synchronous grab
from vimba import *
import cv2

with Vimba.get_instance () as vimba:
    cams = vimba.get_all_cameras ()
    with cams [0] as cam:

        #Camera is set to BayerGR8
        frame = cam.get_frame ()
        frame = frame.as_numpy_ndarray() # replaces the original vimba.Frame object with a numpy.ndarray
        frame = cv2.cvtColor(frame, cv2.COLOR_BAYER_GR2BGR)
        cv2.imshow("test", frame)  # Note that here we use frame directly because with the line above the vimba.Frame object was replaced by a numpy.ndarray object
        cv2.waitKey(0)

Please let me know if this works for you. I would also be interested in you input in regards to the list of OpenCV formats. Does my explanation make sense or do you think we should extend the list of compatible formats to ALL formats that have some compatibility with OpenCV?

To get a color image you should manually read the bytes and only then convert it with BAYER_RG2RGB.

import cv2 
from vimba import * 
import numpy as np


with Vimba.get_instance() as vimba:
    cams = vimba.get_all_cameras()
    with cams[0] as cam:
        cam.set_pixel_format(PixelFormat.BayerRG8)
        frame = cam.get_frame()

        frame = np.frombuffer(frame._buffer,  dtype=np.uint8).reshape(frame._frame.width,frame._frame.height)
        frame = cv2.cvtColor(frame, cv2.COLOR_BAYER_RG2RGB)

        cv2.imshow('frame', frame)
        cv2.waitKey()

I have the Manta G-1620, I face the same issue trying to use one of the colour formats and save with open cv

None of the above examples work for my camera:

Example above from Niklas Kroeger takes image but the colours are very wrong, which is no good for my application

Likewise from kochsebastian, the resulting image is just a mess of colours for my camera

@NiklasKroeger-AlliedVision can you please advise the official example code from Allied Vision that can be used to get a colour image and save with open cv?

Thank you