cv_bridge: CompressedImage conversions with 16bit grayscale images

[NikolausDemmel]

I have 16bit grayscale images (mono16) and would like to convert to and from CompressedImage. It seems cv_bridge is not properly handling this case in most instances.

Possibly related: image_transport also has issues with decoding 16bit grayscale: https://github.com/ros-perception/image_common/issues/77

Python

It looks like cv2_to_compressed_imgmsg works correctly and creates 16bit grayscale png images.

However, compressed_imgmsg_to_cv2 has issues. Firstly, imdecode is given only IMREAD_ANYCOLOR, which means the image is always converted to 8 bit. I think IMREAD_UNCHANGED is what we want here, or at least IMREAD_ANYCOLOR | IMREAD_ANYDEPTH (not sure what the difference is between these two cases). Secondly, if a specific encoding is asked for, it later assumes that the result from imdecode is bgr8 encoded. Not sure about the color encoding you can expect from imdecode, but at least for grayscale it seems wrong, and the actual bitdepth of the input would ideally also be respected here.

C++

I didn't try it for C++, but looking at the code, it seems that for neither conversion 16bit is supported.

https://github.com/ros-perception/vision_opencv/blob/9d49971aa9b32a662a91d9240d8a8b46dc60ca90/cv_bridge/src/cv_bridge.cpp#L480-L506

https://github.com/ros-perception/vision_opencv/blob/9d49971aa9b32a662a91d9240d8a8b46dc60ca90/cv_bridge/src/cv_bridge.cpp#L515-L536

[NikolausDemmel]

Just for the record:

Firstly, imdecode is given only IMREAD_ANYCOLOR, which means the image is always converted to 8 bit. I think IMREAD_UNCHANGED is what we want here, or at least IMREAD_ANYCOLOR | IMREAD_ANYDEPTH (not sure what the difference is between these two cases).

Still not sure, but the difference might be that ANYDEPTH | ANYCOLOR possibly flattens the alpha channel.

Secondly, if a specific encoding is asked for, it later assumes that the result from imdecode is bgr8 encoded. Not sure about the color encoding you can expect from imdecode, but at least for grayscale it seems wrong, and the actual bitdepth of the input would ideally also be respected here.

According to the documentation, imdecode returns either single channel for grayscale or 3-channel BGR for color images (or possibly BGRA for png with alpha channel)

[awesomebytes]

I needed to use IMREAD_UNCHANGED for a 16bit CompressedImage from compressedDepth topic.

[NikolausDemmel]

What happens with IMREAD_ANYCOLOR | IMREAD_ANYDEPTH instead?

[awesomebytes]

Dunno, I don't have the code handy to try. I'll try to remember to give it a try whenever I cross paths with this again. It was quite a pain to get it right.

[NikolausDemmel]

Ah ok, don't worry about it. I thought you did this just now. Anyway, thanks for the input.

[samarth-robo]

What happens with IMREAD_ANYCOLOR | IMREAD_ANYDEPTH instead?

@NikolausDemmel I just tried it. IMREAD_ANYCOLOR | IMREAD_ANYDEPTH works. So do IMREAD_GRAYSCALE | IMREAD_ANYDEPTH, and IMREAD_UNCHANGED.