Add temporary solution for decoding compressed depth message inside cv_bridge

[chjz1024]

I added a temporary solution for directly decoding compressed depth image message inside cv_bridge with respect to https://answers.ros.org/question/249775/display-compresseddepth-image-python-cv2/ and https://github.com/ros-perception/image_transport_plugins/tree/indigo-devel/compressed_depth_image_transport.

It is very inconvenient if no python package is provided to support decoding compressed depth image directly, but it seems like there is no plan for python support of image_transport so I added this temporary solution in cv_bridge. Future improvement should be to add python wrapper of image_transport instead of implementing that functionality inside python.

I'm also wondering is it possible to export python wrapper directly from c++ code like https://github.com/opencv/opencv_contrib and https://docs.opencv.org/trunk/da/d49/tutorial_py_bindings_basics.html?

[ijnek]

Hi @chjz1024 and others who have reacted to the PR, I've recently become the maintinaer of this repo, and I'm going through the list of PRs to figure out which ones are still active and important.

I'm going to close this PR out as I believe it is now stale, but if anyone thinks these changes are still relevant, please comment on this PR and we can get it re-opened.

[amalnanavati]

This is definitely still relevant. The format in this PR is the default way compressed_depth_image_transport represents compressed depth formats, which cannot currently be handled by cv_bridge. image_transport plugins are used commonly by many libraries, including realsense_ros, so it is important for cv_bridge to be able to handle them.

[amalnanavati]

For what it's worth, I verified in my own codebase that the following code correctly encodes and decodes an image message with format 16UC1; compressedDepth to/from a CV2 depth image:

import array
import cv2
import numpy as np
import numpy.typing as npt
from sensor_msgs.msg import CompressedImage

__COMPRESSED_DEPTH_16UC1_HEADER = array.array(
    "B", [0, 0, 0, 0, 46, 32, 133, 4, 192, 24, 60, 78]
)

def compressed_depth_msg_to_cv2(msg: CompressedImage) -> npt.NDArray:
    return cv2.imdecode(
        np.frombuffer(
            msg.data[len(__COMPRESSED_DEPTH_16UC1_HEADER) :], np.uint8
        ),
        cv2.IMREAD_UNCHANGED,
    )

def cv2_to_compressed_depth_msg(img: npt.NDArray, png_level: int = 1) -> CompressedImage:
    success, data = cv2.imencode(
        ".png",
        img,
        [cv2.IMWRITE_PNG_COMPRESSION, 1],
    )
    if not success:
        raise RuntimeError("Failed to compress image")
    msg = CompressedImage(
        format="16UC1; compressedDepth",
        data=__COMPRESSED_DEPTH_16UC1_HEADER.tobytes() + data.tobytes(),
    )

(NOTE: the above code may have minor syntax errors, as I copy-pasted it from my codebase to create a dedicated example)

The decoding function follows the same logic as this PR. Encoding function(s) are not provided by this PR.

I am not opening a PR or reviewing/approving this one, because I did not test the 32UC1 functionality of this PR (although it does seem to mirror the functionality in the codec). Further, note that although this PR and my above code assumes PNG compression, ROS2 Rolling now have RVL compression as well, which should perhaps also be added to this repository.

I don't have the capacity to work on this now, but I hope the above code and that in this PR helps folks who need to decode/encode compressed depth images.