vision_opencv icon indicating copy to clipboard operation
vision_opencv copied to clipboard

feat: extend type adaptation to support cv::cuda::GpuMat

Open AiVerisimilitude opened this issue 1 year ago • 0 comments

Extended ROSCvMatContainer to support type adaptation for cv::cuda::GpuMat.

Internally I changed the frame type to a std::variant so the user can store either cv::Mat or cv::cuda::GpuMat. The user is responsible for checking the stored type first via holds_cv_type.

The use case here is for a pipeline that can use hardware acceleration across all nodes. Being able to pass cv::cuda::GpuMat is thus a requirement. The reasoning for having it type adapted together with cv::Mat is for interoperability.

The constructors from cv types were also condensed to a "pass by value and move" in order to have only 2 constructors (cv::Mat and cv::cuda::Mat) instead of 4(cv::Mat const &; cv::Mat &&; cv::cuda::GpuMat const &; cv::cuda::GpuMat &&)

[!WARNING] Usage of cv::cuda::GpuMat requires building of OpenCV with the CUDA module. I am not aware of this package being built with such module in mind but I am fine with it being a requirement that you pull & build the package as part of your workspace. Let me know your thoughts on this.

Example usage

creating a subscriber:

publisher = this->create_publisher<ROSCvMatContainer::AdaptedType>(topicName, 10);

getting the info:

void MyNode::image_topic_callback(std::unique_ptr<ROSCvMatContainer> image) {
  cv::cuda::GpuMat rbgMat;
   
  if (image->holds_cv_type<cv::Mat>()) {
    auto cvImage = image->cv_mat();
    rbgMat.upload(cvImage);
  } else if (image->holds_cv_type<cv::cuda::GpuMat>()) {
    rbgMat = image->cv_gpu_mat();
  } else {
    throw std::runtime_error("Unsupported image type");
  }
  ...

publishing:

  cv::cuda::GpuMat gpuImage;
  ...
  std_msgs::msg::Header header;
  header.stamp = now();
  
  auto result = std::make_unique<ROSCvMatContainer>(std::move(gpuImage), header);
  publisher->publish(std::move(result));

AiVerisimilitude avatar Jan 22 '24 12:01 AiVerisimilitude