mediapipe

mediapipe copied to clipboard

I customized a Calculator, the rough code is as follows:

::mediapipe::Status MyCalculator::Process(CalculatorContext* cc) {



    /////////////////////////////// hardcoded path.
    cv::Mat background = cv::imread("mediapipe/calculators/image/testdata/clouds2.png");
 
    ......  
}

, in the code, I have to hardcode, but what I hope is that this path can be passed in through parameters? All in all don't want to be hardcoded, any good suggestions?

Hi @Alvazz ,can you please elaborate this issue to investigate further.

@sureshdagooglecom Hi,

I customized a Calculator. My purpose is to use it as a background to replace Selfie Segmentation. The default Mediapipe is that the virtual background is blue, but I want to use a picture as a virtual background, so for this, I wrote a Calculator, but in the "Process(CalculatorContext* cc)" processing function, I cannot accept the hard-coded form to read the image used for the virtual background. What I hope is that the virtual The image path (or data) of the background is passed into the "Process(CalculatorContext* cc)" function (or, I am not sure if the external variable value can be read in this Process(CalculatorContext* cc) function??? ), so that I can control it externally. I am using C++ and the runtime environment is Windows 10. Version v0.8 is used.

Didn't find any useful information, need help.

Are you satisfied with the resolution of your issue? Yes No

Hi @Alvazz, have you found any solution to this? I am not getting any good results, this is my calculator file for background replacement, I dont know where I'm doing this wrong.

   ::mediapipe::Status BackgroundMaskingCalculator::Process(CalculatorContext* cc) {

  ///////////////////////////////// DESKTOP Asset reading method
   cv::Mat background = cv::imread("mediapipe/calculators/image/testdata/dino.jpg");

  std::cout << " Process() of BgMasking" << std::endl;

  const auto& input_img = cc->Inputs().Tag("IMAGE").Get<ImageFrame>();
  cv::Mat input_mat = formats::MatView(&input_img);

  const auto& mask_img = cc->Inputs().Tag("MASK").Get<ImageFrame>();
  cv::Mat mask_mat = formats::MatView(&mask_img);

  cv::cvtColor(background, background, cv::COLOR_BGR2RGB);
  cv::cvtColor(background, background, cv::COLOR_RGB2RGBA);
  cv::resize(background, background, input_mat.size());     

  cv::Mat channels[4];
  cv::split(mask_mat, channels);

  cv::Mat bg_mask;
  cv::Mat fg_mask;

  cv::Mat in[] = {channels[3], channels[3], channels[3],channels[3]};
  cv::merge(in, 4, bg_mask);


  cv::bitwise_not(bg_mask, fg_mask); 
  bg_mask.convertTo(bg_mask,CV_32FC3,1.0/255);
  fg_mask.convertTo(fg_mask,CV_32FC3,1.0/255);

  int k= 9;
  cv::GaussianBlur(bg_mask, bg_mask, cv::Size(k,k), 7);
  cv::GaussianBlur(fg_mask, fg_mask, cv::Size(k,k), 7);

  background.convertTo(background,CV_32FC3);
  input_mat.convertTo(input_mat,CV_32FC3);

  cv::Mat bg_image;
  cv::multiply(background,bg_mask, bg_image);

  cv::Mat fg_image;
  cv::multiply(input_mat,fg_mask, fg_image);

  cv::Mat output_image;
  cv::add(bg_image, fg_image, output_image);

  std::unique_ptr<ImageFrame> output_frame(
      new ImageFrame(input_img.Format(), input_img.Width(), input_img.Height()));

  cv::Mat output_mat = formats::MatView(output_frame.get());

  output_image.convertTo(output_image,CV_8UC3); 
  output_image.copyTo(output_mat);
  cc->Outputs().Tag("OUTPUT_VIDEO").Add(output_frame.release(), cc->InputTimestamp());

  return ::mediapipe::OkStatus();

  }
   REGISTER_CALCULATOR(::mediapipe::BackgroundMaskingCalculator);}

Hello @Alvazz, We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediapPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.

You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions. Thank you

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

This issue was closed due to lack of activity after being marked stale for past 7 days.

Are you satisfied with the resolution of your issue? Yes No