Segmentation fault (core dumped)

[optimaltaskin]

trafficstars

Please make sure that this is a solution issue.

System information (Please provide as much relevant information as possible)

OS Platform and Distribution linux Ubuntu 21.04
MediaPipe version: v0.8.10.2
Bazel version: v1.12.0
Solution (e.g. FaceMesh, Pose, Holistic): Face Effect (Face Mesh)
Programming Language and version: C++ 10.3.0

Describe the expected behavior: Trying to run face effect module to render glasses on detected face.

Standalone code you may have used to try to get what you need : I'm using demo_run_graph_main_gpu.cc script to run face effect module. The BUILD file to I'm using is as below(which is located in graphs/face_effect/):

load(
    "//mediapipe/framework/tool:mediapipe_graph.bzl",
    "mediapipe_binary_graph",
)

licenses(["notice"])

package(default_visibility = ["//visibility:public"])

cc_library(
    name = "face_effect_gpu_deps",
    deps = [
        "//mediapipe/calculators/core:flow_limiter_calculator",
        "//mediapipe/calculators/core:gate_calculator",
        "//mediapipe/calculators/core:immediate_mux_calculator",
        "//mediapipe/calculators/image:image_properties_calculator",
        "//mediapipe/framework/tool:switch_container",
        "//mediapipe/graphs/face_effect/subgraphs:single_face_geometry_from_detection_gpu",
        "//mediapipe/graphs/face_effect/subgraphs:single_face_geometry_from_landmarks_gpu",
        "//mediapipe/modules/face_geometry:effect_renderer_calculator",
        "//mediapipe/modules/face_geometry:env_generator_calculator",
    ],
)

mediapipe_binary_graph(
    name = "face_effect_gpu_binary_graph",
    graph = "face_effect_gpu.pbtxt",
    output_name = "face_effect_gpu.binarypb",
    deps = [":face_effect_gpu_deps"],
)

and this(the one I created under examples folder to build and run this module along with example codes.):

licenses(["notice"])

package(default_visibility = ["//mediapipe/examples:__subpackages__"])

cc_binary(
    name = "face_effect_gpu",
    deps = [
        "//mediapipe/examples/desktop:demo_run_graph_main_gpu",
        "//mediapipe/graphs/face_effect:face_effect_gpu_deps",
    ],
)

If there is a problem, provide a reproducible test case that is the bare minimum necessary to generate the problem. If possible, please share a link to Colab/repo link /any notebook: Here is my forked repo: https://github.com/optimaltaskin/mediapipe

To produce same error, first build the graph with command below:

bazel build -c opt --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 mediapipe/examples/desktop/_face_effect:face_effect_gpu --verbose_failures

then run with command below:

GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/_face_effect/face_effect_gpu --calculator_graph_config_file=mediapipe/graphs/face_effect/face_effect_gpu.pbtxt

Other info / Complete Logs : Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached:

Since script I'm using is same as demo_run_graph_main_gpu.cc I'm not sharing it. The graph config is as below(I only added ConstantSidePacketCalculator node and profiler config setting):

# MediaPipe graph that applies a face effect to the input video stream.

# GPU buffer. (GpuBuffer)
input_stream: "input_video"

# An integer, which indicate which effect is selected. (int)
#
# If `selected_effect_id` is `0`, the Axis effect is selected.
# If `selected_effect_id` is `1`, the Facepaint effect is selected.
# If `selected_effect_id` is `2`, the Glasses effect is selected.
#
# No other values are allowed for `selected_effect_id`.
input_stream: "selected_effect_id"

# Indicates whether to use the face detection as the input source. (bool)
#
# If `true`, the face detection pipeline will be used to produce landmarks.
# If `false`, the face landmark pipeline will be used to produce landmarks.
input_side_packet: "use_face_detection_input_source"

# Output image with rendered results. (GpuBuffer)
output_stream: "output_video"

# A list of geometry data for a single detected face.
#
# NOTE: there will not be an output packet in this stream for this particular
# timestamp if none of faces detected.
#
# (std::vector<face_geometry::FaceGeometry>)
output_stream: "multi_face_geometry"

# Throttles the images flowing downstream for flow control. It passes through
# the very first incoming image unaltered, and waits for downstream nodes
# (calculators and subgraphs) in the graph to finish their tasks before it
# passes through another image. All images that come in while waiting are
# dropped, limiting the number of in-flight images in most part of the graph to
# 1. This prevents the downstream nodes from queuing up incoming images and data
# excessively, which leads to increased latency and memory usage, unwanted in
# real-time mobile applications. It also eliminates unnecessarily computation,
# e.g., the output produced by a node may get dropped downstream if the
# subsequent nodes are still busy processing previous inputs.

node {
  calculator: "ConstantSidePacketCalculator"
  output_side_packet: "PACKET:use_face_detection_input_source"
  node_options: {
    [type.googleapis.com/mediapipe.ConstantSidePacketCalculatorOptions]: {
      packet { bool_value: false }
    }
  }
}
node {
  calculator: "FlowLimiterCalculator"
  input_stream: "input_video"
  input_stream: "FINISHED:output_video"
  input_stream_info: {
    tag_index: "FINISHED"
    back_edge: true
  }
  output_stream: "throttled_input_video"
}

# Generates an environment that describes the current virtual scene.
node {
  calculator: "FaceGeometryEnvGeneratorCalculator"
  output_side_packet: "ENVIRONMENT:environment"
  node_options: {
    [type.googleapis.com/mediapipe.FaceGeometryEnvGeneratorCalculatorOptions] {
      environment: {
        origin_point_location: TOP_LEFT_CORNER
        perspective_camera: {
          vertical_fov_degrees: 63.0  # 63 degrees
          near: 1.0  # 1cm
          far: 10000.0  # 100m
        }
      }
    }
  }
}

# Computes the face geometry for a single face. The input source is defined
# through `use_face_detection_input_source`.
node {
  calculator: "SwitchContainer"
  input_stream: "IMAGE:throttled_input_video"
  input_side_packet: "ENABLE:use_face_detection_input_source"
  input_side_packet: "ENVIRONMENT:environment"
  output_stream: "MULTI_FACE_GEOMETRY:multi_face_geometry"
  node_options: {
    [type.googleapis.com/mediapipe.SwitchContainerOptions] {
      contained_node: {
        calculator: "SingleFaceGeometryFromLandmarksGpu"
      }
      contained_node: {
        calculator: "SingleFaceGeometryFromDetectionGpu"
      }
    }
  }
}

# Renders the selected effect based on `selected_effect_id`.
node {
  calculator: "SwitchContainer"
  input_stream: "SELECT:selected_effect_id"
  input_stream: "IMAGE_GPU:throttled_input_video"
  input_stream: "MULTI_FACE_GEOMETRY:multi_face_geometry"
  input_side_packet: "ENVIRONMENT:environment"
  output_stream: "IMAGE_GPU:output_video"
  node_options: {
    [type.googleapis.com/mediapipe.SwitchContainerOptions] {
      contained_node: {
        calculator: "FaceGeometryEffectRendererCalculator"
        node_options: {
          [type.googleapis.com/mediapipe.FaceGeometryEffectRendererCalculatorOptions] {
            effect_texture_path: "mediapipe/graphs/face_effect/data/axis.pngblob"
            effect_mesh_3d_path: "mediapipe/graphs/face_effect/data/axis.binarypb"
          }
        }
      }
      contained_node: {
        calculator: "FaceGeometryEffectRendererCalculator"
        node_options: {
          [type.googleapis.com/mediapipe.FaceGeometryEffectRendererCalculatorOptions] {
            effect_texture_path: "mediapipe/graphs/face_effect/data/facepaint.pngblob"
          }
        }
      }
      contained_node: {
        calculator: "FaceGeometryEffectRendererCalculator"
        node_options: {
          [type.googleapis.com/mediapipe.FaceGeometryEffectRendererCalculatorOptions] {
            effect_texture_path: "mediapipe/graphs/face_effect/data/glasses.pngblob"
            effect_mesh_3d_path: "mediapipe/graphs/face_effect/data/glasses.binarypb"
          }
        }
      }
    }
  }
}

profiler_config {
  trace_enabled: true
  enable_profiler: true
  trace_log_count: 5
  trace_log_interval_count: 200
  histogram_interval_size_usec: 100
  enable_stream_latency: true
  trace_log_path: "/home/patroclos/Mediapipe/mediapipe/profiler/"
}

Error log is as below:

I20220805 23:54:53.075434 14695 demo_run_graph_main_gpu.cc:59] Initialize the calculator graph.
I20220805 23:54:53.083214 14695 demo_run_graph_main_gpu.cc:63] Initialize the GPU.
I20220805 23:54:53.093971 14695 gl_context_egl.cc:84] Successfully initialized EGL. Major : 1 Minor: 5
I20220805 23:54:53.177253 14704 gl_context.cc:335] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 465.19.01)
I20220805 23:54:53.177417 14695 demo_run_graph_main_gpu.cc:69] Initialize the camera or load the video.
[ WARN:[email protected]] OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
I20220805 23:54:55.176702 14695 demo_run_graph_main_gpu.cc:90] Start running the calculator graph.
I20220805 23:54:55.177738 14695 graph_profiler.cc:246] trace_log_path: /home/patroclos/Mediapipe/mediapipe/profiler/
I20220805 23:54:55.181089 14695 demo_run_graph_main_gpu.cc:95] Start grabbing and processing frames.
Segmentation fault (core dumped)

Finally this is the graph structure:

I don't know why I'm getting Segmentation fault error. Profiler doesn't give any useful information either(at least anything I understand). I appreciate any help. Thanks.

[kuaashish]

Hi @optimaltaskin, Have you looked at this similar closed issue #2170 and comment? Could you please follow the workaround mentioned on above comment. Thank you!

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler[bot]]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[optimaltaskin]

Hi @kuaashish Thanks for your reply. I have already checked that title and in mm mediapipe lib, the mentioned part of the code is as below:

#ifdef __ANDROID__
  cl::InferenceEnvironmentOptions env_options;
  if (!serialized_binary_cache_.empty()) {
    env_options.serialized_binary_cache = serialized_binary_cache_;
  }
  cl::InferenceEnvironmentProperties properties;
  MP_RETURN_IF_ERROR(
      cl::NewInferenceEnvironment(env_options, &cl_environment_, &properties));

  // Try to initialize from serialized model first.
  if (!serialized_model_.empty()) {
    absl::Status init_status = InitializeOpenCLFromSerializedModel(builder);
    if (init_status.ok()) {
      serialized_model_used_ = true;
      return absl::OkStatus();
    }
    VLOG(2) << "Failed to init from serialized model: [" << init_status
            << "]. Trying to init from scratch.";
  }

  // Initialize from scratch.
  cl::InferenceOptions cl_options = GetClInferenceOptions(options_);
  GraphFloat32 graph_cl;
  MP_RETURN_IF_ERROR(graph_cl_->MakeExactCopy(&graph_cl));
  MP_RETURN_IF_ERROR(cl_environment_->NewInferenceBuilder(
      cl_options, std::move(graph_cl), builder));

  return absl::OkStatus();
#else
  return mediapipe::UnimplementedError("Currently only Android is supported");
#endif  // __ANDROID__
}

#ifdef __ANDROID__

So to my understanding, the bug related with devices other than Android is fixed here already. However in my case, I still get that fault.

Thanks.