mediapipe

mediapipe copied to clipboard

Dear Support team.

I am trying to get the annotated video. But I am facing this issue. can you please help me what could be the reason?

GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/model_inference --calculator_graph_config_file=mediapipe/graphs/youtube8m/local_video_model_inference.pbtxt --input_side_packets=input_sequence_example_path=/tmp/mediapipe/features.pb ,input_video_path=video_example/file.mp4,output_video_path=video_example/annotated_video.mp4,segment_size=5,overlap=4

and error is

I20220719 11:48:15.735347 143462 simple_run_graph_main.cc:122] Initialize the calculator graph.
I20220719 11:48:15.738749 143462 simple_run_graph_main.cc:137] Start running the calculator graph.
E20220719 11:48:15.741351 143462 simple_run_graph_main.cc:149] Failed to run the graph: ValidateRequiredSidePackets failed to validate: 
; Side packet "input_video_path" is required but was not provided.
; Side packet "output_video_path" is required but was not provided.
; Side packet "overlap" is required but was not provided.
; Side packet "segment_size" is required but was not provided.

packets in use are Recent version git cloned mediapipe TF 1.15.4 GPU OpenCV 4.6.0.66 bazel release 5.2.0 Python 3.7.13 CENTOS 7.6 gcc version 12.1.0 (GCC)

Please make sure that this is a solution issue.

System information (Please provide as much relevant information as possible)

• Have I written custom code (as opposed to using a stock example script provided in Mediapipe):
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04, Android 11, iOS 14.4):
• MediaPipe version:
• Bazel version:
• Solution (e.g. FaceMesh, Pose, Holistic):
• Programming Language and version ( e.g. C++, Python, Java):

Describe the expected behavior:

Standalone code you may have used to try to get what you need :

If there is a problem, provide a reproducible test case that is the bare minimum necessary to generate the problem. If possible, please share a link to Colab/repo link /any notebook:

Other info / Complete Logs : Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached:

Hi @Tortoise17, It seems to be an issue with input_video_path, output_video_path, overlap and segment_size has not been passed in the correct input side packet.

Could you try to replace the input_side_packets=input_sequence_example_path=/tmp/mediapipe/features.pb toinput_side_packets='input_video_path=video_example/file.mp4,output_video_path=video_example/annotated_video.mp4,segment_size=5,overlap=4'.

Thank you!

@kuaashish Thank you. I have tried this and below is the result.

$ GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/model_inference --calculator_graph_config_file=mediapipe/graphs/youtube8m/local_video_model_inference.pbtxt  input_side_packets=input_video_path=video_example/file.mp4,output_video_path=video_example/annotated_video.mp4,segment_size=5,overlap=4

I20220720 16:00:05.297129 153636 simple_run_graph_main.cc:122] Initialize the calculator graph.
I20220720 16:00:05.301420 153636 simple_run_graph_main.cc:137] Start running the calculator graph.
E20220720 16:00:05.306913 153636 simple_run_graph_main.cc:149] Failed to run the graph: ValidateRequiredSidePackets failed to validate: 
; Side packet "input_sequence_example_path" is required but was not provided.
; Side packet "input_video_path" is required but was not provided.
; Side packet "output_video_path" is required but was not provided.
; Side packet "overlap" is required but was not provided.
; Side packet "segment_size" is required but was not provided.

Can you guess now my mistake?

Hi @Tortoise17, Could you please try with this below command:

GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/model_inference --calculator_graph_config_file=mediapipe/graphs/youtube8m/local_video_model_inference.pbtxt --input_side_packets='input_video_path=video_example/file.mp4,output_video_path=video_example/annotated_video.mp4,segment_size=5,overlap=4'

Thank you!

@kuaashish Thank you. now the error is :

I20220721 11:10:19.114539 164951 simple_run_graph_main.cc:122] Initialize the calculator graph.
I20220721 11:10:19.118959 164951 simple_run_graph_main.cc:137] Start running the calculator graph.
E20220721 11:10:19.121152 164951 simple_run_graph_main.cc:149] Failed to run the graph: ValidateRequiredSidePackets failed to validate: 
; Side packet "input_sequence_example_path" is required but was not provided.

Hi @Tortoise17, Now error is occurring it seems due to "input_sequence_example_path" has not been provided. Please try the below command with the parameter added and let us know if this resolves the issue:

GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/model_inference \
--calculator_graph_config_file=mediapipe/graphs/youtube8m/local_video_model_inference.pbtxt \
--input_side_packets=input_sequence_example_path=/tmp/mediapipe/features.pb,input_video_path=video_example/file.mp4,output_video_path=video_example/annotated_video.mp4,segment_size=5,overlap=4

Attaching relevant thread for your reference. Thank you!

@kuaashish thank you. I have tried this, Now the error is :

2022-07-21 12:08:17.034199: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:301] SavedModel load for tags { serve }; Status: fail: NOT_FOUND: Op type not registered 'Assert' in binary running on machine. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.. Took 237555 microseconds.
W20220721 12:08:17.045639 166536 opencv_video_decoder_calculator.cc:230] Not all the frames are decoded (total frames: 5133 vs decoded frames: 0).
E20220721 12:08:17.047278 166536 simple_run_graph_main.cc:149] Failed to run the graph: CalculatorGraph::Run() failed in Run: 
Calculator::Open() for node "LocalFileContentsCalculator" failed: ; Can't find file: /tmp/mediapipe/features.pb
Calculator::Open() for node "TensorFlowSessionFromSavedModelCalculator" failed: NOT_FOUND: Op type not registered 'Assert' in binary running on machine. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.

The point is that, I have tried to generate the features.pb file, but I guess that is not the the right way I used.

I tried commands below

bazel build -c opt --linkopt=-s \
      --define MEDIAPIPE_DISABLE_GPU=1 --define no_aws_support=true \
      mediapipe/examples/desktop/youtube8m:extract_yt8m_features

GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/extract_yt8m_features --calculator_graph_config_file=mediapipe/graphs/youtube8m/feature_extraction.pbtxt --input_side_packets=input_sequence_example=/tmp/mediapipe/metadata.pb --output_side_packets=output_sequence_example=/tmp/mediapipe/features.pb resulting in error below

2022-07-21 12:11:37.383117: E external/org_tensorflow/tensorflow/core/framework/op_kernel.cc:1676] OpKernel ('op: "AsString" device_type: "CPU"') for unknown op: AsString
I20220721 12:11:37.730674 166786 tensorflow_session_from_frozen_graph_calculator.cc:159] Loaded frozen model in: 852680 microseconds.
W20220721 12:11:37.737413 166777 opencv_video_decoder_calculator.cc:230] Not all the frames are decoded (total frames: 5133 vs decoded frames: 0).
E20220721 12:11:37.744156 166777 extract_yt8m_features.cc:132] Failed to run the graph: CalculatorGraph::Run() failed in Run: 
Calculator::Open() for node "TensorFlowSessionFromFrozenGraphCalculator_1" failed: ; RET_CHECK failure (mediapipe/calculators/tensorflow/tensorflow_session_from_frozen_graph_calculator.cc:144) tf_status.ok()Create failed: NOT_FOUND: Op type not registered 'DecodeJpeg' in binary running on machine. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.

If you can guess which library is making this failure. or which part of my sequence command? Your kind help is very big aid for me.

Hi @sureshdagooglecom, Could you please look into this issue. Thank you!

@sureshdagooglecom Please if you can have a look into the issue. I cannot build the desktop examples too. I don't know where is the proper requirement information guide which can help in the building of correct start.

@kuaashish any help? or guess?

Tortoise17, I assume you are following some of the instructions at: https://github.com/google/mediapipe/blob/master/mediapipe/examples/desktop/youtube8m/README.md I assume you are trying to run steps 1 through 5 to extract the features to "features.pb".
Did all of the steps 1 through 4 succeed before step 5 failed?

The error message seems to indicate that the tensorflow model was found and loaded by TensorFlowSessionFromSavedModelCalculator, but that the tensorflow model makes use of some operators that are not linked to the "extract_yt8m_features" binary for some reason. At least the tensorflow operator 'DecodeJpeg' seems to be not linked to the binary. It might be some problem with the binary build command, or maybe some changes to tensorflow of the mediapipe example targets that create a mismatch in the linked tensorflow operators.

If this seems to be the case, we will try these instructions ourselves and look for the disconnect.

Hi @Tortoise17, Could you please respond back to the above comment. Thank you!

@kuaashish The process failed at this stage

bazel build -c opt --linkopt=-s \
      --define MEDIAPIPE_DISABLE_GPU=1 --define no_aws_support=true \
      mediapipe/examples/desktop/youtube8m:extract_yt8m_features

At least the tensorflow operator 'DecodeJpeg' seems to be not linked to the binary. It might be some problem with the binary build command, or maybe some changes to tensorflow of the mediapipe example targets that create a mismatch in the linked tensorflow operators. to my every try, this is an issue and I tried all types of cuda version, all types of tensorflow versions. But the process fails at the stage above mentioned in my previous reply.

@hadon Dear Sir, is there any update on this issue.

Hello @Tortoise17, We are ending support for these MediaPipe Legacy Solutions, but upgrading the others. However, the libraries, documentation, and source code for all the MediapPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.

You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

Closing as stale. Please reopen if you'd like to work on this further.

Are you satisfied with the resolution of your issue? Yes No