mediapipe
mediapipe copied to clipboard
Published
20 hours ago •
google-ai-edge
google-ai-edge
Error with "Run the MediaPipe binary to extract the features."
I'm running now
bazel build -c opt --linkopt=-s \
--define MEDIAPIPE_DISABLE_GPU=1 --define no_aws_support=true \
mediapipe/examples/desktop/youtube8m:extract_yt8m_features
I've got followed error.
My configuration Ubuntu 18.04 python==3.7 Tensorflow==1.15 g++ --version==7.5.0
Error
DEBUG: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'com_google_absl' because it already exists.
DEBUG: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'com_google_protobuf' because it already exists.
DEBUG: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'com_google_googletest' because it already exists.
DEBUG: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'com_github_gflags_gflags' because it already exists.
DEBUG: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'rules_cc' because it already exists.
DEBUG: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'build_bazel_rules_apple' because it already exists.
DEBUG: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'build_bazel_rules_swift' because it already exists.
DEBUG: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'build_bazel_apple_support' because it already exists.
DEBUG: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'bazel_skylib' because it already exists.
DEBUG: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'pybind11' because it already exists.
WARNING: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/com_google_audio_tools/third_party/fft2d/BUILD:3:11: in linkstatic attribute of cc_library rule @com_google_audio_tools//third_party/fft2d:fft2d: setting 'linkstatic=1' is recommended if there are no object files
WARNING: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/org_tensorflow/tensorflow/core/lib/gtl/BUILD:129:11: in linkstatic attribute of cc_library rule @org_tensorflow//tensorflow/core/lib/gtl:map_util: setting 'linkstatic=1' is recommended if there are no object files. Since this rule was created by the macro 'cc_library', the error might have been caused by the macro implementation
WARNING: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/org_tensorflow/tensorflow/core/BUILD:1227:11: in linkstatic attribute of cc_library rule @org_tensorflow//tensorflow/core:lib_internal: setting 'linkstatic=1' is recommended if there are no object files. Since this rule was created by the macro 'cc_library', the error might have been caused by the macro implementation
WARNING: /home/members/yeoreum/.cache/bazel/_bazel_yeoreum/17e6889d84baa7b06a36a4e093846fd1/external/org_tensorflow/tensorflow/core/BUILD:1546:16: in linkstatic attribute of cc_library rule @org_tensorflow//tensorflow/core:framework_internal: setting 'linkstatic=1' is recommended if there are no object files. Since this rule was created by the macro 'tf_cuda_library', the error might have been caused by the macro implementation
INFO: Analyzed target //mediapipe/examples/desktop/youtube8m:extract_yt8m_features (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /home/members/yeoreum/smartssd/project_HNSW/youtube-8m/mediapipe/mediapipe/calculators/tensorflow/BUILD:697:11: C++ compilation of rule '//mediapipe/calculators/tensorflow:unpack_media_sequence_calculator' failed (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 75 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 75 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox
mediapipe/calculators/tensorflow/unpack_media_sequence_calculator.cc: In member function 'virtual absl::lts_2020_09_23::Status mediapipe::UnpackMediaSequenceCalculator::Open(mediapipe::CalculatorContext*)':
mediapipe/calculators/tensorflow/unpack_media_sequence_calculator.cc:249:48: error: 'const class mediapipe::UnpackMediaSequenceCalculatorOptions' has no member named 'keypoint_names'
keypoint_names_ = absl::StrSplit(options.keypoint_names(), ',');
^~~~~~~~~~~~~~
mediapipe/calculators/tensorflow/unpack_media_sequence_calculator.cc:250:44: error: 'const class mediapipe::UnpackMediaSequenceCalculatorOptions' has no member named 'default_keypoint_location'
default_keypoint_location_ = options.default_keypoint_location();
^~~~~~~~~~~~~~~~~~~~~~~~~
Target //mediapipe/examples/desktop/youtube8m:extract_yt8m_features failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 9.472s, Critical Path: 8.68s
INFO: 278 processes: 128 internal, 150 linux-sandbox.
FAILED: Build did NOT complete successfully
Is there any suggestion?
We are aware of this issue. See the temporary solution at https://github.com/google/mediapipe/issues/1871#issuecomment-819658414.
Thank you @jiuqiant .
However after switch the branch, I've got another error.
Error
ERROR: /home/members/yeoreum/smartssd/project_HNSW/youtube-8m/reverted/mediapipe/mediapipe/examples/desktop/youtube8m/BUILD:17:10: Linking mediapipe/examples/desktop/youtube8m/extract_yt8m_features failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc @bazel-out/k8-opt/bin/mediapipe/examples/desktop/youtube8m/extract_yt8m_features-2.params
Use --sandbox_debug to see verbose messages from the sandbox gcc failed: error executing command /usr/bin/gcc @bazel-out/k8-opt/bin/mediapipe/examples/desktop/youtube8m/extract_yt8m_features-2.params
Use --sandbox_debug to see verbose messages from the sandbox
bazel-out/k8-opt/bin/mediapipe/calculators/tensorflow/_objs/pack_media_sequence_calculator/pack_media_sequence_calculator.o:pack_media_sequence_calculator.cc:function mediapipe::PackMediaSequenceCalculator::Process(mediapipe::CalculatorContext*): error: undefined reference to 'cv::String::allocate(unsigned long)'
bazel-out/k8-opt/bin/mediapipe/calculators/tensorflow/_objs/pack_media_sequence_calculator/pack_media_sequence_calculator.o:pack_media_sequence_calculator.cc:function mediapipe::PackMediaSequenceCalculator::Process(mediapipe::CalculatorContext*): error: undefined reference to 'cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
bazel-out/k8-opt/bin/mediapipe/calculators/tensorflow/_objs/pack_media_sequence_calculator/pack_media_sequence_calculator.o:pack_media_sequence_calculator.cc:function mediapipe::PackMediaSequenceCalculator::Process(mediapipe::CalculatorContext*): error: undefined reference to 'cv::String::deallocate()'
bazel-out/k8-opt/bin/mediapipe/calculators/tensorflow/_objs/pack_media_sequence_calculator/pack_media_sequence_calculator.o:pack_media_sequence_calculator.cc:function mediapipe::PackMediaSequenceCalculator::Process(mediapipe::CalculatorContext*): error: undefined reference to 'cv::Mat::~Mat()'
bazel-out/k8-opt/bin/mediapipe/calculators/tensorflow/_objs/pack_media_sequence_calculator/pack_media_sequence_calculator.o:pack_media_sequence_calculator.cc:function mediapipe::PackMediaSequenceCalculator::Process(mediapipe::CalculatorContext*): error: undefined reference to 'cv::Mat::~Mat()'
bazel-out/k8-opt/bin/mediapipe/calculators/tensorflow/_objs/pack_media_sequence_calculator/pack_media_sequence_calculator.o:pack_media_sequence_calculator.cc:function mediapipe::PackMediaSequenceCalculator::Process(mediapipe::CalculatorContext*): error: undefined reference to 'cv::Mat::~Mat()'
bazel-out/k8-opt/bin/mediapipe/calculators/tensorflow/_objs/pack_media_sequence_calculator/pack_media_sequence_calculator.o:pack_media_sequence_calculator.cc:function mediapipe::PackMediaSequenceCalculator::Process(mediapipe::CalculatorContext*): error: undefined reference to 'cv::String::deallocate()'
bazel-out/k8-opt/bin/mediapipe/util/sequence/_objs/media_sequence/media_sequence.o:media_sequence.cc:function mediapipe::mediasequence::(anonymous namespace)::ReconcileMetadataImages(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tensorflow::SequenceExample*): error: undefined reference to 'cv::Mat::Mat(int, int, int, void*, unsigned long)'
bazel-out/k8-opt/bin/mediapipe/util/sequence/_objs/media_sequence/media_sequence.o:media_sequence.cc:function mediapipe::mediasequence::(anonymous namespace)::ReconcileMetadataImages(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tensorflow::SequenceExample*): error: undefined reference to 'cv::imdecode(cv::_InputArray const&, int)'
bazel-out/k8-opt/bin/mediapipe/util/sequence/_objs/media_sequence/media_sequence.o:media_sequence.cc:function mediapipe::mediasequence::(anonymous namespace)::ReconcileMetadataImages(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tensorflow::SequenceExample*): error: undefined reference to 'cv::Mat::~Mat()'
bazel-out/k8-opt/bin/mediapipe/framework/formats/_objs/location/location.o:location.cc:function mediapipe::Location::GetCvMask() const: error: undefined reference to 'cv::Mat::Mat(int, int, int, cv::Scalar_<double> const&)'
bazel-out/k8-opt/bin/mediapipe/calculators/video/_objs/opencv_video_decoder_calculator/opencv_video_decoder_calculator.o:opencv_video_decoder_calculator.cc:function mediapipe::OpenCvVideoDecoderCalculator::Process(mediapipe::CalculatorContext*): error: undefined reference to 'cv::Mat::Mat()'
bazel-out/k8-opt/bin/mediapipe/calculators/video/_objs/opencv_video_decoder_calculator/opencv_video_decoder_calculator.o:opencv_video_decoder_calculator.cc:function mediapipe::OpenCvVideoDecoderCalculator::Process(mediapipe::CalculatorContext*): error: undefined reference to 'cv::Mat::empty() const'
bazel-out/k8-opt/bin/mediapipe/calculators/video/_objs/opencv_video_decoder_calculator/opencv_video_decoder_calculator.o:opencv_video_decoder_calculator.cc:function mediapipe::OpenCvVideoDecoderCalculator::Process(mediapipe::CalculatorContext*): error: undefined reference to 'cv::Mat::empty() const'
bazel-out/k8-opt/bin/mediapipe/calculators/video/_objs/opencv_video_decoder_calculator/opencv_video_decoder_calculator.o:opencv_video_decoder_calculator.cc:function mediapipe::OpenCvVideoDecoderCalculator::Process(mediapipe::CalculatorContext*): error: undefined reference to 'cv::cvtColor(cv::_InputArray const&, cv::_OutputArray const&, int, int)'
bazel-out/k8-opt/bin/mediapipe/calculators/video/_objs/opencv_video_decoder_calculator/opencv_video_decoder_calculator.o:opencv_video_decoder_calculator.cc:function mediapipe::OpenCvVideoDecoderCalculator::Process(mediapipe::CalculatorContext*): error: undefined reference to 'cv::cvtColor(cv::_InputArray const&, cv::_OutputArray const&, int, int)'
bazel-out/k8-opt/bin/mediapipe/calculators/video/_objs/opencv_video_decoder_calculator/opencv_video_decoder_calculator.o:opencv_video_decoder_calculator.cc:function mediapipe::OpenCvVideoDecoderCalculator::Open(mediapipe::CalculatorContext*): error: undefined reference to 'cv::String::allocate(unsigned long)'
bazel-out/k8-opt/bin/mediapipe/calculators/video/_objs/opencv_video_decoder_calculator/opencv_video_decoder_calculator.o:opencv_video_decoder_calculator.cc:function mediapipe::OpenCvVideoDecoderCalculator::Open(mediapipe::CalculatorContext*): error: undefined reference to 'cv::VideoCapture::VideoCapture(cv::String const&)'
bazel-out/k8-opt/bin/mediapipe/calculators/video/_objs/opencv_video_decoder_calculator/opencv_video_decoder_calculator.o:opencv_video_decoder_calculator.cc:function mediapipe::OpenCvVideoDecoderCalculator::Open(mediapipe::CalculatorContext*): error: undefined reference to 'cv::String::deallocate()'
bazel-out/k8-opt/bin/mediapipe/calculators/video/_objs/opencv_video_decoder_calculator/opencv_video_decoder_calculator.o:opencv_video_decoder_calculator.cc:function mediapipe::OpenCvVideoDecoderCalculator::Open(mediapipe::CalculatorContext*): error: undefined reference to 'cv::Mat::Mat()'
bazel-out/k8-opt/bin/mediapipe/calculators/video/_objs/opencv_video_decoder_calculator/opencv_video_decoder_calculator.o:opencv_video_decoder_calculator.cc:function mediapipe::OpenCvVideoDecoderCalculator::Open(mediapipe::CalculatorContext*): error: undefined reference to 'cv::Mat::empty() const'
bazel-out/k8-opt/bin/mediapipe/calculators/video/_objs/opencv_video_decoder_calculator/opencv_video_decoder_calculator.o:opencv_video_decoder_calculator.cc:function mediapipe::OpenCvVideoDecoderCalculator::Open(mediapipe::CalculatorContext*): warning: the use of `tmpnam' is dangerous, better use `mkstemp'
bazel-out/k8-opt/bin/mediapipe/calculators/video/_objs/opencv_video_decoder_calculator/opencv_video_decoder_calculator.o:opencv_video_decoder_calculator.cc:function mediapipe::OpenCvVideoDecoderCalculator::Open(mediapipe::CalculatorContext*): error: undefined reference to 'cv::String::deallocate()'
bazel-out/k8-opt/bin/mediapipe/framework/formats/_objs/image_frame_opencv/image_frame_opencv.o:image_frame_opencv.cc:function mediapipe::formats::MatView(mediapipe::ImageFrame const*): error: undefined reference to 'cv::Mat::Mat(int, int const*, int, void*, unsigned long const*)'
collect2: error: ld returned 1 exit status
Target //mediapipe/examples/desktop/youtube8m:extract_yt8m_features failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 719.750s, Critical Path: 455.73s
INFO: 3654 processes: 64 internal, 3590 linux-sandbox.
FAILED: Build did NOT complete successfully
Do you have any suggestion?
Your MediaPipe OpenCV config might be incorrect. See https://github.com/google/mediapipe/blob/master/docs/getting_started/troubleshooting.md#incorrect-mediapipe-opencv-config.
@jiuqiant Thank you so much.
I changed configuration for WORKSPACE, opencv_linux.BUILD, and ffmpeg_linux.BUILD then it works well.
@jiuqiant Thank you for last comments.
After troubleshooting I'm now running
GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/extract_yt8m_features \
--calculator_graph_config_file=mediapipe/graphs/youtube8m/feature_extraction.pbtxt \
--input_side_packets=input_sequence_example=/tmp/mediapipe/metadata.pb \
--output_side_packets=output_sequence_example=/tmp/mediapipe/features.pb
for feature extraction.
However I got another error.
Error
I20210510 15:22:15.947022 23400 extract_yt8m_features.cc:104] Initialize the calculator graph.
I20210510 15:22:15.948565 23400 extract_yt8m_features.cc:107] Start running the calculator graph.
I20210510 15:22:15.952020 23401 unpack_media_sequence_calculator.cc:302] Created AudioDecoderOptions:
audio_stream {
stream_index: 0
}
start_time: 0
end_time: 566
I20210510 15:22:15.952102 23401 unpack_media_sequence_calculator.cc:324] Created PacketResamplerOptions:
[mediapipe.PacketResamplerCalculatorOptions.ext] {
frame_rate: 1
base_timestamp: 0
start_time: 0
end_time: 566000000
}
2021-05-10 15:22:15.952197: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE3 SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
I20210510 15:22:15.973664 23406 audio_decoder.cc:646] Created audio processor 0x7f0460022960 for file "/home/members/yeoreum/test.mp4"
I20210510 15:22:15.974036 23406 audio_decoder.cc:374] Opened audio stream (id: 1, channels: 2, sample rate: 44100, time base: 1/44100).
I20210510 15:22:16.316346 23402 tensorflow_session_from_frozen_graph_calculator.cc:152] Loaded frozen model in: 364327 microseconds.
I20210510 15:22:17.058359 23404 tensorflow_session_from_frozen_graph_calculator.cc:152] Loaded frozen model in: 1106248 microseconds.
2021-05-10 15:22:17.242354: I external/org_tensorflow/tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2900000000 Hz
2021-05-10 15:22:18.597322: W external/org_tensorflow/tensorflow/core/framework/op_def_util.cc:371] Op BatchNormWithGlobalNormalization is deprecated. It will cease to work in GraphDef version 9. Use tf.nn.batch_normalization().
W20210510 15:22:28.884815 23413 calculator_graph.cc:1146] Resolved a deadlock by increasing max_queue_size of input stream: pca_inception3_vf to: 101. Consider increasing max_queue_size for better performance.
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
./extract_features_1.sh: line 4: 23400 Aborted (core dumped) GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/extract_yt8m_features --calculator_graph_config_file=mediapipe/graphs/youtube8m/feature_extraction.pbtxt --input_side_packets=input_sequence_example=/tmp/mediapipe/metadata.pb --output_side_packets=output_sequence_example=/tmp/mediapipe/features.pb
I saw other bad_alloc error issues,
- There is no generated "feature.pb" at /tmp/mediapipe
- "feature_extraction.pbtxt" doesn't have "max_queue_size" declaration.
Can you give me some suggestion?
Thank you.
You can add max_queue_size to the end of feature_extraction.pbtxt and adjust the value (higher for faster and lower for safer) accordingly.
diff --git a/mediapipe/graphs/youtube8m/feature_extraction.pbtxt b/mediapipe/graphs/youtube8m/feature_extraction.pbtx
t
index 89d1053..95e36b7 100644
--- a/mediapipe/graphs/youtube8m/feature_extraction.pbtxt
+++ b/mediapipe/graphs/youtube8m/feature_extraction.pbtxt
@@ -293,3 +293,4 @@ node {
output_side_packet: "STRING:output_sequence_example"
}
+max_queue_size: 10
@jiuqiant Thank you for your comment.
I could adjust the value by adding "max_queue_size" to the end of feature_extraction.pbtxt. I tried various value of "max_queue_size", but I got similar error. I think packets cannot be processed so "bad_alloc" error occurs.
When max_queue_size is set to be 32, log
2021-05-11 15:30:41.850452: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE3 SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
I20210511 15:30:41.871831 46380 audio_decoder.cc:646] Created audio processor 0x7f6a5c022960 for file "/home/members/yeoreum/test.mp4"
I20210511 15:30:41.872201 46380 audio_decoder.cc:374] Opened audio stream (id: 1, channels: 2, sample rate: 44100, time base: 1/44100).
I20210511 15:30:42.243535 46375 tensorflow_session_from_frozen_graph_calculator.cc:152] Loaded frozen model in: 393239 microseconds.
I20210511 15:30:43.013437 46378 tensorflow_session_from_frozen_graph_calculator.cc:152] Loaded frozen model in: 1163059 microseconds.
2021-05-11 15:30:43.186479: I external/org_tensorflow/tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2900000000 Hz
2021-05-11 15:30:44.635558: W external/org_tensorflow/tensorflow/core/framework/op_def_util.cc:371] Op BatchNormWithGlobalNormalization is deprecated. It will cease to work in Graph Def version 9. Use tf.nn.batch_normalization().
W20210511 15:30:49.015435 46403 calculator_graph.cc:1146] Resolved a deadlock by increasing max_queue_size of input stream: pca_inception3_vf to: 41. Consider increasing max_queue_s ize for better performance.
W20210511 15:30:57.717023 46375 calculator_graph.cc:1146] Resolved a deadlock by increasing max_queue_size of input stream: log_mel_spectrum_magnitude_with_context to: 83. Consider increasing max_queue_size for better performance.
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
./extract_features_1.sh: line 4: 46373 Aborted (core dumped) GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/extract_yt8m_features --calculator_gra ph_config_file=mediapipe/graphs/youtube8m/feature_extraction.pbtxt --input_side_packets=input_sequence_example=/tmp/mediapipe/metadata.pb --output_side_packets=output_sequence_examp le=/tmp/mediapipe/features.pb
I check Out of Memory Error section, but I couldn't figure out what's wrong.
Can you help me?
Thank you.
@jiuqiant Thank you for your comment.
I could adjust the value by adding "max_queue_size" to the end of feature_extraction.pbtxt. I tried various value of "max_queue_size", but I got similar error. I think packets cannot be processed so "bad_alloc" error occurs.
When max_queue_size is set to be 32, log
2021-05-11 15:30:41.850452: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE3 SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. I20210511 15:30:41.871831 46380 audio_decoder.cc:646] Created audio processor 0x7f6a5c022960 for file "/home/members/yeoreum/test.mp4" I20210511 15:30:41.872201 46380 audio_decoder.cc:374] Opened audio stream (id: 1, channels: 2, sample rate: 44100, time base: 1/44100). I20210511 15:30:42.243535 46375 tensorflow_session_from_frozen_graph_calculator.cc:152] Loaded frozen model in: 393239 microseconds. I20210511 15:30:43.013437 46378 tensorflow_session_from_frozen_graph_calculator.cc:152] Loaded frozen model in: 1163059 microseconds. 2021-05-11 15:30:43.186479: I external/org_tensorflow/tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2900000000 Hz 2021-05-11 15:30:44.635558: W external/org_tensorflow/tensorflow/core/framework/op_def_util.cc:371] Op BatchNormWithGlobalNormalization is deprecated. It will cease to work in Graph Def version 9. Use tf.nn.batch_normalization(). W20210511 15:30:49.015435 46403 calculator_graph.cc:1146] Resolved a deadlock by increasing max_queue_size of input stream: pca_inception3_vf to: 41. Consider increasing max_queue_s ize for better performance. W20210511 15:30:57.717023 46375 calculator_graph.cc:1146] Resolved a deadlock by increasing max_queue_size of input stream: log_mel_spectrum_magnitude_with_context to: 83. Consider increasing max_queue_size for better performance. terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc ./extract_features_1.sh: line 4: 46373 Aborted (core dumped) GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/extract_yt8m_features --calculator_gra ph_config_file=mediapipe/graphs/youtube8m/feature_extraction.pbtxt --input_side_packets=input_sequence_example=/tmp/mediapipe/metadata.pb --output_side_packets=output_sequence_examp le=/tmp/mediapipe/features.pbI check Out of Memory Error section, but I couldn't figure out what's wrong.
Can you help me?
Thank you.
What's your system RAM size? Can you check how much feature_extraction use? And how about setting max_queue_size to 2?
Thank you for your comment @jiuqiant .
System RAM size MemTotal: 196673552 kB MemFree: 51997052 kB MemAvailable: 87010260 kB
And when I monitoring the ram usage running featrue_extraction, it used 1~2%. I tried max_queue_size to 2, I've got following logs.
Log
2021-05-12 07:27:24.901283: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE3 SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
I20210512 07:27:24.925179 4142 audio_decoder.cc:646] Created audio processor 0x7f7688022960 for file "/home/members/yeoreum/test.mp4"
I20210512 07:27:24.925747 4142 audio_decoder.cc:374] Opened audio stream (id: 1, channels: 2, sample rate: 44100, time base: 1/44100).
I20210512 07:27:25.288141 4139 tensorflow_session_from_frozen_graph_calculator.cc:152] Loaded frozen model in: 387062 microseconds.
I20210512 07:27:26.039988 4138 tensorflow_session_from_frozen_graph_calculator.cc:152] Loaded frozen model in: 1138799 microseconds.
2021-05-12 07:27:26.230599: I external/org_tensorflow/tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2900000000 Hz
2021-05-12 07:27:27.469195: W external/org_tensorflow/tensorflow/core/framework/op_def_util.cc:371] Op BatchNormWithGlobalNormalization is deprecated. It will cease to work in GraphDef version 9. Use tf.nn.batch_normalization().
W20210512 07:27:27.858343 4156 calculator_graph.cc:1146] Resolved a deadlock by increasing max_queue_size of input stream: pca_inception3_vf to: 4. Consider increasing max_queue_size for better performance.
W20210512 07:27:35.814270 4159 calculator_graph.cc:1146] Resolved a deadlock by increasing max_queue_size of input stream: pca_inception3_vf to: 60. Consider increasing max_queue_size for better performance.
W20210512 07:27:44.398288 4154 calculator_graph.cc:1146] Resolved a deadlock by increasing max_queue_size of input stream: pca_inception3_vf to: 119. Consider increasing max_queue_size for better performance.
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
./extract_features_1.sh: line 4: 4117 Aborted (core dumped) GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/extract_yt8m_features --calculator_graph_config_file=mediapipe/graphs/youtube8m/feature_extraction.pbtxt --input_side_packets=input_sequence_example=/tmp/mediapipe/metadata.pb --output_side_packets=output_sequence_example=/tmp/mediapipe/features.pb
According to memory usage, I cannot understand why bad_alloc occurs..
metadata.pb
Above feature is my metadata.pb. Its size is only 4KB.
Do you think it could be the problem?
Hi @limetus,
This issue is fixed in the latest release of mediapipe 0.8.11, Could you please let us know if this is still an issue at your end. Thank you!
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.
![google-ml-butler[bot] avatar](https://avatars.githubusercontent.com/in/43729?v=4 "Published by a pub.dev verified publisher"){kind=small}
Closing as stale. Please reopen if you'd like to work on this further.