openpose icon indicating copy to clipboard operation
openpose copied to clipboard
verified publisher icon CMU-Perceptual-Computing-Lab

OOM on a GTX 1060 with 6Gb of VRAM on CUDA 11.1 and cuDNN 8.04 on Ubuntu

Open kognat-docs opened this issue 4 years ago • 10 comments

Issue Summary

Fresh build from the main branch is OOMing , yet almost 6Gb of VRAM is available

discobot@mirrorball:~/Documents/github/openpose$ ./build/examples/openpose/openpose.bin --camera 0
Starting OpenPose demo...
Configuring OpenPose...
Starting thread(s)...
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Auto-detecting all available GPUs... Detected 1 GPU(s), using 1 of them starting at GPU 0.
F0904 16:51:04.484730 32478 syncedmem.cpp:71] Check failed: error == cudaSuccess (2 vs. 0)  out of memory
*** Check failure stack trace: ***
    @     0x7fb5c6d4c1c3  google::LogMessage::Fail()
    @     0x7fb5c6d5125b  google::LogMessage::SendToLog()
    @     0x7fb5c6d4bebf  google::LogMessage::Flush()
    @     0x7fb5c6d4c6ef  google::LogMessageFatal::~LogMessageFatal()
    @     0x7fb5c6a8ce2a  caffe::SyncedMemory::mutable_gpu_data()
    @     0x7fb5c69066a6  caffe::Blob<>::mutable_gpu_data()
    @     0x7fb5c6ac79ee  caffe::CuDNNConvolutionLayer<>::Forward_gpu()
    @     0x7fb5c6a4cb62  caffe::Net<>::ForwardFromTo()
    @     0x7fb5c7403364  op::NetCaffe::forwardPass()
    @     0x7fb5c74204aa  op::PoseExtractorCaffe::forwardPass()
    @     0x7fb5c741b52b  op::PoseExtractor::forwardPass()
    @     0x7fb5c7418fc8  op::WPoseExtractor<>::work()
    @     0x7fb5c744a2cf  op::Worker<>::checkAndWork()
    @     0x7fb5c744a45b  op::SubThread<>::workTWorkers()
    @     0x7fb5c745812d  op::SubThreadQueueInOut<>::work()
    @     0x7fb5c744f131  op::Thread<>::threadFunction()
    @     0x7fb5c708bde4  (unknown)
    @     0x7fb5c6d8c609  start_thread
    @     0x7fb5c6eca293  clone
Aborted (core dumped)

Memory

discobot@mirrorball:~/Documents/github/openpose$ nvidia-smi 
Sat Sep  4 16:51:11 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.84       Driver Version: 460.84       CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 106...  Off  | 00000000:0F:00.0  On |                  N/A |
|  0%   49C    P0    27W / 120W |    118MiB /  6077MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       965      G   /usr/lib/xorg/Xorg                 84MiB |
|    0   N/A  N/A      1285      G   /usr/bin/gnome-shell               30MiB |
+-----------------------------------------------------------------------------+

No missing libraries

discobot@mirrorball:~/Documents/github/openpose$ ldd ./build/examples/openpose/openpose.bin
	linux-vdso.so.1 (0x00007ffef01eb000)
	libopenpose.so.1.7.0 => /home/discobot/Documents/github/openpose/build/src/openpose/libopenpose.so.1.7.0 (0x00007faab0c71000)
	libgflags.so.2.2 => /lib/x86_64-linux-gnu/libgflags.so.2.2 (0x00007faab0c27000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007faab0a45000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007faab0a2a000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007faab0838000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007faab0813000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007faab080d000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007faab0802000)
	libglog.so.0 => /lib/x86_64-linux-gnu/libglog.so.0 (0x00007faab07cf000)
	libcaffe.so.1.0.0 => /home/discobot/Documents/github/openpose/build/caffe/lib/libcaffe.so.1.0.0 (0x00007faab0254000)
	libopencv_highgui.so.4.2 => /lib/x86_64-linux-gnu/libopencv_highgui.so.4.2 (0x00007faab023a000)
	libopencv_video.so.4.2 => /lib/x86_64-linux-gnu/libopencv_video.so.4.2 (0x00007faab01be000)
	libopencv_videoio.so.4.2 => /lib/x86_64-linux-gnu/libopencv_videoio.so.4.2 (0x00007faab013d000)
	libopencv_imgcodecs.so.4.2 => /lib/x86_64-linux-gnu/libopencv_imgcodecs.so.4.2 (0x00007faab00e4000)
	libopencv_objdetect.so.4.2 => /lib/x86_64-linux-gnu/libopencv_objdetect.so.4.2 (0x00007faab007e000)
	libopencv_calib3d.so.4.2 => /lib/x86_64-linux-gnu/libopencv_calib3d.so.4.2 (0x00007faaafeab000)
	libopencv_imgproc.so.4.2 => /lib/x86_64-linux-gnu/libopencv_imgproc.so.4.2 (0x00007faaaf904000)
	libopencv_core.so.4.2 => /lib/x86_64-linux-gnu/libopencv_core.so.4.2 (0x00007faaaf552000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007faaaf403000)
	/lib64/ld-linux-x86-64.so.2 (0x00007faab1137000)
	libunwind.so.8 => /lib/x86_64-linux-gnu/libunwind.so.8 (0x00007faaaf3e6000)
	libboost_thread.so.1.71.0 => /lib/x86_64-linux-gnu/libboost_thread.so.1.71.0 (0x00007faaaf3ba000)
	libboost_filesystem.so.1.71.0 => /lib/x86_64-linux-gnu/libboost_filesystem.so.1.71.0 (0x00007faaaf39c000)
	libprotobuf.so.17 => /lib/x86_64-linux-gnu/libprotobuf.so.17 (0x00007faaaf0a2000)
	libhdf5_serial.so.103 => /lib/x86_64-linux-gnu/libhdf5_serial.so.103 (0x00007faaaed25000)
	libhdf5_serial_hl.so.100 => /lib/x86_64-linux-gnu/libhdf5_serial_hl.so.100 (0x00007faaaecfe000)
	libcudart.so.11.0 => /usr/local/cuda/lib64/libcudart.so.11.0 (0x00007faaaea79000)
	libcurand.so.10 => /usr/local/cuda/lib64/libcurand.so.10 (0x00007faaa9b49000)
	libcublas.so.11 => /usr/local/cuda/lib64/libcublas.so.11 (0x00007faaa172d000)
	libcudnn.so.8 => /usr/local/cuda/lib64/libcudnn.so.8 (0x00007faaa1502000)
	libcblas.so.3 => /lib/x86_64-linux-gnu/libcblas.so.3 (0x00007faaa14dd000)
	libgtk-3.so.0 => /lib/x86_64-linux-gnu/libgtk-3.so.0 (0x00007faaa0d2c000)
	libgdk-3.so.0 => /lib/x86_64-linux-gnu/libgdk-3.so.0 (0x00007faaa0c27000)
	libcairo.so.2 => /lib/x86_64-linux-gnu/libcairo.so.2 (0x00007faaa0b04000)
	libgdk_pixbuf-2.0.so.0 => /lib/x86_64-linux-gnu/libgdk_pixbuf-2.0.so.0 (0x00007faaa0adc000)
	libgobject-2.0.so.0 => /lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007faaa0a7a000)
	libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007faaa0951000)
	libdc1394.so.22 => /lib/x86_64-linux-gnu/libdc1394.so.22 (0x00007faaa08d9000)
	libgstreamer-1.0.so.0 => /lib/x86_64-linux-gnu/libgstreamer-1.0.so.0 (0x00007faaa0792000)
	libgstapp-1.0.so.0 => /lib/x86_64-linux-gnu/libgstapp-1.0.so.0 (0x00007faaa0780000)
	libgstriff-1.0.so.0 => /lib/x86_64-linux-gnu/libgstriff-1.0.so.0 (0x00007faaa076f000)
	libgstpbutils-1.0.so.0 => /lib/x86_64-linux-gnu/libgstpbutils-1.0.so.0 (0x00007faaa072f000)
	libavcodec.so.58 => /lib/x86_64-linux-gnu/libavcodec.so.58 (0x00007faa9f172000)
	libavformat.so.58 => /lib/x86_64-linux-gnu/libavformat.so.58 (0x00007faa9eefa000)
	libavutil.so.56 => /lib/x86_64-linux-gnu/libavutil.so.56 (0x00007faa9edd1000)
	libswscale.so.5 => /lib/x86_64-linux-gnu/libswscale.so.5 (0x00007faa9ed3c000)
	libjpeg.so.8 => /lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007faa9ecb7000)
	libwebp.so.6 => /lib/x86_64-linux-gnu/libwebp.so.6 (0x00007faa9ec4b000)
	libpng16.so.16 => /lib/x86_64-linux-gnu/libpng16.so.16 (0x00007faa9ec13000)
	libgdcmMSFF.so.3.0 => /lib/x86_64-linux-gnu/libgdcmMSFF.so.3.0 (0x00007faa9e96d000)
	libtiff.so.5 => /lib/x86_64-linux-gnu/libtiff.so.5 (0x00007faa9e8ec000)
	libIlmImf-2_3.so.24 => /lib/x86_64-linux-gnu/libIlmImf-2_3.so.24 (0x00007faa9e60d000)
	libgdal.so.26 => /lib/libgdal.so.26 (0x00007faa9d286000)
	libgdcmDSED.so.3.0 => /lib/x86_64-linux-gnu/libgdcmDSED.so.3.0 (0x00007faa9d152000)
	libopencv_features2d.so.4.2 => /lib/x86_64-linux-gnu/libopencv_features2d.so.4.2 (0x00007faa9d07e000)
	libopencv_flann.so.4.2 => /lib/x86_64-linux-gnu/libopencv_flann.so.4.2 (0x00007faa9d01f000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007faa9d003000)
	libtbb.so.2 => /lib/x86_64-linux-gnu/libtbb.so.2 (0x00007faa9cfbd000)
	liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007faa9cf92000)
	libsz.so.2 => /lib/x86_64-linux-gnu/libsz.so.2 (0x00007faa9cf8d000)
	libcublasLt.so.11 => /usr/local/cuda/lib64/libcublasLt.so.11 (0x00007faa8ef99000)
	libatlas.so.3 => /lib/x86_64-linux-gnu/libatlas.so.3 (0x00007faa8ec0d000)
	libgmodule-2.0.so.0 => /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007faa8ec07000)
	libpangocairo-1.0.so.0 => /lib/x86_64-linux-gnu/libpangocairo-1.0.so.0 (0x00007faa8ebf3000)
	libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007faa8eab6000)
	libXi.so.6 => /lib/x86_64-linux-gnu/libXi.so.6 (0x00007faa8eaa4000)
	libXfixes.so.3 => /lib/x86_64-linux-gnu/libXfixes.so.3 (0x00007faa8ea9c000)
	libcairo-gobject.so.2 => /lib/x86_64-linux-gnu/libcairo-gobject.so.2 (0x00007faa8ea90000)
	libatk-1.0.so.0 => /lib/x86_64-linux-gnu/libatk-1.0.so.0 (0x00007faa8ea66000)
	libatk-bridge-2.0.so.0 => /lib/x86_64-linux-gnu/libatk-bridge-2.0.so.0 (0x00007faa8ea2d000)
	libepoxy.so.0 => /lib/x86_64-linux-gnu/libepoxy.so.0 (0x00007faa8e8fa000)
	libfribidi.so.0 => /lib/x86_64-linux-gnu/libfribidi.so.0 (0x00007faa8e8dd000)
	libgio-2.0.so.0 => /lib/x86_64-linux-gnu/libgio-2.0.so.0 (0x00007faa8e6fc000)
	libpangoft2-1.0.so.0 => /lib/x86_64-linux-gnu/libpangoft2-1.0.so.0 (0x00007faa8e6e3000)
	libpango-1.0.so.0 => /lib/x86_64-linux-gnu/libpango-1.0.so.0 (0x00007faa8e694000)
	libharfbuzz.so.0 => /lib/x86_64-linux-gnu/libharfbuzz.so.0 (0x00007faa8e58d000)
	libfontconfig.so.1 => /lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007faa8e546000)
	libfreetype.so.6 => /lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007faa8e487000)
	libXinerama.so.1 => /lib/x86_64-linux-gnu/libXinerama.so.1 (0x00007faa8e482000)
	libXrandr.so.2 => /lib/x86_64-linux-gnu/libXrandr.so.2 (0x00007faa8e475000)
	libXcursor.so.1 => /lib/x86_64-linux-gnu/libXcursor.so.1 (0x00007faa8e466000)
	libXcomposite.so.1 => /lib/x86_64-linux-gnu/libXcomposite.so.1 (0x00007faa8e461000)
	libXdamage.so.1 => /lib/x86_64-linux-gnu/libXdamage.so.1 (0x00007faa8e45c000)
	libxkbcommon.so.0 => /lib/x86_64-linux-gnu/libxkbcommon.so.0 (0x00007faa8e41a000)
	libwayland-cursor.so.0 => /lib/x86_64-linux-gnu/libwayland-cursor.so.0 (0x00007faa8e40f000)
	libwayland-egl.so.1 => /lib/x86_64-linux-gnu/libwayland-egl.so.1 (0x00007faa8e40a000)
	libwayland-client.so.0 => /lib/x86_64-linux-gnu/libwayland-client.so.0 (0x00007faa8e3f9000)
	libXext.so.6 => /lib/x86_64-linux-gnu/libXext.so.6 (0x00007faa8e3e2000)
	libpixman-1.so.0 => /lib/x86_64-linux-gnu/libpixman-1.so.0 (0x00007faa8e33b000)
	libxcb-shm.so.0 => /lib/x86_64-linux-gnu/libxcb-shm.so.0 (0x00007faa8e336000)
	libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007faa8e30c000)
	libxcb-render.so.0 => /lib/x86_64-linux-gnu/libxcb-render.so.0 (0x00007faa8e2fd000)
	libXrender.so.1 => /lib/x86_64-linux-gnu/libXrender.so.1 (0x00007faa8e0f1000)
	libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 (0x00007faa8e0e5000)
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007faa8e072000)
	libraw1394.so.11 => /lib/x86_64-linux-gnu/libraw1394.so.11 (0x00007faa8de63000)
	libusb-1.0.so.0 => /lib/x86_64-linux-gnu/libusb-1.0.so.0 (0x00007faa8de47000)
	libgstbase-1.0.so.0 => /lib/x86_64-linux-gnu/libgstbase-1.0.so.0 (0x00007faa8ddc9000)
	libgstaudio-1.0.so.0 => /lib/x86_64-linux-gnu/libgstaudio-1.0.so.0 (0x00007faa8dd4a000)
	libgsttag-1.0.so.0 => /lib/x86_64-linux-gnu/libgsttag-1.0.so.0 (0x00007faa8dd0c000)
	libgstvideo-1.0.so.0 => /lib/x86_64-linux-gnu/libgstvideo-1.0.so.0 (0x00007faa8dc5b000)
	libswresample.so.3 => /lib/x86_64-linux-gnu/libswresample.so.3 (0x00007faa8dc39000)
	libvpx.so.6 => /lib/x86_64-linux-gnu/libvpx.so.6 (0x00007faa8da01000)
	libwebpmux.so.3 => /lib/x86_64-linux-gnu/libwebpmux.so.3 (0x00007faa8d9f5000)
	librsvg-2.so.2 => /lib/x86_64-linux-gnu/librsvg-2.so.2 (0x00007faa8d0c8000)
	libzvbi.so.0 => /lib/x86_64-linux-gnu/libzvbi.so.0 (0x00007faa8d039000)
	libsnappy.so.1 => /lib/x86_64-linux-gnu/libsnappy.so.1 (0x00007faa8d02e000)
	libaom.so.0 => /lib/x86_64-linux-gnu/libaom.so.0 (0x00007faa8cb90000)
	libcodec2.so.0.9 => /lib/x86_64-linux-gnu/libcodec2.so.0.9 (0x00007faa8bdac000)
	libgsm.so.1 => /lib/x86_64-linux-gnu/libgsm.so.1 (0x00007faa8bd9d000)
	libmp3lame.so.0 => /lib/x86_64-linux-gnu/libmp3lame.so.0 (0x00007faa8bd23000)
	libopenjp2.so.7 => /lib/x86_64-linux-gnu/libopenjp2.so.7 (0x00007faa8bccd000)
	libopus.so.0 => /lib/x86_64-linux-gnu/libopus.so.0 (0x00007faa8bc6e000)
	libshine.so.3 => /lib/x86_64-linux-gnu/libshine.so.3 (0x00007faa8ba63000)
	libspeex.so.1 => /lib/x86_64-linux-gnu/libspeex.so.1 (0x00007faa8ba45000)
	libtheoraenc.so.1 => /lib/x86_64-linux-gnu/libtheoraenc.so.1 (0x00007faa8ba08000)
	libtheoradec.so.1 => /lib/x86_64-linux-gnu/libtheoradec.so.1 (0x00007faa8b9e6000)
	libtwolame.so.0 => /lib/x86_64-linux-gnu/libtwolame.so.0 (0x00007faa8b9c0000)
	libvorbis.so.0 => /lib/x86_64-linux-gnu/libvorbis.so.0 (0x00007faa8b992000)
	libvorbisenc.so.2 => /lib/x86_64-linux-gnu/libvorbisenc.so.2 (0x00007faa8b8e7000)
	libwavpack.so.1 => /lib/x86_64-linux-gnu/libwavpack.so.1 (0x00007faa8b8bb000)
	libx264.so.155 => /lib/x86_64-linux-gnu/libx264.so.155 (0x00007faa8b5fd000)
	libx265.so.179 => /lib/x86_64-linux-gnu/libx265.so.179 (0x00007faa8a68d000)
	libxvidcore.so.4 => /lib/x86_64-linux-gnu/libxvidcore.so.4 (0x00007faa8a57a000)
	libva.so.2 => /lib/x86_64-linux-gnu/libva.so.2 (0x00007faa8a551000)
	libxml2.so.2 => /lib/x86_64-linux-gnu/libxml2.so.2 (0x00007faa8a397000)
	libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007faa8a384000)
	libgme.so.0 => /lib/x86_64-linux-gnu/libgme.so.0 (0x00007faa8a335000)
	libopenmpt.so.0 => /lib/x86_64-linux-gnu/libopenmpt.so.0 (0x00007faa8a148000)
	libchromaprint.so.1 => /lib/x86_64-linux-gnu/libchromaprint.so.1 (0x00007faa8a132000)
	libbluray.so.2 => /lib/x86_64-linux-gnu/libbluray.so.2 (0x00007faa8a0e1000)
	libgnutls.so.30 => /lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007faa89f0b000)
	libssh-gcrypt.so.4 => /lib/x86_64-linux-gnu/libssh-gcrypt.so.4 (0x00007faa89e7f000)
	libva-drm.so.2 => /lib/x86_64-linux-gnu/libva-drm.so.2 (0x00007faa89e78000)
	libva-x11.so.2 => /lib/x86_64-linux-gnu/libva-x11.so.2 (0x00007faa89e70000)
	libvdpau.so.1 => /lib/x86_64-linux-gnu/libvdpau.so.1 (0x00007faa89e6a000)
	libdrm.so.2 => /lib/x86_64-linux-gnu/libdrm.so.2 (0x00007faa89e56000)
	libOpenCL.so.1 => /usr/local/cuda/lib64/libOpenCL.so.1 (0x00007faa89c4f000)
	libgdcmDICT.so.3.0 => /lib/x86_64-linux-gnu/libgdcmDICT.so.3.0 (0x00007faa8999d000)
	libgdcmjpeg8.so.3.0 => /lib/x86_64-linux-gnu/libgdcmjpeg8.so.3.0 (0x00007faa89973000)
	libgdcmjpeg12.so.3.0 => /lib/x86_64-linux-gnu/libgdcmjpeg12.so.3.0 (0x00007faa89949000)
	libgdcmjpeg16.so.3.0 => /lib/x86_64-linux-gnu/libgdcmjpeg16.so.3.0 (0x00007faa8991f000)
	libCharLS.so.2 => /lib/x86_64-linux-gnu/libCharLS.so.2 (0x00007faa898d0000)
	libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007faa898c7000)
	libjson-c.so.4 => /lib/x86_64-linux-gnu/libjson-c.so.4 (0x00007faa898b5000)
	libgdcmIOD.so.3.0 => /lib/x86_64-linux-gnu/libgdcmIOD.so.3.0 (0x00007faa8989b000)
	libgdcmCommon.so.3.0 => /lib/x86_64-linux-gnu/libgdcmCommon.so.3.0 (0x00007faa89871000)
	libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007faa897c8000)
	libjbig.so.0 => /lib/x86_64-linux-gnu/libjbig.so.0 (0x00007faa895ba000)
	libHalf.so.24 => /lib/x86_64-linux-gnu/libHalf.so.24 (0x00007faa89575000)
	libIex-2_3.so.24 => /lib/x86_64-linux-gnu/libIex-2_3.so.24 (0x00007faa89552000)
	libIlmThread-2_3.so.24 => /lib/x86_64-linux-gnu/libIlmThread-2_3.so.24 (0x00007faa89548000)
	libarmadillo.so.9 => /lib/libarmadillo.so.9 (0x00007faa89535000)
	libpoppler.so.97 => /lib/x86_64-linux-gnu/libpoppler.so.97 (0x00007faa891f0000)
	libfreexl.so.1 => /lib/x86_64-linux-gnu/libfreexl.so.1 (0x00007faa891e5000)
	libqhull.so.7 => /lib/x86_64-linux-gnu/libqhull.so.7 (0x00007faa88f8b000)
	libgeos_c.so.1 => /lib/x86_64-linux-gnu/libgeos_c.so.1 (0x00007faa88f48000)
	libepsilon.so.1 => /lib/x86_64-linux-gnu/libepsilon.so.1 (0x00007faa88f2c000)
	libodbc.so.2 => /lib/x86_64-linux-gnu/libodbc.so.2 (0x00007faa88eba000)
	libodbcinst.so.2 => /lib/x86_64-linux-gnu/libodbcinst.so.2 (0x00007faa88ea2000)
	libkmlbase.so.1 => /lib/x86_64-linux-gnu/libkmlbase.so.1 (0x00007faa88e85000)
	libkmldom.so.1 => /lib/x86_64-linux-gnu/libkmldom.so.1 (0x00007faa88de4000)
	libkmlengine.so.1 => /lib/x86_64-linux-gnu/libkmlengine.so.1 (0x00007faa88daa000)
	libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007faa88d7a000)
	libxerces-c-3.2.so => /lib/x86_64-linux-gnu/libxerces-c-3.2.so (0x00007faa889e7000)
	libnetcdf.so.15 => /lib/x86_64-linux-gnu/libnetcdf.so.15 (0x00007faa888c2000)
	libmfhdfalt.so.0 => /lib/libmfhdfalt.so.0 (0x00007faa88898000)
	libdfalt.so.0 => /lib/libdfalt.so.0 (0x00007faa887f3000)
	libogdi.so.4.1 => /lib/libogdi.so.4.1 (0x00007faa887d7000)
	libgif.so.7 => /lib/x86_64-linux-gnu/libgif.so.7 (0x00007faa887ca000)
	libgeotiff.so.5 => /lib/x86_64-linux-gnu/libgeotiff.so.5 (0x00007faa88795000)
	libcfitsio.so.8 => /lib/x86_64-linux-gnu/libcfitsio.so.8 (0x00007faa88496000)
	libpq.so.5 => /lib/x86_64-linux-gnu/libpq.so.5 (0x00007faa88446000)
	libproj.so.15 => /lib/x86_64-linux-gnu/libproj.so.15 (0x00007faa88157000)
	libdapclient.so.6 => /lib/x86_64-linux-gnu/libdapclient.so.6 (0x00007faa8810f000)
	libdap.so.25 => /lib/x86_64-linux-gnu/libdap.so.25 (0x00007faa87f6f000)
	libspatialite.so.7 => /lib/x86_64-linux-gnu/libspatialite.so.7 (0x00007faa879e0000)
	libcurl-gnutls.so.4 => /lib/x86_64-linux-gnu/libcurl-gnutls.so.4 (0x00007faa87951000)
	libfyba.so.0 => /lib/x86_64-linux-gnu/libfyba.so.0 (0x00007faa878f9000)
	libmysqlclient.so.21 => /lib/x86_64-linux-gnu/libmysqlclient.so.21 (0x00007faa87202000)
	libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007faa86f2c000)
	libaec.so.0 => /lib/x86_64-linux-gnu/libaec.so.0 (0x00007faa86f21000)
	libdbus-1.so.3 => /lib/x86_64-linux-gnu/libdbus-1.so.3 (0x00007faa86ed0000)
	libatspi.so.0 => /lib/x86_64-linux-gnu/libatspi.so.0 (0x00007faa86e99000)
	libmount.so.1 => /lib/x86_64-linux-gnu/libmount.so.1 (0x00007faa86e39000)
	libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007faa86e0c000)
	libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007faa86df0000)
	libthai.so.0 => /lib/x86_64-linux-gnu/libthai.so.0 (0x00007faa86de5000)
	libgraphite2.so.3 => /lib/x86_64-linux-gnu/libgraphite2.so.3 (0x00007faa86db8000)
	libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007faa86db2000)
	libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007faa86da8000)
	libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007faa86d7b000)
	liborc-0.4.so.0 => /lib/x86_64-linux-gnu/liborc-0.4.so.0 (0x00007faa86cf8000)
	libsoxr.so.0 => /lib/x86_64-linux-gnu/libsoxr.so.0 (0x00007faa86c8d000)
	libogg.so.0 => /lib/x86_64-linux-gnu/libogg.so.0 (0x00007faa86c7e000)
	libnuma.so.1 => /lib/x86_64-linux-gnu/libnuma.so.1 (0x00007faa86c71000)
	libicuuc.so.66 => /lib/x86_64-linux-gnu/libicuuc.so.66 (0x00007faa86a8b000)
	libmpg123.so.0 => /lib/x86_64-linux-gnu/libmpg123.so.0 (0x00007faa86a2b000)
	libvorbisfile.so.3 => /lib/x86_64-linux-gnu/libvorbisfile.so.3 (0x00007faa86a20000)
	libp11-kit.so.0 => /lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007faa868e8000)
	libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007faa868c7000)
	libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x00007faa86745000)
	libtasn1.so.6 => /lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007faa8672f000)
	libnettle.so.7 => /lib/x86_64-linux-gnu/libnettle.so.7 (0x00007faa866f5000)
	libhogweed.so.5 => /lib/x86_64-linux-gnu/libhogweed.so.5 (0x00007faa866be000)
	libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007faa86638000)
	libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007faa8651a000)
	libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007faa864f7000)
	libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007faa864aa000)
	libblas.so.3 => /lib/x86_64-linux-gnu/libblas.so.3 (0x00007faa860e2000)
	liblapack.so.3 => /lib/x86_64-linux-gnu/liblapack.so.3 (0x00007faa85a1a000)
	libarpack.so.2 => /lib/x86_64-linux-gnu/libarpack.so.2 (0x00007faa859d2000)
	libsuperlu.so.5 => /lib/x86_64-linux-gnu/libsuperlu.so.5 (0x00007faa85962000)
	liblcms2.so.2 => /lib/x86_64-linux-gnu/liblcms2.so.2 (0x00007faa85907000)
	libnss3.so => /lib/x86_64-linux-gnu/libnss3.so (0x00007faa857b8000)
	libsmime3.so => /lib/x86_64-linux-gnu/libsmime3.so (0x00007faa85786000)
	libnspr4.so => /lib/x86_64-linux-gnu/libnspr4.so (0x00007faa85746000)
	libgeos-3.8.0.so => /lib/x86_64-linux-gnu/libgeos-3.8.0.so (0x00007faa8557d000)
	libltdl.so.7 => /lib/x86_64-linux-gnu/libltdl.so.7 (0x00007faa85572000)
	libminizip.so.1 => /lib/x86_64-linux-gnu/libminizip.so.1 (0x00007faa85367000)
	liburiparser.so.1 => /lib/x86_64-linux-gnu/liburiparser.so.1 (0x00007faa85346000)
	libssl.so.1.1 => /lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007faa852b3000)
	libldap_r-2.4.so.2 => /lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007faa8525d000)
	libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007faa85134000)
	libnghttp2.so.14 => /lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007faa85109000)
	librtmp.so.1 => /lib/x86_64-linux-gnu/librtmp.so.1 (0x00007faa850e9000)
	libssh.so.4 => /lib/x86_64-linux-gnu/libssh.so.4 (0x00007faa8507b000)
	libpsl.so.5 => /lib/x86_64-linux-gnu/libpsl.so.5 (0x00007faa85068000)
	liblber-2.4.so.2 => /lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007faa85057000)
	libbrotlidec.so.1 => /lib/x86_64-linux-gnu/libbrotlidec.so.1 (0x00007faa85049000)
	libfyut.so.0 => /lib/x86_64-linux-gnu/libfyut.so.0 (0x00007faa8503b000)
	libfygm.so.0 => /lib/x86_64-linux-gnu/libfygm.so.0 (0x00007faa85032000)
	libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007faa84f83000)
	libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007faa84f2c000)
	libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007faa84e9c000)
	libdatrie.so.1 => /lib/x86_64-linux-gnu/libdatrie.so.1 (0x00007faa84e90000)
	libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007faa84e76000)
	libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007faa84e34000)
	libicudata.so.66 => /lib/x86_64-linux-gnu/libicudata.so.66 (0x00007faa83373000)
	libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007faa83296000)
	libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007faa83263000)
	libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007faa8325c000)
	libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007faa8324d000)
	libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007faa82f85000)
	libnssutil3.so => /lib/x86_64-linux-gnu/libnssutil3.so (0x00007faa82f52000)
	libplc4.so => /lib/x86_64-linux-gnu/libplc4.so (0x00007faa82f49000)
	libplds4.so => /lib/x86_64-linux-gnu/libplds4.so (0x00007faa82f44000)
	libsasl2.so.2 => /lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007faa82f27000)
	libgssapi.so.3 => /lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007faa82ee2000)
	libbrotlicommon.so.1 => /lib/x86_64-linux-gnu/libbrotlicommon.so.1 (0x00007faa82ebf000)
	liblz4.so.1 => /lib/x86_64-linux-gnu/liblz4.so.1 (0x00007faa82e9c000)
	libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007faa82e95000)
	libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007faa82e4b000)
	libheimntlm.so.0 => /lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007faa82e3f000)
	libkrb5.so.26 => /lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007faa82dac000)
	libasn1.so.8 => /lib/x86_64-linux-gnu/libasn1.so.8 (0x00007faa82d03000)
	libhcrypto.so.4 => /lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007faa82ccb000)
	libroken.so.18 => /lib/x86_64-linux-gnu/libroken.so.18 (0x00007faa82cb2000)
	libwind.so.0 => /lib/x86_64-linux-gnu/libwind.so.0 (0x00007faa82c88000)
	libheimbase.so.1 => /lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007faa82c76000)
	libhx509.so.5 => /lib/x86_64-linux-gnu/libhx509.so.5 (0x00007faa82c26000)
	libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007faa82beb000)

Executed Command (if any)

Note: add --logging_level 0 --disable_multi_thread to get higher debug information.

$ ./build/examples/openpose/openpose.bin --camera 0 --logging_level 0 --disable_multi_thread

OpenPose Output (if any)

Starting OpenPose demo...
Configuring OpenPose...
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToProducer():210
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToProducerType():170
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToPoseMode():11
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToPoseModel():33
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToScaleMode():82
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToHeatMapScaleMode():116
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToDetector():147
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToDetector():147
Starting thread(s)...
Running configureThreadManager...
/home/discobot/Documents/github/openpose/src/openpose/producer/producer.cpp:createProducer():417
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
renderModePose = 3
renderModeFace = 3
renderModeHand = 3
renderOutput = 1
renderOutputGpu = 1
renderFace = 0
renderHand = 0
renderHandGpu = 0
/home/discobot/Documents/github/openpose/src/openpose/wrapper/wrapperAuxiliary.cpp:wrapperConfigureSanityChecks():17
/home/discobot/Documents/github/openpose/src/openpose/wrapper/wrapperAuxiliary.cpp:wrapperConfigureSanityChecks():192
userInputAndPreprocessingWsEmpty = 1
userOutputWsEmpty = 1
numberGpuThreads = -1
gpuNumberStart = 0
Auto-detecting all available GPUs... Detected 1 GPU(s), using 1 of them starting at GPU 0.
writeImagesCleaned = 
writeKeypointCleaned = 
writeJsonCleaned = 
writeHeatMapsCleaned = 
modelFolder = models/
finalOutputSize = [1280,720]
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():370
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():435
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():440
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():451
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():505
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():557
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():601
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():645
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():662
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():679
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():698
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():714
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():747
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():772
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():781
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():797
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():798
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():807
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():824
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():844
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():865
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():874
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():878
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():884
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():943
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():959
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():965
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1041
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1075
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1179
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1203
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1210
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapper.hpp:exec():419
/home/discobot/Documents/github/openpose/include/openpose/thread/threadManager.hpp:exec():185
/home/discobot/Documents/github/openpose/include/openpose/thread/queueBase.hpp:addPusher():362
/home/discobot/Documents/github/openpose/include/openpose/thread/queueBase.hpp:addPusher():362
/home/discobot/Documents/github/openpose/include/openpose/thread/queueBase.hpp:addPusher():362
/home/discobot/Documents/github/openpose/include/openpose/thread/threadManager.hpp:exec():190
/home/discobot/Documents/github/openpose/include/openpose/thread/thread.hpp:threadFunction():182
/home/discobot/Documents/github/openpose/include/openpose/thread/thread.hpp:initializationOnThread():167
Starting initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseExtractorCaffe.cpp:netInitializationOnThread():172
Finished initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseExtractorCaffe.cpp:netInitializationOnThread():191
Starting initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseGpuRenderer.cpp:initializationOnThread():74
Finished initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseGpuRenderer.cpp:initializationOnThread():84
/home/discobot/Documents/github/openpose/include/openpose/thread/thread.hpp:threadFunction():185
/home/discobot/Documents/github/openpose/include/openpose/producer/wDatumProducer.hpp:workProducer():64
/home/discobot/Documents/github/openpose/include/openpose/producer/wDatumProducer.hpp:workProducer():81
/home/discobot/Documents/github/openpose/include/openpose/thread/wIdGenerator.hpp:work():62
/home/discobot/Documents/github/openpose/include/openpose/thread/wIdGenerator.hpp:work():78
/home/discobot/Documents/github/openpose/include/openpose/core/wScaleAndSizeExtractor.hpp:work():62
/home/discobot/Documents/github/openpose/include/openpose/core/wScaleAndSizeExtractor.hpp:work():76
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpInput.hpp:work():61
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpInput.hpp:work():72
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpOutput.hpp:work():62
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpOutput.hpp:work():75
/home/discobot/Documents/github/openpose/include/openpose/pose/wPoseExtractor.hpp:work():69
F0904 16:57:44.499598 32903 syncedmem.cpp:71] Check failed: error == cudaSuccess (2 vs. 0)  out of memory
*** Check failure stack trace: ***
    @     0x7f960b43a1c3  google::LogMessage::Fail()
    @     0x7f960b43f25b  google::LogMessage::SendToLog()
    @     0x7f960b439ebf  google::LogMessage::Flush()
    @     0x7f960b43a6ef  google::LogMessageFatal::~LogMessageFatal()
    @     0x7f960b17ae2a  caffe::SyncedMemory::mutable_gpu_data()
    @     0x7f960aff46a6  caffe::Blob<>::mutable_gpu_data()
    @     0x7f960baf1331  op::NetCaffe::forwardPass()
    @     0x7f960bb0e4aa  op::PoseExtractorCaffe::forwardPass()
    @     0x7f960bb0952b  op::PoseExtractor::forwardPass()
    @     0x7f960bb06fc8  op::WPoseExtractor<>::work()
    @     0x7f960bb382cf  op::Worker<>::checkAndWork()
    @     0x7f960bb3845b  op::SubThread<>::workTWorkers()
    @     0x7f960bb4612d  op::SubThreadQueueInOut<>::work()
    @     0x7f960bb3d131  op::Thread<>::threadFunction()
    @     0x7f960bb3d4d5  op::Thread<>::exec()
    @     0x7f960bb40194  op::ThreadManager<>::exec()
    @     0x7f960bb9909e  op::WrapperT<>::exec()
    @     0x55e271d4322c  openPoseDemo()
    @     0x55e271d3f6c7  main
    @     0x7f960b4bd0b3  __libc_start_main
    @     0x55e271d3f6fe  _start
Aborted (core dumped)

Errors (if any)

OOM

Type of Issue

Select the topic(s) on your post, delete the rest:

  • Execution error
  • Help wanted

Your System Configuration

  1. Whole console output 
Starting OpenPose demo...
Configuring OpenPose...
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToProducer():210
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToProducerType():170
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToPoseMode():11
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToPoseModel():33
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToScaleMode():82
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToHeatMapScaleMode():116
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToDetector():147
/home/discobot/Documents/github/openpose/src/openpose/utilities/flagsToOpenPose.cpp:flagsToDetector():147
Starting thread(s)...
Running configureThreadManager...
/home/discobot/Documents/github/openpose/src/openpose/producer/producer.cpp:createProducer():417
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
renderModePose = 3
renderModeFace = 3
renderModeHand = 3
renderOutput = 1
renderOutputGpu = 1
renderFace = 0
renderHand = 0
renderHandGpu = 0
/home/discobot/Documents/github/openpose/src/openpose/wrapper/wrapperAuxiliary.cpp:wrapperConfigureSanityChecks():17
/home/discobot/Documents/github/openpose/src/openpose/wrapper/wrapperAuxiliary.cpp:wrapperConfigureSanityChecks():192
userInputAndPreprocessingWsEmpty = 1
userOutputWsEmpty = 1
numberGpuThreads = -1
gpuNumberStart = 0
Auto-detecting all available GPUs... Detected 1 GPU(s), using 1 of them starting at GPU 0.
writeImagesCleaned = 
writeKeypointCleaned = 
writeJsonCleaned = 
writeHeatMapsCleaned = 
modelFolder = models/
finalOutputSize = [1280,720]
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():370
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():435
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():440
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():451
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():505
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():557
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():601
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():645
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():662
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():679
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():698
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():714
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():747
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():772
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():781
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():797
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():798
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():807
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():824
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():844
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():865
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():874
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():878
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():884
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():943
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():959
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():965
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1041
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1075
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1179
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1203
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1210
/home/discobot/Documents/github/openpose/include/openpose/wrapper/wrapper.hpp:exec():419
/home/discobot/Documents/github/openpose/include/openpose/thread/threadManager.hpp:exec():185
/home/discobot/Documents/github/openpose/include/openpose/thread/queueBase.hpp:addPusher():362
/home/discobot/Documents/github/openpose/include/openpose/thread/queueBase.hpp:addPusher():362
/home/discobot/Documents/github/openpose/include/openpose/thread/queueBase.hpp:addPusher():362
/home/discobot/Documents/github/openpose/include/openpose/thread/threadManager.hpp:exec():190
/home/discobot/Documents/github/openpose/include/openpose/thread/thread.hpp:threadFunction():182
/home/discobot/Documents/github/openpose/include/openpose/thread/thread.hpp:initializationOnThread():167
Starting initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseExtractorCaffe.cpp:netInitializationOnThread():172
Finished initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseExtractorCaffe.cpp:netInitializationOnThread():191
Starting initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseGpuRenderer.cpp:initializationOnThread():74
Finished initialization on thread. In /home/discobot/Documents/github/openpose/src/openpose/pose/poseGpuRenderer.cpp:initializationOnThread():84
/home/discobot/Documents/github/openpose/include/openpose/thread/thread.hpp:threadFunction():185
/home/discobot/Documents/github/openpose/include/openpose/producer/wDatumProducer.hpp:workProducer():64
/home/discobot/Documents/github/openpose/include/openpose/producer/wDatumProducer.hpp:workProducer():81
/home/discobot/Documents/github/openpose/include/openpose/thread/wIdGenerator.hpp:work():62
/home/discobot/Documents/github/openpose/include/openpose/thread/wIdGenerator.hpp:work():78
/home/discobot/Documents/github/openpose/include/openpose/core/wScaleAndSizeExtractor.hpp:work():62
/home/discobot/Documents/github/openpose/include/openpose/core/wScaleAndSizeExtractor.hpp:work():76
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpInput.hpp:work():61
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpInput.hpp:work():72
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpOutput.hpp:work():62
/home/discobot/Documents/github/openpose/include/openpose/core/wCvMatToOpOutput.hpp:work():75
/home/discobot/Documents/github/openpose/include/openpose/pose/wPoseExtractor.hpp:work():69
F0904 16:57:44.499598 32903 syncedmem.cpp:71] Check failed: error == cudaSuccess (2 vs. 0)  out of memory
*** Check failure stack trace: ***
    @     0x7f960b43a1c3  google::LogMessage::Fail()
    @     0x7f960b43f25b  google::LogMessage::SendToLog()
    @     0x7f960b439ebf  google::LogMessage::Flush()
    @     0x7f960b43a6ef  google::LogMessageFatal::~LogMessageFatal()
    @     0x7f960b17ae2a  caffe::SyncedMemory::mutable_gpu_data()
    @     0x7f960aff46a6  caffe::Blob<>::mutable_gpu_data()
    @     0x7f960baf1331  op::NetCaffe::forwardPass()
    @     0x7f960bb0e4aa  op::PoseExtractorCaffe::forwardPass()
    @     0x7f960bb0952b  op::PoseExtractor::forwardPass()
    @     0x7f960bb06fc8  op::WPoseExtractor<>::work()
    @     0x7f960bb382cf  op::Worker<>::checkAndWork()
    @     0x7f960bb3845b  op::SubThread<>::workTWorkers()
    @     0x7f960bb4612d  op::SubThreadQueueInOut<>::work()
    @     0x7f960bb3d131  op::Thread<>::threadFunction()
    @     0x7f960bb3d4d5  op::Thread<>::exec()
    @     0x7f960bb40194  op::ThreadManager<>::exec()
    @     0x7f960bb9909e  op::WrapperT<>::exec()
    @     0x55e271d4322c  openPoseDemo()
    @     0x55e271d3f6c7  main
    @     0x7f960b4bd0b3  __libc_start_main
    @     0x55e271d3f6fe  _start
Aborted (core dumped)

  1. OpenPose version: Latest GitHub code? latest b1cb2b69cf8c4c288921e48c37f339a64db26f58

  2. General configuration:

    • Installation mode: CMake, Linux Makefile

    • Operating system (lsb_release -a in Ubuntu): Distributor ID: Ubuntu Description: Ubuntu 20.04.2 LTS Release: 20.04 Codename: focal

    • Operating system version : Ubuntu 20.04.2 LTS

    • Release or Debug mode: Release

    • Compiler : gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)

  3. Non-default settings: All default

  4. 3rd-party software:

    • Caffe version: Default from OpenPose, 1.0.0
    • CMake version : cmake version 3.16.3
    • OpenCV version: pre-compiled apt-get install libopencv-dev

  5. If GPU mode issue:

    • CUDA version: 11.1.1
    • cuDNN version: 8.0.4
    • GPU model (nvidia-smi in Ubuntu): Geforce GTX 1060 6Gb

  6. If CPU-only mode issue:

    • CPU brand & model: Intel(R) Xeon(R) CPU X5650 @ 2.67GHz 8 cores
    • Total RAM memory available: 24Gb

  7. If Python API:

    • Python version: 3.8.10
    • Numpy version 1.21.2

  8. If Windows system:

    • Portable demo or compiled library?

  9. If speed performance issue:

kognat-docs avatar Sep 04 '21 07:09 kognat-docs

Never mind , I will try v1.6.0 on CUDA 10.1 and CuDNN 7.6 and see if the memory use is better it runs with --network_resolution "-1x128" but it is jittery

kognat-docs avatar Sep 06 '21 04:09 kognat-docs

Try turn of blending

424778940z avatar Sep 06 '21 05:09 424778940z

Still got out of memory after turning off blending by --disable_blending

AlexLuya avatar Sep 14 '21 23:09 AlexLuya

Try cudnn 7.x, don't remember exact version. 8.x seems take lots of vram

424778940z avatar Sep 14 '21 23:09 424778940z

Thanks,it seems like no suitable cuda+cudnn 7x for ubuntu 21.04.Any clue about this huge memory consumptions.

AlexLuya avatar Sep 15 '21 05:09 AlexLuya

well nvidia cudnn 8.x currently supported to 20.04. you are using an unsupported OS. Try install Ubuntu 20.04, or Xubuntu to save some vram since you only have 6g...

424778940z avatar Sep 15 '21 05:09 424778940z

You need to choose an older version of gcc/g++ to be compatible with nvcc in cudnn 7.x and cuda 10.x if you have an ampere card with only 6Gb of VRAM you might be in dire straits.

kognat-docs avatar Sep 15 '21 19:09 kognat-docs

Anyone know the inside problem which cause such bigger memory comsuption?

AlexLuya avatar Sep 18 '21 08:09 AlexLuya

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Jan 09 '22 05:01 stale[bot]

Our setup with Openpose 1.5.1 uses <2gb (cuda 10.1.105, cudnn 7.6) Openpose(master) consumes 4.4GB for the same task, same models (cuda 11.6, cudnn 8.4.1)

LeroyR avatar Dec 07 '23 12:12 LeroyR