Frame rate instability

[cloudinnng]

Issue Summary

The openpose plugin was successfully run in unity, but sometimes the frame rate is unstable. The FPS drops to 2FPS every two seconds or so. Sometimes the frame rate is stable at 30FPS. unity console no any error. I am troubled. My environment is window10, cuda8.0,gtx1060. Thank you very much for your reply.

Executed Command (if any)

Note: add --logging_level 0 --disable_multi_thread to get higher debug information.

OpenPose Output (if any)

Errors (if any)

Type of Issue

You might select multiple topics, delete the rest:

• Compilation/installation error
• Execution error
• Help wanted
• Question
• Enhancement / offering possible extensions / pull request / etc
• Other (type your own type)

Your System Configuration

1. Whole console output (if errors appeared), paste the error to PasteBin and then paste the link here: LINK

2. OpenPose version: Latest GitHub code? Or specific commit (e.g., d52878f)? Or specific version from Release section (e.g., 1.2.0)?

3. General configuration:

   • Installation mode: CMake, sh script, manual Makefile installation, ... (Ubuntu); CMake, ... (Windows); ...?
   • Operating system (lsb_release -a in Ubuntu):
   • Release or Debug mode? (by default: release):
   • Compiler (gcc --version in Ubuntu or VS version in Windows): 5.4.0, ... (Ubuntu); VS2015 Enterprise Update 3, VS2017 community, ... (Windows); ...?

4. Non-default settings:

   • 3-D Reconstruction module added? (by default: no):
   • Any other custom CMake configuration with respect to the default version? (by default: no):

5. 3rd-party software:

   • Caffe version: Default from OpenPose, custom version, ...?
   • CMake version (cmake --version in Ubuntu):
   • OpenCV version: pre-compiled apt-get install libopencv-dev (only Ubuntu); OpenPose default (only Windows); compiled from source? If so, 2.4.9, 2.4.12, 3.1, 3.2?; ...?

6. If GPU mode issue:

   • CUDA version (cat /usr/local/cuda/version.txt in most cases):
   • cuDNN version:
   • GPU model (nvidia-smi in Ubuntu):

7. If CPU-only mode issue:

   • CPU brand & model:
   • Total RAM memory available:

8. If Python API:

   • Python version: 2.7, 3.7, ...?
   • Numpy version (python -c "import numpy; print numpy.version.version" in Ubuntu):

9. If Windows system:

   • Portable demo or compiled library?

10. If speed performance issue:

    • Report OpenPose timing speed based on this link.

[gineshidalgo99]

Are you running body? Body + hands? on which kind of video? Single person? multiple people?

[cloudinnng]

I am running on my Logitech c920 webcam. Only the body. Up to two people

[gineshidalgo99]

Only body should be quite stable (specially for few people). Hand/face get slower the more people there is.

The way this unity plugin estimates the FPS is different to how I do it in the original OpenPose, so I can't be so certain about whether it is normal or not, but it should be more stable. If you set num_gpu to 0, then it should only run the webcam (without running OpenPose on it). Could you try this and be sure than the C920 is working in a steady frame rate much higher than the OpenPose speed?

[cloudinnng]

@gineshidalgo99 Thanks for your reply, I changed /* gpuNumber */ -1 in the fourth line below to 0. Do you mean this?

OPWrapper.OPConfigurePose(
            /* enable */ true, /* netInputSize */ netResolution, /* outputSize */ null,
            /* keypointScaleMode */ ScaleMode.InputResolution,
            /* gpuNumber */ 0, /* gpuNumberStart */ 0, /* scalesNumber */ 1, /* scaleGap */ 0.3f,
            /* renderMode */ RenderMode.Gpu, /* poseModel */ PoseModel.BODY_25,
            /* blendOriginalFrame */ true, /* alphaKeypoint */ 0.6f, /* alphaHeatMap */ 0.7f,
            /* defaultPartToRender */ 0, /* modelFolder */ null,
            /* heatMapTypes */ HeatMapType.None, /* heatMapScaleMode */ ScaleMode.UnsignedChar,
            /* addPartCandidates */ false, /* renderThreshold */ renderThreshold, /* numberPeopleMax */ maxPeople,
            /* maximizePositives */ false, /* fpsMax fps_max */ -1.0,
            /* protoTxtPath */ "", /* caffeModelPath */ "");

Then I run unity. At this time there is no joint tracking, only the screen of the webcam, in this case. The frame rate is still unstable. So I guess the Openpose plugin is not particularly good for webcam support. So I used the unity WebCamTexture class to run my webcam separately, this time the frame rate is stable at about 30FPS. In addition, how to set the resolution used by the OpenPose plugin to run the webcam? The default resolution is 1280x720. Thanks again.

[gineshidalgo99]

That feed-back is really useful. Thanks. What is the resolution of the images provided by WebCamTexture? And when you mean unstable, how unstable? Could you let me know the approximate frame rate at which it is running?

Resolution: Check OPConfigureInput and its field cameraResolution (or something like this).

(And yes, setting gpuNumber 0 disables body estimation and simply runs the webcam)

[cloudinnng]

WebCamTexture I also use the same resolution 1280x720, like the code below.

_webCamTexture=new WebCamTexture(1280,720);

The FPS calculation method in the plugin is sometimes 30FPS. But in a few seconds, it dropped to 10FPS. Sometimes it is near 20FPS, even I have seen the 5FPS situation, and after few seconds, It return 30FPS, reciprocating like this. This is all in the case of only tracking the body, up to two people.

[gineshidalgo99]

Thanks for the feed-back. We will compare it against our machines and try to fix it. I will post any update in here.

[KathySwe]

@ttw1993 , I also trying to use webcam texture instead of using openpose producertype.webcam, do you think, it is possible to replace with using webcam texture? if you successfully used with webcam Texture, Please Could you show your code? Thank you so much @ttw1993

[cloudinnng]

No, I didn't replace it with WebcamTexture, I just tested the frame rate with WebcamTexture, sorry I didn't help you.

@ttw1993 , I also trying to use webcam texture instead of using openpose producertype.webcam, do you think, it is possible to replace with using webcam texture? if you successfully used with webcam Texture, Please Could you show your code? Thank you so much @ttw1993