mediapipe

mediapipe copied to clipboard

Hi. I have a problem with the mediapipe performance on android. I really got stuck with that and do need help please!

It seems like fps rate is around 10 or maybe even low.

I've built hands solution app by the documentational tutorial using android studio. I think, I've investigated all the official code and haven't found the way to increase to the certain 30 fps. That's exactly what I need because the following step in my apk is to collect information about landmarks' coordinates for further sign language recognition (my tensorflow model, of course converted to the tflite, has been trained on 30fps). Maybe is there a problem in my device? I use

• HUAWEI AUM-L41
• APL 26
• Android 8.0

If it needs, I can provide any additional information.

The mediapipe class CameraInput is being used, in which then CameraXPreviewHelper.class is being imported:

MainActivity.java

import com.google.mediapipe.solutioncore.CameraInput;

CameraInput.class

import com.google.mediapipe.components.CameraXPreviewHelper;

So I had an idea to edit CameraXPreviewHelper.class. There is a class ImageCapture.Builder. CameraXPreviewHelper.class

import androidx.camera.core.ImageCapture.Builder;
...
public class CameraXPreviewHelper extends CameraHelper {
  ...
  private Builder imageCaptureBuilder;
  ...
  public void startCamera(Activity activity, @Nonnull Builder imageCaptureBuilder, 
  CameraFacing cameraFacing, @Nullable Size targetSize) {
        this.imageCaptureBuilder = imageCaptureBuilder;
        this.startCamera((Context)activity, (LifecycleOwner)((LifecycleOwner)activity), 
        (CameraFacing)cameraFacing, targetSize);
  }
  ...
}

And I thought it would work to add a Camera2Interop option as I saw it had worked with ImageAnalysis from this issue's second answer. Something like that: CameraXPreviewHelper.class

import androidx.camera.core.ImageCapture.Builder;
\\\add imports
import androidx.camera.camera2.interop.Camera2Interop;
import android.hardware.camera2.CaptureRequest;
...
public class CameraXPreviewHelper extends CameraHelper {
  ...
  private Builder imageCaptureBuilder;
  ...
  public void startCamera(Activity activity, @Nonnull Builder imageCaptureBuilder, 
  CameraFacing cameraFacing, @Nullable Size targetSize) {
        this.imageCaptureBuilder = imageCaptureBuilder;
        \\\and this
        Camera2Interop.Extender ext = new Camera2Interop.Extender<> 
        (this.imageCaptureBuilder);
        ext.setCaptureRequestOption(CaptureRequest.CONTROL_AE_TARGET_FPS_RANGE, new 
        Range<Integer>(30, 30));
        \\\
        this.startCamera((Context)activity, (LifecycleOwner)((LifecycleOwner)activity), 
        (CameraFacing)cameraFacing, targetSize);
  }
  ...
}

But I still have a doubt because here ImageCapture is been used. Moreover, after I'd decompiled class, edited code and tried to compile that again to .class I received a bunch of errors.

Hi @KornilovaK , Could you check this closed issue https://github.com/google/mediapipe/issues/460 , please check and revert back if you have still further queries.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

Closing as stale. Please reopen if you'd like to work on this further.

Are you satisfied with the resolution of your issue? Yes No