mediapipe

mediapipe copied to clipboard

Chrome: 96.0.4664.110 "node": "14.15.4", "npm": "6.14.x" React: 16.13.1

perhaps there is some garbage collection that I can do to 'reset'/clear the connection?

RuntimeError: abort(Module.arguments has been replaced with plain arguments_ (the initial value can be provided on Module, but after startup the value is only looked for on a local variable of that name)) at Error
    at jsStackTrace (https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/face_mesh_solution_simd_wasm_bin.js:9:69351)
    at stackTrace (https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/face_mesh_solution_simd_wasm_bin.js:9:69527)
    at abort (https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/face_mesh_solution_simd_wasm_bin.js:9:43094)
    at Object.get (https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/face_mesh_solution_simd_wasm_bin.js:9:24082)
    at https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/face_mesh_solution_simd_wasm_bin.js:9:23905
    at pa.h (https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/face_mesh.js:53:414)
    at sa (https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/face_mesh.js:14:299)
    at ta.next (https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/face_mesh.js:15:91)
    at https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/face_mesh.js:42:1377
    at new Promise (<anonymous>)
    at Z (https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/face_mesh.js:42:1147)
    at https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/face_mesh.js:53:254
    at async Promise.all (index 0)

from this code:

const MediaPipeVisualiser = (params) => {
  const {
    mode,
    maskedParticipant,

  } = params;

  const maskParticipant = async (
    mode: string,
    maskedParticipant: Array
  ) => {
    try {

canvasName = <yourcanvasname>
videoEl = <yourvideoelementid>
      canvasOverlayBuff = document.querySelector(
        `#${canvasName}`
      ) as HTMLCanvasElement;

      cOverlay = canvasOverlayBuff.getContext("2d");
      videoEl = document.querySelector(
        `#${videoElementName}`
      ) as HTMLCanvasElement;
    } catch (e) {
      console.log("error in mediapipe vis");
    }
  };


  useEffect(() => {
    if (faceMesh == null || faceMesh == undefined) {
      faceMesh = new FaceMesh({
        locateFile: (file) => {
          return `https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/${file}`;
        },
      });

      faceMesh.setOptions({
        maxNumFaces: 1,
        refineLandmarks: true,
        minDetectionConfidence: 0.5,
        minTrackingConfidence: 0.5,
      });
    }
    faceMesh.onResults(onResults);

    if (videoEl !== null) {
      camera = new Camera(videoEl, {
        onFrame: async () => {
          await faceMesh.send({ image: videoEl });
        },
        width: 1280,
        height: 720,
      });
      camera.start();
    }
     return () => {
      // try to resolve abort/cache issue but doesn't help
      faceMesh = null;
// if (camera) camera.stop(); --interferes with camera function
videoEl = null;
    };

  }, [videoEl, maskedParticipant]);

  return <div />;
};

I don't see where you call .initialize() on the solutions in the code above, but sometimes this issue is the result of instantiating multiple solution instances and not using await. (See issue #2823). Can you check to see if that's the problem here?

ok, so invoke initialize after new?

I am awaiting the faceMesh.send... eg

faceMesh = new FaceMesh({
        locateFile: (file) => {
          return `https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/${file}`;
        },
      });

     await faceMesh.initialize();

hello- I'm running into the same issue again. Can someone advise - it's been a few months! I've broken my code down further:

const fetchJSElements = async () => {
    try {
      // there will be only one local, actually

      canvasName = "overlayVideo";
      canvasOverlayBuff = document.querySelector(
        `#${canvasName}`
      ) as HTMLCanvasElement;

      cOverlay = canvasOverlayBuff.getContext("2d", { alpha: true });
      videoEl = document.querySelector("#myVideo") as HTMLVideoElement;
    } catch (e) {
      console.log("error in mediapipe vis");
    }
  };

 
  async function initFaceMesh() {
    if (faceMesh == null || faceMesh == undefined) {
      faceMesh = new FaceMesh({
        locateFile: file => {
          return `https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/${file}`;
        }
      });
      faceMesh.setOptions({
        maxNumFaces: 1,
        refineLandmarks: false, // maybe this will stop the abort errors
        minDetectionConfidence: 0.5,
        minTrackingConfidence: 0.5
      });

      await faceMesh.initialize(); // does this work? seems to be also causing an error
    }
  }

  useEffect(() => {
    if (mpFaceMesh) {
      // if window
      initFaceMesh();
      faceMesh?.onResults(onResults);
    }

    fetchJSElements();
  }, []);

  useEffect(() => {
    try {
      faceMesh.onResults(onResults);
    } catch(e) {
      console.log("error onResults " + e);
    }
    console.log("videoEl is " + videoEl);

    if (videoEl !== null) {
      if (camera == null || camera == undefined) {
        camera = new Camera(videoEl, {
          onFrame: async () => {
            await faceMesh.send({ image: videoEl });
          },
          width: videoEl.videoWidth,
          height: videoEl.videoHeight
        });
        camera.start();
      }
    }
    if (camera.video.paused) {
      camera.start();
    }

    return () => {
      if (camera) {
        camera.stop();
      }
      faceMesh = null;
      videoEl = null;

      camera = null;
    };
  }, [videoEl, canvasOverlayBuff]);

still an issue :(

Uncaught (in promise) RuntimeError: abort(undefined) at Error
    at jsStackTrace (face_mesh_solution_simd_wasm_bin.js:9:69351)
    at stackTrace (face_mesh_solution_simd_wasm_bin.js:9:69527)
    at abort (face_mesh_solution_simd_wasm_bin.js:9:43094)
    at _abort (VM3108 face_mesh_solution_simd_wasm_bin.js:9:180736)
    at face_mesh_solution_simd_wasm_bin.wasm:0x4fd04e
    at face_mesh_solution_simd_wasm_bin.wasm:0x4fcf09
    at face_mesh_solution_simd_wasm_bin.wasm:0x4fd067
    at face_mesh_solution_simd_wasm_bin.wasm:0x198ad
    at face_mesh_solution_simd_wasm_bin.wasm:0x1493c
    at SolutionWasm.SolutionWasm$send [as send] (eval at new_ (face_mesh_solution_simd_wasm_bin.js:9:161510), <anonymous>:9:1)
    at pa.h (face_mesh.js:67:341)
    at sa (face_mesh.js:14:299)
    at ta.next (face_mesh.js:15:91)
    at g (face_mesh.js:42:1204)
    at abort (VM3108 face_mesh_solution_simd_wasm_bin.js:9:43125)
    at _abort (VM3108 face_mesh_solution_simd_wasm_bin.js:9:180736)
    at face_mesh_solution_simd_wasm_bin.wasm:0x4fd04e
    at face_mesh_solution_simd_wasm_bin.wasm:0x4fcf09
    at face_mesh_solution_simd_wasm_bin.wasm:0x4fd067
    at face_mesh_solution_simd_wasm_bin.wasm:0x198ad
    at face_mesh_solution_simd_wasm_bin.wasm:0x1493c
    at SolutionWasm.SolutionWasm$send [as send] (eval at new_ (face_mesh_solution_simd_wasm_bin.js:9:161510), <anonymous>:9:1)
    at pa.h (face_mesh.js:67:341)
    at sa (face_mesh.js:14:299)
    at ta.next (face_mesh.js:15:91)
    at g (face_mesh.js:42:1204)

I am potentially experiencing something similar (see #3506)

@cindyloo do the official CodePen examples (for Face Detection and Face Mesh) work for you?

my colleague just tried the FaceMesh codepen and also received the wasm errors as well

my colleague just tried the FaceMesh codepen and also received the wasm errors as well

Seems one of the WASM builds was bugged; it's now working for me.

Are you able to run the same MediaPipe code without React? If so, my first hunch would be to refactor your effect hook. Best of luck!

EDIT: SEE MY REPLY BELOW FOR CORRECTION OF MASSIVE TYPO. But basically, a hacky workaround in react is to NOT use React.StrictMode

There seems to be a number of people experiencing a similar style of issue to this. I'm developing custom code using the Hands and DrawingUtils modules from MediaPipe. I am using react / JS. I experienced a very similar message. The message would appear most of the time, but there would be times when the error would go away. The code I was using, is very standard and similar to that provided in the docs. I actually found that React was causing the issue (or rather, a react setting could provide a sloppy workaround for the issue). I was testing my code in dev mode (using npm start). However, by default, React uses strict mode when in dev mode. Amongst other things, this means that each component is rendered twice. So in order to mitigate this, I added a flag that made it so that useEffect would only run once. This fixed my errors. However, it was a nightmare to maintain and I don't think only running the code once can guarantee no errors for all use cases. Ultimately, I simply edited my react app so that it wouldn't run in strict mode. This fixed the issues that produced error messages similar to what you and others have described. It is a workaround and a poor one (as strict mode has a lot of useful dev features). However, if you want a quick fix, it's worth a try. So my index.js file went from roughly:

`import React from 'react'; import ReactDOM from 'react-dom/client'; import './index.css'; import App from './App'; import reportWebVitals from './reportWebVitals';

const root = ReactDOM.createRoot(document.getElementById('root')); root.render( <App /> ); reportWebVitals();`

TO:

`import React from 'react'; import ReactDOM from 'react-dom/client'; import './index.css'; import App from './App'; import reportWebVitals from './reportWebVitals';

const root = ReactDOM.createRoot(document.getElementById('root')); root.render( <React.StrictMode> <App /> </React.StrictMode> ); reportWebVitals();`

Hope this helps in some way. Though, this still wouldn't explain why the codepen demo doesn't work. I've attached the working version of my project in case you want to have a deeper look at my code. Good luck! project.zip

There seems to be a number of people experiencing a similar style of issue to this. I'm developing custom code using the Hands and DrawingUtils modules from MediaPipe. I am using react / JS. I experienced a very similar message. The message would appear most of the time, but there would be times when the error would go away. The code I was using, is very standard and similar to that provided in the docs. I actually found that React was causing the issue (or rather, a react setting could provide a sloppy workaround for the issue). I was testing my code in dev mode (using npm start). However, by default, React uses strict mode when in dev mode. Amongst other things, this means that each component is rendered twice. So in order to mitigate this, I added a flag that made it so that useEffect would only run once. This fixed my errors. However, it was a nightmare to maintain and I don't think only running the code once can guarantee no errors for all use cases. Ultimately, I simply edited my react app so that it wouldn't run in strict mode. This fixed the issues that produced error messages similar to what you and others have described. It is a workaround and a poor one (as strict mode has a lot of useful dev features). However, if you want a quick fix, it's worth a try. So my index.js file went from roughly:

`import React from 'react'; import ReactDOM from 'react-dom/client'; import './index.css'; import App from './App'; import reportWebVitals from './reportWebVitals';

const root = ReactDOM.createRoot(document.getElementById('root')); root.render( ); reportWebVitals();`

TO:

`import React from 'react'; import ReactDOM from 'react-dom/client'; import './index.css'; import App from './App'; import reportWebVitals from './reportWebVitals';

const root = ReactDOM.createRoot(document.getElementById('root')); root.render( <React.StrictMode> </React.StrictMode> ); reportWebVitals();`

Hope this helps in some way. Though, this still wouldn't explain why the codepen demo doesn't work. I've attached the working version of my project in case you want to have a deeper look at my code. Good luck! project.zip

I had the exact opposite situation，I delete React.StrictMode, and the error disappeared, that's weired.

thanks!

That's great. Turns out there was a typo in my original answer. I would like to clarify that I fixed my issue by deleting React.StrictMode. So, if you want a quick fix, DELETE React.StrictMode. Apologies about the typo and glad it worked in your case!

Any updates on this? Thank you!

DON'T Delete the React.StrictMode. We want this there to check if we run into some strange states, effects or callbacks. Which it seems like we do with the mediapipe, the problem seems to be when you trying to setOptions or intialize the same face detection twice (which is bad practice). However we can solve this by saving the current faceDetection and don't do these calls if it is already setup.

I solved it like this:

let faceDetection = null;
export function initiateFaceTracking() {
 if (!faceDetection) {
   console.log("SETUP NEW FACETRACKING");
   faceDetection = new FaceDetection({
     locateFile: (file: any) => {
       return `${cdnBaseUrl}/${"face_detection"}@${FaceDetectionVersion}/${file}`;
     },
   });

   faceDetection.setOptions({
     ...defaultOptions,
   });

   faceDetection.initialize().then(() => {
     return faceDetection;
   });
 }
}

You could also ofcourse do .stop() threw it away and start up a new FaceDetection if you rather want that.

Hello @cindyloo, We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediaPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.

You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions.

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

Thanks @philipmortimer, the fact that it run twice in quick , parallel, succession was indeed the issue!

We are closing this issue for now due to lack of activity.

Are you satisfied with the resolution of your issue? Yes No