[tasks-vision] ReferenceError: self is not defined

[arghideutis]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

No

OS Platform and Distribution

Ubuntu 22.04

Mobile device if the issue happens on mobile device

No response

Browser and version if the issue happens on browser

No response

Programming Language and version

Typescript

MediaPipe version

No response

Bazel version

No response

Solution

Pose

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

Importing @mediapipe/tasks-vision module leads to an error

Describe the expected behaviour

Should import the module

Standalone code/steps you may have used to try to get what you need

In SvelteKit 1.20.4:

import { FilesetResolver, PoseLandmarker } from "@mediapipe/tasks-vision";

const vision = await FilesetResolver.forVisionTasks(
      "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision/wasm"
    );

const poseLandmarker = await PoseLandmarker.createFromOptions(vision, {
            baseOptions: {
                modelAssetPath: `https://storage.googleapis.com/mediapipe-models/pose_landmarker/pose_landmarker_lite/float16/1/pose_landmarker_lite.task`,
                delegate: "GPU"
            },
            runningMode: "VIDEO",
        });

Other info / Complete Logs

7:23:29 AM [vite] Error when evaluating SSR module /src/lib/detection/video/PoseLandmarkDetector.ts: failed to import "@mediapipe/tasks-vision"
 |- ReferenceError: self is not defined
     at file:///app/node_modules/@mediapipe/tasks-vision/vision_bundle.mjs:1:7
     at ModuleJob.run (node:internal/modules/esm/module_job:194:25)

[schmidt-sebastian]

This seems like something you should be able to fix by configuring Svelte. See how this problem is supposed to be fixed for window: https://www.okupter.com/blog/sveltekit-window-is-not-defined

self should exist almost everywhere: https://developer.mozilla.org/en-US/docs/Web/API/Window/self, so I don't think this is something we can fix.

[lc-spxl]

I guess what happens in Svelte (and other framework) is server side rendering.

What you suggest is correct, however not practically feasible.

The self error happens after an import of the mediapipe library, not in user code.

[schmidt-sebastian]

We have yet to build the testing infrastructure to ensure that our libraries work for Servers/Node. This is already in our backlog. Will hopefully get to this soon.

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[schmidt-sebastian]

Marking this as a feature request.

[akanshSirohi]

Same issue I faced when I updated my tasks-vision from v0.10.4 to v0.10.6 My OS: Windows 11 Framework: NextJS

[mattkenefick]

Similar:

 FAIL  test/unit/Comment.spec.ts
  ● Test suite failed to run

    ReferenceError: self is not defined

      at Object.<anonymous> (node_modules/@mediapipe/tasks-vision/vision_bundle.cjs:1:157)

[rahul3v]

I find the same issue in v10.0.7 So I am using v0.10.4 for now (as not causing this issue)

[schmidt-sebastian]

We changed our build process with 0.10.5 onwards, which resulted in a much smaller binary size. I am asking around internally if there is a way to not use self directly.

[schmidt-sebastian]

This should be fixed in the next release. You can get a nightly build by installing https://www.npmjs.com/package/@mediapipe/tasks-vision/v/0.0.0-nightly-20231023

[schmidt-sebastian]

Hm - that version seems to be broken. Will report back when we have a fix.

[schmidt-sebastian]

0.0.0-nightly-20231024 is working and contains the fix.

[andypotato]

I am running into the same issue - getting "ReferenceError: document is not defined" when trying to run any vision task inside nodejs

[schmidt-sebastian]

You might be running into this issue because of this: https://github.com/google/mediapipe/blob/bb4906bcd36513bd3ba6d948bf98f561a869626b/mediapipe/web/graph_runner/run_script_helper.ts.template#L15

Can you check if https://github.com/JamesJansson/importScripts helps?

[dbaynard]

I confirm 0.10.8 works.

@schmidt-sebastian Did you reopen because of the document is not defined error, or something else?

[schmidt-sebastian]

I re-opened based on @andypotato 's comment.

[andypotato]

@schmidt-sebastian if necessary I can provide some minimal example to reproduce the document is not defined error but it seems plausible the issue is related to importScripts as you suspect. I will find some time later today to try if your suggested fix works and report back

Thank you for looking into it!

[dbaynard]

Actually it seems that 0.10.8 is broken, for me — I'll have to gather error messages to see what has happened (the impact is vision loads but doesn't run).

I can use 0.10.6, though; the error that led me to this thread seemed to go away by itself, and I'm not sure why.

[Edit: I think I've figured out what I was seeing; it's an artefact of hot module reloading, where vite is trying to run ssr on a view of the site ~~that would never be visible on the server in production~~ nope it also breaks builds. So there was/is a genuine bug, but I was only hitting it in development. The component it tries to load relies on browser-only functionality.]

I suspect the document is not defined error is something else, given code expecting document should run in a browser, and code expecting self has probably been designed to run in a worker, right?

[andypotato]

So just to clarify, @mediapipe/tasks-vision is not meant to be run in a nodejs environment?

EDIT: for what it's worth, here is the minimal code example that reproduces the document is not defined error:

const vision = require("@mediapipe/tasks-vision");
const { ImageSegmenter } = vision;

let imageSegmenter;

const runningMode = "IMAGE";

async function createImageSegmenter() {

  imageSegmenter = await ImageSegmenter.createFromOptions(vision, {
    baseOptions: {
      modelAssetPath:
        "./data/selfie_multiclass_256x256.tflite",
    },
    outputCategoryMask: true,
    outputConfidenceMasks: false,
    runningMode: runningMode
  });

}

createImageSegmenter();

[wasimsafdar]

vision I am also getting the error "document is not defined" in node.js.

[schmidt-sebastian]

We currently only support browser environments and don't have testing infrastructure that can validate that our libraries work in Node. I will try to make this work though.

(Note that there doesn't seem to be good support for WebGL2 in Node, so I am not sure we can offer a solution anytime soon).

[davidstrahm]

Getting the same document is not defined error when running tasks-vision inside a webworker. Works outside of webworker.

Version 0.10.9.