Provide a stable method of bundling the WASM vision binary

[robintown]

Let's say I'm building a web application which uses a bundler like Vite and depends on @mediapipe/tasks-vision for image segmentation. The recommended way of constructing a WASM fileset is to point to a URL which happens to serve the right files on a CDN:

const vision = await FilesetResolver.forVisionTasks(
    "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision/wasm"
);

But I really would prefer for my application to not depend on an external CDN at runtime. I already have @mediapipe/tasks-vision in my node_modules, so I could do something like:

const vision = {
  wasmLoaderPath: new URL(
    "../node_modules/@mediapipe/tasks-vision/wasm/vision_wasm_internal.js",
    import.meta.url,
  ).href,
  wasmBinaryPath: new URL(
    "../node_modules/@mediapipe/tasks-vision/wasm/vision_wasm_internal.wasm",
    import.meta.url,
  ).href,
}

Now my bundler knows to copy vision_wasm_internal.js and vision_wasm_internal.wasm into the build directory, but this doesn't seem especially reliable. If I move my JS file, the path will be wrong. Similarly if the structure of node_modules happens to change, or if I switch to another dependency loader like Yarn's Plug-n-Play, the link will break. And as the file names indicate, these are internal files which I assume my own code shouldn't be referencing in the first place.

Instead I would like it if there were some official way, in a Node or Vite project, to load the WASM vision binary via the stable public API of @mediapipe/tasks-vision. For instance, what's keeping the Mediapipe module from exporting a fileset statically?

export const visionFileset = {
  wasmLoaderPath: new URL("./wasm/vision_wasm_internal.js", import.meta.url).href,
  wasmBinaryPath: new URL("./wasm/vision_wasm_internal.wasm", import.meta.url).href,
}

Constructing a URL based on import.meta.url, like this, would allow the binaries to be picked up by any bundler that understands ES modules.

[kuaashish]

Hi @whhone,

Could you please review this suggestion and let us know your thoughts?

Thank you!!

[mattsoulanille]

This may be where things move in the future, but currently, the new URL("relative_path", import.meta.url) syntax doesn't work with other ways loading mediapipe, like non-module script tags and bundlers that don't rewrite import.meta.url when bundling (e.g. esbuild without vite).

There are a few competing syntaxes for loading static assets (static asset handling from Vite, content types from esbuild, etc.), and we may want to wait until one of these wins out.