Web Workers support

[nmontavon]

trafficstars

Awesome little package - I am currently experimenting with converting audio files to smaller versions for use in our AI platform, and this fits just the case!

As that can obviously heavily block the main thread, I am trying to create a client side worker for this, but seem to be running into Browser API issues:

Uncaught (in promise) TypeError: (self.AudioContext || self.webkitAudioContext) is not a constructor at self.onmessage (encodeMp3Worker.js:6:24) self.onmessage @ encodeMp3Worker.js:6

We are using SvelteKit 2 with Svelte 5, which is all based around Vite. My code looks like this:

import { createEncoder } from "wasm-media-encoders";

self.onmessage = async function (e) {
  const { pcmLeft, pcmRight, sampleRate, numberOfChannels } = e.data;

  try {
    const encoder = await createEncoder(
      "audio/mpeg",
      new URL("wasm-media-encoder/wasm/mp3", import.meta.url),
    );

    encoder.configure({
      sampleRate: sampleRate,
      channels: numberOfChannels,
      vbrQuality: 6,
    });

    let outBuffer = new Uint8Array(1024 * 1024);
    let offset = 0;
    let moreData = true;

    while (true) {
      const mp3Data = moreData
        ? encoder.encode([pcmLeft, pcmRight])
        : encoder.finalize();

      if (mp3Data.length + offset > outBuffer.length) {
        const newBuffer = new Uint8Array(mp3Data.length + offset);
        newBuffer.set(outBuffer);
        outBuffer = newBuffer;
      }

      outBuffer.set(mp3Data, offset);
      offset += mp3Data.length;

      if (!moreData) {
        break;
      }

      moreData = false;
    }

    const mp3Blob = new Blob([new Uint8Array(outBuffer.buffer, 0, offset)], {
      type: "audio/mp3",
    });

    self.postMessage({
      mp3Blob,
    });
  } catch (error) {
    console.error("Error during MP3 encoding:", error);
    self.postMessage({
      error: error.message,
    });
  }
};

I was under the impression, that the imported package should basically only be WASM, yet it is searching for Browser API types - is this expected behaviour?

Thank you for the help!

[arseneyr]

No, this is not expected. This package is just a WASM binary with some JS glue code, but the JS doesn't do anything too fancy. Definitely nothing with AudioContexts.

If I had to guess, either Vite or SvelteKit or a combination of both is pulling in some polyfills for WebAudio stuff that is not supported inside web workers. You would have to look at the transpiled code to see what's actually getting run though.

[vincerubinetti]

I'm not sure what, if anything, changed since you created this issue, but I was able to create a basic working example of this with Vite + React:

encode-worker.js

import { createMp3Encoder } from "wasm-media-encoders";
self.onmessage = async function (event) {
  const { data } = event.data;
  const encoder = await createMp3Encoder();
  encoder.configure({ sampleRate: 44100, channels: 1, bitrate: 192 });
  const buffer = new Float32Array(data.length);
  const peak = 2 ** (16 - 1);
  for (let i = 0; i < data.length; i++)
    buffer[i] = data[i] / (data[i] < 0 ? peak : peak - 1);
  const frames = encoder.encode([buffer]);
  const lastFrames = encoder.finalize();
  const blob = new Blob([frames, lastFrames], { type: "audio/mpeg" });
  self.postMessage(blob);
};

download.ts

import Encoder from "./encode-worker?worker";

export const save = async (data: Int16Array) => {
  const encoder = new Encoder();
  encoder.postMessage({ data });
  const { data: blob } = await new Promise(
    (resolve) => (encoder.onmessage = resolve),
  );
  const url = window.URL.createObjectURL(blob);
  // ... etc
};

Going to look into using comlink and vite-plugin-comlink for DX ergonomics... will report back on whether that's successful as well.

[vincerubinetti]

Tried comlink and it also worked! Here's a fuller example:

import { wrap } from "comlink";
import Worker from "./encode?worker";
import type { WasmMediaEncoder } from "wasm-media-encoders";

type MP3Params = Parameters<WasmMediaEncoder<"audio/mpeg">["configure"]>[0];

/** download file */
export const download = (url: string, filename: string) => {
  const link = document.createElement("a");
  link.href = url;
  link.download = filename;
  link.click();
};

type Encode = typeof import("./encode.ts");
const worker = wrap<Encode>(new Worker());

/** encode and save mp3 */
export const saveMp3 = async (data: Int16Array, params: MP3Params, filename: string) => {
  const blob = await worker.encodeMp3(data, params);
  const url = window.URL.createObjectURL(blob);
  download(url, filename + ".mp3");
  window.URL.revokeObjectURL(url);
};

import { expose } from "comlink";
import { createMp3Encoder } from "wasm-media-encoders";
import type { WasmMediaEncoder } from "wasm-media-encoders";

type MP3Params = Parameters<WasmMediaEncoder<"audio/mpeg">["configure"]>[0];

/** encode raw audio buffer to mp3 */
export const encodeMp3 = async (data: Int16Array, params: MP3Params) => {
  const encoder = await createMp3Encoder();
  encoder.configure(params);
  const buffer = new Float32Array(data.length);
  const peak = 2 ** (16 - 1);
  for (let i = 0; i < data.length; i++)
    buffer[i] = data[i]! / (data[i]! < 0 ? peak : peak - 1);
  const frames = encoder.encode([buffer]);
  const lastFrames = encoder.finalize();
  return new Blob([frames, lastFrames], { type: "audio/mpeg" });
};

expose({ encodeMp3 });

[nmontavon]

I completely messed up by not thinking about the AudioContext - AudioContext doesn't work in WebWorkers. For my usecase I just have to decode in the main thread (which is non-blocking anyways), then send over the Audio Data. Sorry for the confusion! Works great now :)

Thank you for all the Examples and Suggestions!

[nmontavon]

EDIT: Code for reference:

import { createMp3Encoder } from "wasm-media-encoders";

self.onmessage = async function (e) {
  const { audioData } = e.data;

  try {
    const encoder = await createMp3Encoder();
    encoder.configure({
      sampleRate: audioData.sampleRate,
      channels: audioData.numberOfChannels,
      vbrQuality: 6,
    });

    let outBuffer = new Uint8Array(1024 * 1024);
    let offset = 0;
    let moreData = true;

    while (true) {
      const mp3Data = moreData
        ? encoder.encode([audioData.pcmLeft, audioData.pcmRight])
        : encoder.finalize();

      if (mp3Data.length + offset > outBuffer.length) {
        const newBuffer = new Uint8Array((mp3Data.length + offset) * 1.5);
        newBuffer.set(outBuffer);
        outBuffer = newBuffer;
      }

      outBuffer.set(mp3Data, offset);
      offset += mp3Data.length;

      if (!moreData) break;
      moreData = false;
    }

    const mp3Blob = new Blob([outBuffer.subarray(0, offset)], {
      type: "audio/mp3",
    });

    self.postMessage({ mp3Blob });
  } catch (error) {
    console.error("Error during MP3 encoding: ", error);
    self.postMessage({ error: error.message });
  }
};