Improve WASM support/packaging

[TibixDev]

Currently the WASM models are extremely painful to get working once you start integrating the example code you get from downloading the WASM releases. We had to rewrite app-vad-asr.js entirely to get it running inside a traditional Vue app, and even then it took messy hacks. For web, the API should be an ES6 module you can import similarly to the Node example, like this recent commit, and allow specifying the data files.

[csukuangfj]

Would you like to improve it?

[csukuangfj]

By the way, all you need to use https://github.com/k2-fsa/sherpa-onnx/commit/3a1de0bfc16e282e60ec264cf3de4811355a86af#diff-1cc91c84bb20688b69f30626692d6df6d721635145bb09a1aedcc531f18fed9d is

npm install sherpa-onnx

and then everything is ready. You can now download model files and run the example code.

[TibixDev]

By the way, all you need to use 3a1de0b#diff-1cc91c84bb20688b69f30626692d6df6d721635145bb09a1aedcc531f18fed9d is

npm install sherpa-onnx

and then everything is ready. You can now download model files and run the example code.

This would be the ideal API path yeah, but these examples are for NodeJS and they do not work for Web, right?

[csukuangfj]

If you want to run it inside your browser, please read our doc https://k2-fsa.github.io/sherpa/onnx/wasm/index.html

It is not related to packaging, I think.

You MUST build it by yourself, since you need to pre-package the model files.

[csukuangfj]

We had to rewrite app-vad-asr.js entirely to get it running inside a traditional Vue app,

app-vad-asr.js is just an example, showing how to use our JS API.

You MUST write your own for your APP.

[TibixDev]

I have read this page of the documentation, that's not the problem. The problem is the output that this build process produces. It is not easy to integrate it into an application that uses standard ES6 modules (like most apps do), this example works if you click the HTML file. Now, say you want to include this in another app, what do you do? You can't just import the module, since it's not an ES6 module. You have go and painstakingly integrate it somehow into your build process that tries works with ES6 modules. React, Vue, Angular, you name it, they don't support or like when you do modules like this.

If possible, there should be a better API or integration like the ones you provide for NodeJS for example, which is much easier to reason about.

I don't think many people want to write their own API, what I did is I took app-vad-asr.js, made it into a class, and I guess this: speech.ts: https://gist.github.com/TibixDev/797a7e7869f45676625cb7fd3b0ba55c index.html: https://gist.github.com/TibixDev/65e399b7ec4308c12df5d644efccebe3

And after you can do something like:

import { SpeechRecognizer } from "../logic/speech";

async function waitForVariable(checkFn: Function, timeout = 5000, interval = 100) {
    const startTime = Date.now();

    // Check immediately first
    if (checkFn()) return;

    while (Date.now() - startTime < timeout) {
        if (checkFn()) return;
        await new Promise(resolve => setTimeout(resolve, interval));
    }

    throw new Error('Timeout waiting for variable');
}

onMounted(async () => {
    recognizer.value = new SpeechRecognizer(window.Module);

    await waitForVariable(() => window.modelInitialized)
    recognizer.value?.init()
});

But I'm sure you understand this is a terrible way for webapps and that there should be a better API provided. People want to use these technologies on web in a straightforward manner without having to resort to ugly hacks. There barely is any good speech recognition today on the web and this project is very promising.

[csukuangfj]

It is not easy to integrate it into an application that uses standard ES6 modules

Do you have any experience making the wasm module a standard ES6 module and also allowing it to load the .data file?

[TibixDev]

Not really, but I will give it a shot. I will get back to you in this issue thread if I get it running.

[csukuangfj]

Thanks!

[TibixDev]

Small update, it seems like it won't build for me? Says it's an LLVM issue. https://gist.github.com/TibixDev/c1478120fbc39d1676425db374074a44

[csukuangfj]

please read the build script carefully and don't change the.emsdk version.

[csukuangfj]

you.are using emsdk 3.1.72

That is the.reason.

[OneSeven]

Looking forward to your es6 module improvement, I also need to use it in vue