Web-AI-showcase

This project is a website which demonstrates some emerging Web AI demos that run locally in your browser, powered by WebAssembly, WebGPU and WebNN technologies.

Requirements

• node 16+
• npm

Environment Setup

Install necessary dependencies

npm install

Model downloading

This website itself is a static page without dependency to any server APIs. However the models needed by every AI tasks must be downloaded from somewhere to the users' browser.

To better adapt to various deployment environments, we provide two modes for model downloading:

1. Remote mode (Download models from huggingface)

   In this mode, the end users'(who use the browser to access this web page) browsers will fetch model files from huggingface.

   

   This mode is useful when the end users can access huggingface easily or the hosting server can't store large files.

2. Hosting mode (Download models from hosting server)

   In this mode, we can download the required models to the hosting server in advance, and the end users' browsers will fetch model files from the server hosting this web page.

   

   This mode is useful when the end users don't have access to huggingface or their network is too slow to download large files from huggingface.

   In Hosting mode only models are deployed on the hosting server, the end users' browser still need to fetch some other resources (e.g. some wasm files) through CDN, this means Hosting mode is not suitable for pure offline environment.

Note: LLM-Gemma uses the gemma-2b-it-gpu-int4 model, which must be downloaded and loaded manually before inference. See this for more details.

Build the showcase

The build phase will bundle the source code.

If using Hosting mode, it will also download required models and put into correct directories.

Build with Remote mode

  npm run prod:use-remote-models

Build with Hosting mode

  npm run prod

This may take a long time to download model files (~3GB to download).

Launch the showcase

Setup https server

SSL Preparation

HTTPS is required since we use WebGPU in some samples and WebGPU is only available in secure-context.

We provide a npm script to generate ssl based on openssl. Try to install openssl if you don't have it on your system.

• Linux

  # install openssl
  sudo apt-get install libssl-dev
  # generate `cert.pem` and `key.pem`
  npm run generate-ssl
  

• Windows: openssl is bundled with Git so that you can directly generate cert.pem and key.pem with following command in gitbash if you have Git installed.

  openssl req -newkey rsa:2048 -new -nodes -x509 -days 3650 -keyout key.pem -out cert.pem
  

  Then move the cert.pem and key.pem into the project root directory.

You can also setup https server with other solutions(like Express and Caddy) and set the ./dist as root directory.

Launch the server

  npm run startup

Once finished, open the browser and navigate to: https://localhost:8080 or https://your-server-ip:8080

LICENSE

Some samples for this repository are based on modifications of examples from Transformers.js(Apache-2.0) and MediaPipe(Apache-2.0).