GPU Device Error: Uncaught (in promise) TypeError: lib$1.exports.detectGPUDevice is not a function

[devluixos]

Hey WebLLM Team! 🌟

I've been diving into integrating WebLLM within an Obsidian plugin and stumbled upon a little hiccup. Running on Windows 11 with an Intel processor (which should have integrated graphics), I encountered the following error when trying to get my chat module up and running: TypeError: lib$1.exports.detectGPUDevice is not a function

This pops up during the ChatModule's reload method execution. Here's a quick peek at the code causing the problem:

import * as webllm from "@mlc-ai/web-llm";

const chat = new webllm.ChatModule();

export async function generateResponse(prompt: string): Promise<string> {
  const myAppConfig: webllm.AppConfig = {
    model_list: [
      {
        "model_url": "https://huggingface.co/mlc-ai/Llama-2-7b-chat-hf-q4f32_1-MLC/resolve/main/",
        "local_id": "Llama-2-7b-chat-hf-q4f32_1",
        "model_lib_url": "https://raw.githubusercontent.com/mlc-ai/binary-mlc-llm-libs/main/Llama-2-7b-chat-hf/Llama-2-7b-chat-hf-q4f32_1-ctx4k_cs1k-webgpu.wasm",
      },
    ]
  }

  const selectedModel = "Llama-2-7b-chat-hf-q4f32_1"
  await chat.reload(selectedModel, undefined, myAppConfig);

  const reply = await chat.generate(prompt);
  return reply;
}

Since the plugin runs inside Obsidian, I'm curious if the environment might be affecting WebGPU support or if there's something else at play. Any tips, tricks, or insights you could share would be super appreciated!

Thanks in advance!

[DavidGOrtega]

This has happened to me also with electron and react 18 (hidden webpack 5). Same electron was able to run it with parcel. To my mind this is more an issue with the bundler.

[DavidGOrtega]

@Luix-dev if you share more details I might be able to figure out a bit.

[devluixos]

@Luix-dev if you share more details I might be able to figure out a bit.

Thank you! What information do you require? Here are some things:

• @mlc-ai/web-llm": "^0.2.23
• @tsconfig/svelte": "^5.0.2
• @types/node": "^16.11.6
• @typescript-eslint/eslint-plugin": "5.29.0
• @typescript-eslint/parser": "5.29.0
• esbuild": "^0.20.0
• typescript": "4.7.4

If the project itself would be a bigger help, here's the git link: https://github.com/Luix-dev/TextSight

If you want to try it in obsidian you'll have to clone the folder into the plugins folder. Otherwise, I can provide further information if you tell me what would be important

Thanks in advance!

[devluixos]

I haven't found any solutions to this problem. It's the environment, inside obsidian, that causes it, especially those related to hardware access like GPU detection. (Or at least I think it is)

[DavidGOrtega]

@devluixos I have managed to identify whats wrong with obsidian but I do not have yet the solution (almost). the issue is that obsidian is electron and probably has the feature contextIsolation to false for some reason that I have yet to identify it does not like it. I know it because I have also an electron app using web-llm and this has happened to me. I will come back

[HahaBill]

@devluixos I have managed to identify whats wrong with obsidian but I do not have yet the solution (almost). the issue is that obsidian is electron and probably has the feature contextIsolation to false for some reason that I have yet to identify it does not like it. I know it because I have also an electron app using web-llm and this has happened to me. I will come back

Hi! Have you got any updates with this?

[devluixos]

@devluixos I have managed to identify whats wrong with obsidian but I do not have yet the solution (almost). the issue is that obsidian is electron and probably has the feature contextIsolation to false for some reason that I have yet to identify it does not like it. I know it because I have also an electron app using web-llm and this has happened to me. I will come back

Hi! Have you got any updates with this?

Sorry nothing yet. Wanted to use this for my master thesis, had to go with an alternative for now since I have deadlines. But I'll come back soon to take a look at it

[HahaBill]

@devluixos I have managed to identify whats wrong with obsidian but I do not have yet the solution (almost). the issue is that obsidian is electron and probably has the feature contextIsolation to false for some reason that I have yet to identify it does not like it. I know it because I have also an electron app using web-llm and this has happened to me. I will come back

Hi! Have you got any updates with this?

Sorry nothing yet. Wanted to use this for my master thesis, had to go with an alternative for now since I have deadlines. But I'll come back soon to take a look at it

Alright, great! Thank you, and good luck with your thesis!

If you don't mind, what is your thesis about? You don't have to answer, and I am just curious.

[JohnReginaldShutler]

Hi! Has anyone gotten any updates on this? Facing the same issue!

@customautosys