web-llm icon indicating copy to clipboard operation
web-llm copied to clipboard

Published 20 hours ago • verified publisher icon mlc-ai

simple-chat: error during loading params onto WebGPU, GPUPipelineError: A valid external Instance reference no longer exists

Open lebron8dong opened this issue 1 year ago • 3 comments

image

lebron8dong avatar Mar 13 '24 07:03 lebron8dong

Thanks for reporting the error. Could you show the log in console? Not sure if there is more info there. Besides, does this issue occur in all models? Could you perhaps try a smaller model like gemma 2b?

CharlieFRuan avatar Mar 13 '24 23:03 CharlieFRuan

Llama：log in console image

WizardMath no error occurred；

lebron8dong avatar Mar 14 '24 12:03 lebron8dong

This is a bit strange. WizardMath has q4f16_1, could you also try Llama-2-7B-q4f16_1? Besides, how much RAM do you have? My guess is that it is some OOM issue. Looking at the field vram_required_MB in https://github.com/mlc-ai/web-llm/blob/main/examples/simple-chat/src/gh-config.js, Llama-2-7B-q4f32_1 requires roughly 2GB more vram than Llama-2-7B-q4f16_1 (hence also WizardMath).

CharlieFRuan avatar Mar 14 '24 16:03 CharlieFRuan