mlc-ai
Phi 3 Mini output near random (Phi-3-mini-4k-instruct-q4f16_1-MLC)
This model outputs complete noise? I can't get it to do anything useful. Is there a reason to include it at all? Is there any use case where it would be useful? Or is there a bug?
Hi! I don't seem to be able to reproduce it. What device are you using? And would Phi-3-mini-4k-instruct-q4f16_1-MLC-1k work?
Oh interesting! I'll give it a go. I'm using Windows 10 + Chrome 127.0.6533.72 (Official Build) (64-bit)
(Could maybe something have gone wrong during the download? I did notice while it was downloading the model that the tab got recycled by Chrome a few times, so I had to resend the message. The download seemed like it resumed from where it left off, though! If there's a way I can get the model file on my machine so we can compare an md5 hash or something, that might test that hypothesis)
I'll kick off the model you mentioned now :+1:
Same behaviour, I'm afraid. Also it seemed like it didn't download any new model files? It just seemed to re-use what was in my cache already.
Hmm that is a bit weird. I don't think it is due to corrupted downloaded weights. To triage a bit, could you try a smaller model like Qwen2-0.5B-Instruct-q4f16_1, or is there any model that is working on your end?
Besides, is there anything weird on the developer console (cmd+shift+c ->console on mac, not sure about windows)?
Lastly, you could change logging level to DEBUG and check console again:
It works fine here, both with the cached version, and then with a fresh model download. I'm using the current version (from two days ago, which added Gemma 2 2B support). (Mac OS, Brave)
I tried Qwen2-0.5B-Instruct-q4f16_1-MLC and even weirder output!
The console seems pretty quiet:
Tried in an incognito window, and same thing :/
And a different browser, Brave (Version 1.68.134 Chromium: 127.0.6533.88 (Official Build) (64-bit)), same thing
I am guessing it is due to WebGPU not being compatible with the usage of WebLLM. Could you share your output of https://webgpureport.org/ in Chrome if you do not mind?
No problem, here it is. Everything looks mostly green, but not sure what I'm looking for here. Thank you for your help!
webgpureport-2024-08-04T17-38-51-324Z.txt
I got the same random output on MacOS Safari with webGPU feature flag enabled. Worked fine in Chrome, in the same machine
