whisper.cpp icon indicating copy to clipboard operation
whisper.cpp copied to clipboard

Published 20 hours ago • verified publisher icon ggerganov

Possible to run on Microcontroller?

Open TheRealUnBot opened this issue 2 years ago • 2 comments

I got interested in this project because of its pure cpp and c implication. Mainly because of the idea of being able to port anywhere.

So the question is, can you run this on the microcontroller? Specifically the ESP32S3 due to the multiple cores and it's small form factor. I have not actually tried this yet but I already am wondering if the ESP32S3 has enough RAM to even run on the tiny model as it needs at least 300MB of RAM and the ESP32S3 has at max 8 MB (though if used raw, can be given 1GB so maybe).

I just want to find this out, high chance it will be too slow to be usable but just for fun.

TheRealUnBot avatar Dec 06 '22 16:12 TheRealUnBot

Very unlikely :) You cannot fit the model in memory - tiny.en is 75 MB on disk and it needs about 400 MB at runtime. I'm not aware of microcontrollers that have this amount of memory.

ggerganov avatar Dec 07 '22 06:12 ggerganov

Whelp time to break out the Allwinner SOCs :)

Thanks anyways. Although technically it does support up to 1GB of RAM and Flash.

TheRealUnBot avatar Dec 07 '22 16:12 TheRealUnBot