willow
willow copied to clipboard
toverainc
Support flashing from Windows
As learned from #60 (again)
Currently Windows users are somewhat "left out in the cold" and passing USB through VMs, etc is a mess.
We should create a Powershell or similar script so Windows users can flash a willow-dist.bin from the build process using esptool natively within Windows.
Hey - Using my normal account now.... Will we assume that the user has a willow-dist.bin on their local Window system to start? This shoud be an easy script to create, happy to look at it.
That's what I'm thinking! Definitely bail if the file can't be found but other than that should be fairly straightforward?
In the main project (in the container) we could also start (with an explicit command) a tiny HTTP server or something and fetch willow-dist.bin from there whenever it's invoked. That way people are always flashing the last dist file if they do subsequent builds.
I like the HTTP server idea - makes sense, and then the user doesnt need to worry about copying it to the right location / etc...
I just wrote up a quick PS script that downloads Python 3.7 and installs it to c:\willow (Thought it would be easier to put it here to keep track of things)
Then pip deploys esptools
The esptools flashes the firmware.
I need to test on a new Windows system though, my normal laptop has too many apps deployed and paths / etc ... so I want to make sure it works on a different Windows system.
I can add the download of the firmware from the container - Is the thought that they would have the container running on their Windows system in Docker Desktop? Im not sure if the current continer exposes 80 (guessing it doesnt yet?) - but if we do that, it woud be easy to add one line to the script to download the firmware to c:\willow
Last thing is the serial port - I can query and assum the first found port has the ESP on it or just ask the user for the COMx port when they start the script.
Wow, thanks a lot! That was quick!
Can Docker Desktop on Windows run Linux containers? Sorry for the ignorance, I've been almost exclusively a Linux/Mac user for many years now. Given these issues, etc we will make sure to get some Windows installs going so we can support Windows ourselves.
I would think that the usual EXPOSE/-p args to Docker would work but there are all kinds of other issues that could be introduced with this approach.
Sorry quick edit - worst case scenario we could leave it up to the user to get willow-dist.bin to their Windows machine somehow, possible with a command line argument to the PS script to skip HTTP?
COM port - almost forgot! I assume ACM devices are standard COM ports in Windows? If so, that's probably a safe assumption these days.
@ssurovich not sure if you saw the commit but we should be ready on the HTTP server side
Excellent... I didn't see that, been a busy Saturday - I'll take a look tonight and add the download portion. Should have it all done and fully tested tomorrow.
No worries! Just wanted to @ you to make sure you saw it so we can keep moving this forward.
Thanks again!
Absolutely- I appreciate the nudge, I would have missed it, no doubt.
I'm happy to help out - this has so much potential in my opinion. I showed my wife the speed last night and she was impressed. Of course her first question was about asking non HA intents, I told her it's just a start. She wants me to integrate it with ChatGPT and have all voice prompts / information read back. It's all there, just need some time....
Thanks! We're a little obsessed with interactivity/speed (with quality, of course) so we appreciate that!
While ChatGPT integration is of course possible via any number of ways the Willow Inference Server supports self-hosted LLMs natively. It does 4 bit quantization of any LLaMa based model (currently) and I've personally had good experiences with Vicuna 13B. Total VRAM usage ends up around 18GB so all-in so higher end GPU for sure in terms of self-hosting.
We're still working out the flow but the idea is to support pipelining natively in WIS so:
Willow speech input -> STT -> LLM -> TTS -> Willow playback
With pipelining in WIS the only missing piece there is the Willow playback component but fundamentally it's easy enough. We're just very deliberate about things.
Regarding HA intents, given that Willow (either in local or WIS mode) is so capable we plan on creating a repo of HA intents that dramatically add to what HA supports natively.
That's awesome to hear - very exciting, brings me back to when I was writing my own home assistant long before HA or vendor solutions using Microsoft Kinects - the mics were great and the IR camera was good for occupancy detection. With your work, I'm more excited than I was back then :) I'm around for any testing or potential work, just beginning another book (Kubernetes) that will take a lot of time, but there's always time for the fun stuff in between that and my real paying job
Hah, I have a Kinect in my hardware pile as well!
We're happy to have you help out!
Wow, thanks a lot! That was quick!
Can Docker Desktop on Windows run Linux containers? Sorry for the ignorance, I've been almost exclusively a Linux/Mac user for many years now. Given these issues, etc we will make sure to get some Windows installs going so we can support Windows ourselves.
I would think that the usual EXPOSE/-p args to Docker would work but there are all kinds of other issues that could be introduced with this approach.
Sorry quick edit - worst case scenario we could leave it up to the user to get willow-dist.bin to their Windows machine somehow, possible with a command line argument to the PS script to skip HTTP?
COM port - almost forgot! I assume ACM devices are standard COM ports in Windows? If so, that's probably a safe assumption these days.
Yep - It can run Linux containers -great thing about Docker Desktop on Windows.
On Windows the COM will be different for most people, Mine was COM8 but thats because I flash a lot of ESP devices and some were in still in the registry as a port, so 8 was next.
I think we can take two ways - Ill get the final script running without HTTP download - just for people who may have a firmware. Ill update it to a Docker support script for people who may want to build it on Windows and then just download the FW usinf HTTP.
I think the options would be good for people.
QQ - Did you want me to do a PR or did you want me to just send the Powershell script over with the short readme with the steps. Im doing a final test now but it should have all checks to make it fairly easy for people to execute. It will error out if the folder I need isnt created and it will also fail if willow-dist.bin isnt in that folder. I cant verify the COM port right now, getting ports isnt too bad in Powershell, but verifying which one has the ESP attached is a bit tricky. So right now, the need to manually supply the COM port (found by looking at device manager).
If you could do a PR with the Powershell script under misc/ that would be great! On any host platform getting the device port to flash is something that can be done programmatically but I'm always weary of cases like this where you're operating in unknown environments with any number of variables that can't possibly be accounted for. For the time being I'd like to stick with users having to explicitly define the flash/serial monitor port.
Speaking of device serial monitor - do you have a suggested terminal implementation? Last time I was on Windows Putty seemed like a fairly standard approach people used but it's been many years and I'm not sure if that's still viable/ideal. Ideally it could use the same approach we have today where define the port for flash and monitor once.
Cool - Ill do a PR today. Ive tested the script a number of times and its working great. I had a failure during one flash, it got to around 45% and timed out, but rerunning the script flashed it successfully.
Putty would be a good solution, I do still use that alot for both SSH and for a Serial Console. Could add that to the script as we continue to add features to it?
Added the readme and script - anyone willing to test it out, let me know of any issues. Ive flashed mine over 20 times with this script without any issues (assuming c:\willow exists with the fimware in it - for now)
I think it would be ok if we even fetch Putty like we do Python as long as we "install" in our own directory. I use install in quotes because from what I remember Putty is just a binary without installation. If Putty can take the already user-specified port (and our baud rate) as command line arguments that would be even better.
Yep, its a single binary, so I could download it to the c:\willow directory and using the COM port value the script asks for to flash, start a session with that COM port for the serial console.
The container side is a bit more work. Nothing too difficult, but will take more time that just the flashing script. I dont want to do the installation for Docker in a script, I would rather leave that to the user so we dont stomp on anything or upgrade something that could cause other issues.
I'll check out your utils.sh and look at the steps you're doing and just make that work in Windows with a script, adding a final step of copying the willow-dist.bin file into the c:\willow folder.
Great!
What else needs to be done container side other than commit aded0ab14d333b0bf9e451f7c801733eb8f0aae9?
I think thats perfect - exposing the container on the web port is all we need (and of couse, a web sever with the just the willow-dist.bin FW). Ill do a simple download using the port and that should be it - Nice and simple...
I hope Docker Desktop on Windows supports opening and exposing ports but it's so common I would have to imagine it does.
Note that like all variables defined at the top of utils they can be overridden via .env in the root dir but yes, the powershell script so match the current default of 10000.
For what it's worth, I did a build and flash using WSL.
Docker desktop is able to integrate with WSL 2: https://docs.docker.com/desktop/windows/wsl/
And I used usbipd to attach the device. Only caveat is I had to run the flash as root.
I was debating some WSL steps, but I wanted to keep it simple for people that may not know about WSL / etc... Its also one reason I wanted to create an easy flash in Powershell - easy for most Windows users. At one point, I assume Willow will have a more generic firmware image available and we will be able to include it to allow most Windows users to just run the script and flash.
Do we have something ready I can test from my Windows box? I've been attempting with a docker image on Ubuntu but can't seem to get the docker image to start so a Windows port would be great. I am well versed in POSH. If it runs into any issues I can debug.
Correct me if I'm wrong. Within Ubuntu we are merging the software and our configuration settings onto a bin file? Once this is created esptools can flash? I got the configuration menu to start, then I was away for a week. Now I can't find any evidence of an output. :( Love to try efforts of ssurovich
Hey @tombloemker - Yes, you will need to copy the bin (firmware) from the Ubuntu machine to your Windows machine and then you can flash it using my Powershell script or just try the ESPtools for Windows.
A web server was added to the build, Ive been packed at work and havent been able to test that out yet - It would let you download the bin from a small web server to make it easier to pull it from your build machine.
On that, just to makes sure - the step youre stuck on now is getting the firmware bin from your Ubuntu to Windows but you cant fine the built firmware after following the build steps?
No, things have progressed. Since I know nothing of Ubuntu but by the power of Youtube I successfully completed the build. Per the read.me instructions, I stopped at the PORT assignment assuming WLS2 is an issue. I'm running Ubuntu within my windows machine. What is the name of the flash bin file? Am I looking for willow-dist.bin? I searched and I don't have one.
Once I know I have the right .bin I'll try ESPtools of Windows first.
So if you want to get your file, inside the willow docker (run ./utils.sh docker from the directory you have willow cloned), then when inside the container you run ./utils.sh serve, it will make a small http server you can access anywhere on your network to download the distro bin:
Note mine is running on port 20000 as I am running this in the same docker network as my WIS instance and the default port 10000 conflicts with my installation. By default it would host on port 10000. The console messages will tell you this:
Also be sure to follow the instructions in the readme under the Advanced Usage closely, specifically of note on flashing at 0x0!
@nikito, Wow that explanation helps a lot! Sorry for the noob question, I was under the impression a container was built as part of the script and goes away after the processing is done. I searched for containers within the willow directory and didn't find any. Are you saying create a new container and run it from there? I tried that from within willow which results with the error "Command Run: not found". So I guess I don't understand what you mean by "Willow container".
Standby, I think I figured it out...
yeah what I mean is, when you are in the willow directory you run "./utils.sh docker". That then runs the willow docker container; you will notice your shell change to something like "root@ea3457bn#". At that point you are inside the willow docker container. Once inside there you would then run the ./utils.sh serve command. 😃