espruino
[Gesture Lock] Create Testing App
This app adds a ML model to the Bangle.js that has a twist out gesture that locks the bangle. Give it a try here! Current Data Contributors:
- @bobrippling
- @thyttan
- me :)
Edit by @thyttan: Removed the link pointing to the app since it will turn Bangles to illuminated bricks, not for eternity but for a while.
Just tried it - but it just made my Bangle lock up! It was even hard to trigger the watchdog.
No debugging info right now. Could try gathering if you want.
@thyttan Oh, wow, I'm so sorry about that! I hope your bangle is back to normal! It could be because it's flashing model.lite onto the watch directly as .tfmodel. The other gesture app has it uploaded as a .js file which is then decompressed, but according to the thread I opened, @gfwilliams said it should be fine... Any clue as to how I should proceed?
No debugging info right now. Could try gathering if you want.
@thyttan If you feel comfortable, maybe try gathering some info, but I would completely understand if you don't want to!
I'll see if there's some output to the web ide 👍
And no worries, I got the watch back to normal. I managed to reboot and uninstall via app loader without too much trouble.
Failed when trying to install now with these errors:
OUT OF MEMORY at :1:41
...age').write(".tfmodel",atob("IAAAAFRGTDMAAAAAFAAgABwAGAAUABA...
^
Uncaught SyntaxError: Got UNFINISHED STRING expected EOF
at REPL (:1:42)
...ge').write(".tfmodel",atob("IAAAAFRGTDMAAAAAFAAgABwAGAAUABAA...
^
Execution Interrupted
New interpreter error: LOW_MEMORY,MEMORY
Uncaught ReferenceError: "NHy9zY89bksK7NR37EyWxO6CgP5EbbaBNHjpQo1ug1VMkdL9fTruxQ58DVY4VX9" is not defined
at REPL (:1:77)
...TruxQ58DVY4VX9PEKvFijVyCbdT/32OANOrrzOVeDpWjoavSoC28sFviqxza...
^
>
It's {"name":".tfmodel","url":"model.js","evaluate":true} - it does what it says on the tin - it's evaluating the entire JS file in RAM in order to create the .tfmodel file.
You should be able to write the .tfmodel as binary like any other file. If that's actually your problem, perhaps it's something to do with GitHub treating the tfmodel file as a text file when it should be binary, and swapping what it thinks are line endings?
I attempted to upload it through the it for me, but the same issue popped up, and I cannot access the watchdog, and the bangle is absolutely frozen. @thyttan how did you recover the bangle? Bluetooth is disabled, is there a fix? Otherwise I'll just let it drain, and attempt a clean boot.
@gfwilliams
The first upload was with the binary, and that also ran out of memory I assume. Do you know how to tell GitHub it's a binary file?
I attempted to upload it through the it for me, but the same issue popped up, and I cannot access the watchdog, and the bangle is absolutely frozen. @thyttan how did you recover the bangle? Bluetooth is disabled, is there a fix? Otherwise I'll just let it drain, and attempt a clean boot.
Yeah, that sounds like what happened to mine.
For my case, I was connected to my phone via bluetooth. Disconnecting the bangle from within Gadgetbridge (or disabling phone bluetooth I guess) made it so the watch became responsive to the watchdog. I think that was what did it at least.
If you can't do the same to toggle the bluetooth connection, maybe see if connecting and disconnecting the charger nudges the Bangle in a similar way?
@thyttan sigh... I tried all those, but no luck. I'll just wait for a complete discharge. Thanks!
Failed when trying to install now with these errors:
OUT OF MEMORY at :1:41 ...age').write(".tfmodel",atob("IAAAAFRGTDMAAAAAFAAgABwAGAAUABA... ^ Uncaught SyntaxError: Got UNFINISHED STRING expected EOF at REPL (:1:42) ...ge').write(".tfmodel",atob("IAAAAFRGTDMAAAAAFAAgABwAGAAUABAA... ^ Execution Interrupted New interpreter error: LOW_MEMORY,MEMORY Uncaught ReferenceError: "NHy9zY89bksK7NR37EyWxO6CgP5EbbaBNHjpQo1ug1VMkdL9fTruxQ58DVY4VX9" is not defined at REPL (:1:77) ...TruxQ58DVY4VX9PEKvFijVyCbdT/32OANOrrzOVeDpWjoavSoC28sFviqxza... ^ >
And to clarify, this was during upload via the RKBoss6's app loader and not after the app was installed and in use.
So as @gfwilliams said, it's probably the upload of the model, and evaluating it, which takes up memory. How do we bypass that? I'll try reverting the upload back to the first version, which has it as the raw model, not the atob(...). @thyttan or @gfwilliams If you feel up for doing another test with that to collect some debug info, that would be really great. Be careful though...
It just so happens I have a second Bangle 2 laying around - perfect for this ;)
Sadly still the same problem occurs on my other watch. Latest cutting edge firmware. Made sure to charge it very little so it should flatten the battery soon.
Edit: Hm, as I was writing this it became responsive again. Maybe again a disconnect of bluetooth did something. Not sure.
Huh, I guess we have to just wait until someone knows something, if you guys know someone who might be able to help, feel free to ping them...
It just so happens I have a second Bangle 2 laying around - perfect for this ;)
I only have one haha! Do you know whether a full drain and clean boot solves it? Getting a bit worried, as mine is still unresponsive...
Thomas Vikström was at least active on the previous incarnation of the forum, maybe you can contact him via youtube comments: https://www.youtube.com/watch?v=Drn6Nd0vjbo
I'm not sure of his github handle or if he has one.
Maybe helpful: https://github.com/edgeimpulse/expert-projects/blob/main/accelerometer-and-activity-projects/gesture-recognition-banglejs-smartwatch.md
https://github.com/orgs/espruino/discussions/2926
https://github.com/orgs/espruino/discussions?discussions_q=is%3Aopen+%40thomasvikstr%C3%B6m+
Maybe you can contact him via LinkedIn: https://www.linkedin.com/posts/thomas-vikstr%C3%B6m-71b6767_banglejs2-deeplearning-ai-activity-6893241261231357953-gKBh
It just so happens I have a second Bangle 2 laying around - perfect for this ;)
I only have one haha! Do you know whether a full drain and clean boot solves it? Getting a bit worried, as mine is still unresponsive...
When your battery has run flat and you've charged it (I suggest only a little) keep holding the button when turning the Bangle on. That will trigger the recovery screen where you can choose to factory reset the watch. I can't imagine that would not work.
If you don't want to reset the watch, I think it would also do if you leave the watch laying flat on the table (so you don't accidentally trigger the gesture and problem) and then turn it on normally. Then you'll probably be able to connect to either Web IDE or the app loader to uninstall/remove the app.
Depending on the battery level it may take a while though...
@thyttan Would a clean boot work instead of a factory reset? I'm not sure if the model would be started with that, but maybe it would work?
Edit: As far as I can tell, it only stops calling the boot scripts. It would be a good debug to see whether the boot is at fault, since when I tried reloading the watch after uploading that from the IDE I ran into the memory issue...
Would a clean boot work instead of a factory reset?
I think so yes. Also thinking some more I believe you should be able to connect to Web IDE or app loader also when the recovery screen is on.
Yeah, watchdog (holding the button) should reset it, odd that it isn't for this. Maybe the watchdog is disabled during the file upload? Or its timeout altered?
Maybe the watchdog is disabled during the file upload? Or its timeout altered?
Clarifying again, the hangup happens when the app is installed and does not produce any error message.
The errors during install from app loader was fixed.
Re timeout I don't know.
I cannot access the watchdog
This is very odd - it should pretty much always work. Not the ~2sec press, but the 10 second one that totally reboots the watch.
The only things that don't allow reboot are storage writes/compaction (as that might corrupt storage) but as far as I can see Tensorflow doesn't mess with the watchdog.
Please can you run E.CRC32(require("Storage").read(".tfmodel")) on your working version? That might help us to tell if the version the app loader is installing is the same, or corrupted
@gfwilliams yep, nothings seems to be working for me, I tried holding it down for a full minute! Just waiting until it discharges, which might take weeks... it was at 70%
Alright, my bangle is back in business! It's charged a fair bit now, so I'll wait for it to get to 10%, so I won't need to wait... Any updates from you, @thyttan?
Just took a look at this, it's a strange one. When I upload your code, I can still use the Bangle, but whenever I physically move it, it tries to load the model and crashes (but the watchdog auto-reboots it). So if it's sitting on the desk it's fine.
But I think the main issue is actually the size of the model. The one in the gesture app is 3616 bytes, but the one in gesturelock is 54768 bytes.
The tensorflow arena size that gets automatically created when handling gestures is 4000 bytes, and I'm not sure but I guess that the 54768 model might use more than that (although it should fail with an error if so).
Did the model ever work for you, or is it just the upload to the app store that causes issues? If it's the app store, please can you upload your model file here, and I'll see if I can spot any difference between what's not on my watch and what you have