rust-wasm-ai-demo
rust-wasm-ai-demo copied to clipboard
second-state
Is this code supposed to work?
MacOS catalina.
- Does not work from VSCode at all (does not even launch the app.js)
- Followed your how to (docker build, wasm build, node run..)
Result (expected like so):
root@e158f954ea64:/app/node# node test.js
/node_modules/ssvm/index.js:7
process.dlopen(module, binding_path,
^
Error: libLLVM-10.so.1: cannot open shared object file: No such file or directory
at Object.<anonymous> (/node_modules/ssvm/index.js:7:9)
at Module._compile (internal/modules/cjs/loader.js:1063:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
at Module.load (internal/modules/cjs/loader.js:928:32)
at Function.Module._load (internal/modules/cjs/loader.js:769:14)
at Module.require (internal/modules/cjs/loader.js:952:19)
at require (internal/modules/cjs/helpers.js:88:18)
at Object.<anonymous> (/app/pkg/csdn_ai_demo_lib.js:17:14)
at Module._compile (internal/modules/cjs/loader.js:1063:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
root@e158f954ea64:/app/node#
I had to manually install llvm again in order to make it work
apt-get install llvm
Well, it doesn't work. It just hangs.
root@7b283e894e71:/app/node# node server.js
Listening at http://0.0.0.0:8080
Received cat.png with size: 4881866
Ok, let's forget about
Hello, We have another (and much faster way) to do tensorflow in SSVM these days. Please checkout this:
https://www.secondstate.io/articles/wasi-tensorflow/
On Wed, Dec 23, 2020 at 11:42 PM neilyoung [email protected] wrote:
Well, it doesn't work. It just hangs.
root@7b283e894e71:/app/node# node server.js Listening at http://0.0.0.0:8080 Received cat.png with size: 4881866
Ok, let's forget about
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/second-state/rust-wasm-ai-demo/issues/4#issuecomment-750355775, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACUNSIVHOZZ5IYHGOFXMLSWIFXRANCNFSM4VG6BSLQ .
--
Michael Yuan, PhD Co-founder at Second State https://www.secondstate.io/ Fast, safe, portable & serverless Rust functions in Node.js https://cloud.secondstate.io/server-side-webassembly/getting-started Interested in Rust? Learn by examples https://rust-by-example-ext.com/!
Will consider that. Do you have any experience for how many fps you are able to achieve if you feed this algorithm with life video? How does it compare to TF Lite on a Coral TPU on a Raspberry Pi 4B?
It depends on the underlying hardware that runs the FaaS service. We can do full GPU. But the current public FaaS service we have available is deployed on a CPU-only instance — it takes about 300ms to run inference on an image.
Cheers Michael
On Thu, Dec 24, 2020 at 1:48 AM neilyoung [email protected] wrote:
Will consider that. Do you have any experience for how many fps you are able to achieve if you feed this algorithm with life video? How does it compare to TF Lite on a Coral TPU on a Raspberry Pi 4B?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/second-state/rust-wasm-ai-demo/issues/4#issuecomment-750406762, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACUNUYNFPVYFRNPIK35WDSWIUQRANCNFSM4VG6BSLQ .
-- Michael Yuan, PhD Fast, safe, portable and serverless Rust programs on Node.js, or as a FaaS http://SecondState.io http://cloud.secondstate.io
Ok, that would render to 3fps.
I think I would give your GPU version a try once I have a Jetson Nano
Thanks. I also have a Jetson Nano on back order. :) We would love to hear your feedback. Just make sure that you use the SSVM wasi for Tensorflow API in Rust.
This repo runs TF models in interpreter mode and is very slow.
On Thu, Dec 24, 2020 at 3:40 AM neilyoung [email protected] wrote:
Ok, that would render to 3fps.
I think I would give your GPU version a try once I have a Jetson Nano
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/second-state/rust-wasm-ai-demo/issues/4#issuecomment-750445023, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACUNWET7VSMJZMZ47SE5LSWJBUJANCNFSM4VG6BSLQ .
-- Michael Yuan, PhD Fast, safe, portable and serverless Rust programs on Node.js, or as a FaaS http://SecondState.io http://cloud.secondstate.io
Also with the starter kit the same result. The test.js (adapted to what the wa-ai-demo does) just hangs. The Rust lib is invoked, but does not return.
I spread some println! in order to see where it hangs, but it seems, infer_impl is not even invoked.
Also, the ssvmup build is not practicable, at least not with TF. This alone takes 5 minutes on my MBP. The longest time is spent in the optimization step. Is there a way to drop the optimization step during debugging?
root@0eb47cbfef94:/app# ssvmup build
[INFO]: Checking for the Wasm target...
[INFO]: Compiling to Wasm...
Finished release [optimized] target(s) in 0.59s
[INFO]: Installing wasm-bindgen...
[INFO]: Optimizing wasm binaries with `wasm-opt`...
[INFO]: :-) Done in 4m 29s
[INFO]: :-) Your wasm pkg is ready to publish at /app/pkg.
I can't drink that much coffee as I have to wait after making a simple change to the Rust code :)
Can you create an issue in the repo for the code example you adapted from? It sounds like something in NodeJS is mis-configured.
Did you try to do this inside Docker Ubuntu 20.04 on an MBP? Mine does not take nearly as long ...
Cheers Michael
On Thu, Dec 24, 2020 at 7:55 PM neilyoung [email protected] wrote:
Also with the starter kit the same result. The test.js (adapted to what the wa-ai-demo does) just hangs. The Rust lib is invoked, but does not return.
I spread some println! in order to see where it hangs, but it seems, infer_impl is not even invoked.
Also, the ssvmup build is not practicable, at least not with TF. This alone takes 5 minutes on my MBP. The longest time is spent in the optimization step. Is there a way to drop the optimization step during debugging?
root@0eb47cbfef94:/app# ssvmup build [INFO]: Checking for the Wasm target... [INFO]: Compiling to Wasm... Finished release [optimized] target(s) in 0.59s [INFO]: Installing wasm-bindgen... [INFO]: Optimizing wasm binaries with
wasm-opt... [INFO]: :-) Done in 4m 29s [INFO]: :-) Your wasm pkg is ready to publish at /app/pkg.I can't drink that much coffee as I have to wait after making a simple change to the Rust code :)
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/second-state/rust-wasm-ai-demo/issues/4#issuecomment-750861623, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACUNWVGNJBNE2TZZ34MKDSWMT3BANCNFSM4VG6BSLQ .
-- Michael Yuan, PhD Fast, safe, portable and serverless Rust programs on Node.js, or as a FaaS http://SecondState.io http://cloud.secondstate.io
Yes, maybe later. And yes, I did it in the docker container created from docker pull secondstate/ssvm-nodejs-starter.
The starter worked well, but then I added all the tensorflow stuff from the other sample.
I find it interesting, but it is not that important for me. Just the Rust/Tensorflow/TPU thing is what I'm mostly interested in.
Thanks for now. It's Christmas now :)
Oh, it returned... 30 minutes later :)
Detected object id 654 with probability 0.3256046
EDIT: Actually 4 minutes later...:)
Hmm. It sounds like you are running the Rust tract crate code to drive the TF model. It could be very slow. I would highly recommend using the WASI tensorflow examples.
https://www.secondstate.io/articles/wasi-tensorflow/
https://www.secondstate.io/articles/faas-image-classification/
Here is a demo running a small sized Azure instance.
https://second-state.github.io/wasm-learning/faas/mobilenet/html/index.html
Merry Christmas!
Cheers Michael
On Thu, Dec 24, 2020 at 9:30 PM neilyoung [email protected] wrote:
Oh, it returned... 30 minutes later :)
Detected object id 654 with probability 0.3256046
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/second-state/rust-wasm-ai-demo/issues/4#issuecomment-750883818, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACUNRRYP6N4DCSZAW5SYTSWM67NANCNFSM4VG6BSLQ .
-- Michael Yuan, PhD Fast, safe, portable and serverless Rust programs on Node.js, or as a FaaS http://SecondState.io http://cloud.secondstate.io
Thanks. I just quickly checked the web demo. It took 43 seconds to detect this from the cute cat image:
["tiger cat","medium"]
The actual execution time on the server is typically 0.5s or less. I think much of time is spent on uploading the photo.
Cheers Michael
On Thu, Dec 24, 2020 at 9:56 PM neilyoung [email protected] wrote:
Thanks. I just quickly checked the web demo. It took 43 seconds to detect this from the cute cat image:
["tiger cat","medium"]
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/second-state/rust-wasm-ai-demo/issues/4#issuecomment-750890104, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACUNTIXHMZDIPMMRJJRLDSWNCCTANCNFSM4VG6BSLQ .
-- Michael Yuan, PhD Fast, safe, portable and serverless Rust programs on Node.js, or as a FaaS http://SecondState.io http://cloud.secondstate.io
It starts uploading when you clicks on the detection button.
Cheers Michael
On Thu, Dec 24, 2020 at 10:51 PM neilyoung [email protected] wrote:
43 s after hitting the detection button
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/second-state/rust-wasm-ai-demo/issues/4#issuecomment-750902993, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACUNQJEWX35PWVHL2IE7LSWNIQ5ANCNFSM4VG6BSLQ .
-- Michael Yuan, PhD Fast, safe, portable and serverless Rust programs on Node.js, or as a FaaS http://SecondState.io http://cloud.secondstate.io
I doubt it. I have an upload of 50 MBit/s. I tested with several other servers in the world. Uploading cat.png takes 2-3 seconds.
Anyway, don't bother. I think I need something else :)
That’s cool. You can also run it locally in nodejs.
https://github.com/second-state/wasm-learning/tree/master/faas/mobilenet
On my end, the uploading takes 1-2 seconds and the inference takes 0.5 second. ;)
Cheers Michael
On Fri, Dec 25, 2020 at 4:43 PM neilyoung [email protected] wrote:
I doubt it. I have an upload of 50 MBit/s. I tested with several other servers in the world. Uploading cat.png takes 2-3 seconds.
Anyway, don't bother. I think I need something else :)
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/second-state/rust-wasm-ai-demo/issues/4#issuecomment-751210653, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACUNQRITNF344UW2YDY6LSWRGEFANCNFSM4VG6BSLQ .
-- Michael Yuan, PhD Fast, safe, portable and serverless Rust programs on Node.js, or as a FaaS http://SecondState.io http://cloud.secondstate.io