bytecodealliance
[WASI-NN] Add support for a PyTorch backend for wasi-nn
This change adds a PyTorch backend for wasi-nn.
tch crate is used for Libtorch bindings. I have added an image classification example to demonstrate its usage, which uses a torchscript model.
This backend is currently gated behind a wasi-nn feature flag --features pytorch as due to dynamic linking, a Libtorch v2.4.0 installation on the system (specified by LIBTORCH=/path/to/libtorch) is needed for building.
The cargo vet situation is a bit much:
cargo vet diff zstd-safe 5.0.1+zstd.1.5.2 5.0.2+zstd.1.5.2
gyscos zstd 2 files changed, 4 insertions(+), 4 deletions(-)
cargo vet diff zstd 0.11.1+zstd.1.5.2 0.11.2+zstd.1.5.2
gyscos zip 3 files changed, 5 insertions(+), 5 deletions(-)
cargo vet diff num-complex 0.4.2 0.4.6 cuviper ndarray 6 files changed, 188 insertions(+), 48 deletions(-)
NOTE: this project trusts Josh Stone (cuviper) - consider cargo vet trust num-complex or cargo vet trust --all cuviper
cargo vet inspect constant_time_eq 0.1.5 cesarb zip 311 lines
cargo vet diff sha1 0.10.5 0.10.6 newpavlov zip 7 files changed, 302 insertions(+), 20 deletions(-)
cargo vet inspect rawpointer 0.2.1 bluss ndarray and matrixmultiply 559 lines
cargo vet diff zip 0.6.4 0.6.6 Plecra tch and torch-sys 14 files changed, 604 insertions(+), 109 deletions(-)
cargo vet inspect inout 0.1.3 newpavlov cipher 1112 lines
NOTE: cargo vet import zcash would eliminate this
cargo vet inspect pbkdf2 0.9.0 tarcieri zip 1120 lines
cargo vet inspect bzip2 0.4.4 alexcrichton zip 2094 lines
NOTE: this project trusts Alex Crichton (alexcrichton) - consider cargo vet trust bzip2 or cargo vet trust --all alexcrichton
cargo vet inspect safetensors 0.3.3 Narsil tch 2200 lines
cargo vet inspect cipher 0.4.4 newpavlov aes 2635 lines
NOTE: cargo vet import zcash would reduce this to a [130](https://github.com/bytecodealliance/wasmtime/actions/runs/10836457564/job/30070281197?pr=9234#step:6:131)0-line diff
cargo vet inspect password-hash 0.3.2 tarcieri pbkdf2 3139 lines
cargo vet inspect base64ct 1.6.0 tarcieri password-hash 3381 lines
cargo vet diff half 1.8.2 2.4.1 starkat99 tch 19 files changed, 2546 insertions(+), 958 deletions(-)
cargo vet inspect time 0.1.44 jhpratt zip 3915 lines
cargo vet inspect aes 0.7.5 tarcieri zip 6822 lines
cargo vet inspect matrixmultiply 0.3.8 bluss ndarray 7934 lines
cargo vet inspect ndarray 0.15.6 jturner314 tch 41996 lines
cargo vet inspect torch-sys 0.17.0 LaurentMazare tch 52119 lines
cargo vet inspect bzip2-sys 0.1.11+1.0.8 alexcrichton bzip2 264[133](https://github.com/bytecodealliance/wasmtime/actions/runs/10836457564/job/30070281197?pr=9234#step:6:134) lines
NOTE: this project trusts Alex Crichton (alexcrichton) - consider cargo vet trust bzip2-sys or cargo vet trust --all alexcrichton
cargo vet inspect tch 0.17.0 LaurentMazare wasmtime-wasi-nn 2287297 lines
This is a good start. The main thing to fix is the handling of the input and output tensors.
Thanks for the review, Andrew. I've marked smaller Nits as resolved, and I've addressed other comments as well, but kept them 'unresolved' as of now until you take a look.
I rolled back 2 commits, but there's an issue with the lock file. I had previously attempted to fix this by deleting my lockfile, rebasing off of latest main, and then doing a cargo build to generate any of my lock file changes on top of that.
The failing tests fail due to
error: failed retrieving file 'mingw-w64-x86_64-headers-git-12.0.0.r329.g8f7b5ce36-1-any.pkg.tar.zst.sig' from mirror.clarkson.edu : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds
mingw-w64-x86_64-gcc-14.2.0-1-any downloading...
error: failed retrieving file 'mingw-w64-x86_64-mpfr-4.2.1-2-any.pkg.tar.zst' from mirror.msys2.org : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds
I'm hoping a rerun of CI would help.