wasi-nn icon indicating copy to clipboard operation
wasi-nn copied to clipboard

Problem with wasi-nn 0.6.0 TensorType

Open Scramjet911 opened this issue 10 months ago • 1 comments

I was running a wasmedge server with inference support cloned from here, but with newer dependencies and was facing an error in the GraphExecutionContext's set_input function.
It was an issue with the TensorType parameter. I was passing U8 but it was causing an error in the function [error] [WASI-NN] Expect tensor type U8, but got I32

It works in wasi-nn 0.4, 0.5 but breaks in 0.6 Now I almost found the change and reason it is happening, This commit is changing the order of the generated TensorType enum (The new position of U8 is the previous position of I32). So I tried setting the input with TensorType::F64 and it worked.

I couldn't figure out where it is breaking though. Since these are just the bindings for rust, I would like to know where the implementation of it is. I would love to help if someone can point me in the right direction...

Scramjet911 avatar Apr 22 '24 18:04 Scramjet911

Isn't the redundant move eliminator still useful when dealing with fixed-reg constraints? Since regalloc2 fundamentally can only model a vreg in a single location at a time, multiple fixed constaints can often introduce unnecessary back-and-forth copying which can be eliminated by the redundant move eliminator.

Amanieu avatar Dec 06 '23 01:12 Amanieu

We do support overlapping liveranges now, so perhaps we could address this by allowing the value to exist in multiple places at once?

elliottt avatar Dec 06 '23 01:12 elliottt

I benchmarked this PR today and found that it's a pretty significant regression for the execution time of most of our benchmarks. As such, I'm going to close the PR. There could still be a path forward to removing the RME, but it's beginning to feel less worthwhile :)

elliottt avatar Jul 04 '24 00:07 elliottt