wasi-nn
wasi-nn copied to clipboard
Problem with wasi-nn 0.6.0 TensorType
I was running a wasmedge server with inference support cloned from here, but with newer dependencies and was facing an error in the GraphExecutionContext
's set_input
function.
It was an issue with the TensorType
parameter. I was passing U8
but it was causing an error in the function
[error] [WASI-NN] Expect tensor type U8, but got I32
It works in wasi-nn 0.4, 0.5 but breaks in 0.6
Now I almost found the change and reason it is happening, This commit is changing the order of the generated TensorType enum (The new position of U8 is the previous position of I32).
So I tried setting the input with TensorType::F64
and it worked.
I couldn't figure out where it is breaking though. Since these are just the bindings for rust, I would like to know where the implementation of it is. I would love to help if someone can point me in the right direction...
Isn't the redundant move eliminator still useful when dealing with fixed-reg constraints? Since regalloc2 fundamentally can only model a vreg in a single location at a time, multiple fixed constaints can often introduce unnecessary back-and-forth copying which can be eliminated by the redundant move eliminator.
We do support overlapping liveranges now, so perhaps we could address this by allowing the value to exist in multiple places at once?
I benchmarked this PR today and found that it's a pretty significant regression for the execution time of most of our benchmarks. As such, I'm going to close the PR. There could still be a path forward to removing the RME, but it's beginning to feel less worthwhile :)