sonos
QLinearAdd support
Hi,
I don't know anything ML... but I've been using Tract as part of a benchmark suite to look at WebAssembly performance. Today I've tried running quantized models, but I got an error:
Translating node #61 "mobilenetv20_features_linearbottleneck2_elemwise_add0_quant" Unimplemented(QLinearAdd) ToTypedTranslator
I can see from the README that QLinearAdd is indeed not listed as a supported operator, are there any plans to add it?
If all the supported quantized operators begin with 'Q', there doesn't seem to be many, or does ONNX simply not provide many? I guess my real question is: should I be bothering to run quantized models, with integer arithmetic, with Tract? If so, any pointers would be greatly appreciated.
Cheers! Sam
Hello !
QLinearAdd is not an ONNX operator. See https://onnx.ai/onnx/operators/ for the list.
Microsoft ONNXRuntime implements it as an extension, and there is an ongoing request to bring it to ONNX (see https://github.com/onnx/onnx/issues/5895 )
At this current point in time, there is no concerted effort to implements Microsoft extensions in tract... That said, depending on the actual semantics of these extensions, it is quite likely that they are already somewhat implemented in tract-core in which case it would just be a matter of mapping the ONNX extension to some existing code in tract-core.
Ah! I used the quantize script from onnx-runtime, so makes more sense. I can see I can control which operators are quantized, so I will try with just MatMul and Conv.
Many thanks!