tract
tract copied to clipboard
Decluttering a `DequntizeLinearF32` followed by `QuantizeLinearI8` results in an error at model patching
Quantizing the first part of Tensorflow's model from this tutorial fails when trying to declutter:
cargo run path-to-model/model_quant.onnx -i 1,784,u8
2024-01-10T13:33:10.956561599Z ERROR tract] ModelBuildingError
Caused by:
0: Error at stage declutter
1: running pass declutter
2: declutter node #1 "serving_default_flatten_input:0_dequant" DequantizeLinearF32
3: Trying to substitute a 1,784,I8 by 1,784,U8.
The issue seems to be here:
if !original_fact.compatible_with(new_fact) {
bail!("Trying to substitute a {:?} by {:?}.\n{:?}", original_fact, new_fact, self);
}
Since U8
is not "compatible" with an I8
, the code bails out.
Two very uninformed ideas would be to either:
- implement
compatible_with
forU8
<->I8
(in fact, removing the compatibility check seems to work and replaces the two Dequant, Quant operations with a singleLookupTable
. I'm not sure whether that's the expected behaviour, but looking at the code I think yes). - Call
shunt_outside_unchecked
instead ofshunt_outside
inside theif incoming_dt == DatumType::I8 || incoming_dt == DatumType::U8
condition, where we're guaranteed (?) to have u8/i8 only.
Without decluttering, all works fine:
cargo run path-to-model/model_quant.onnx -i 1,784,u8 --declutter-step 0
Resulting in:
┏ 0 Source serving_default_flatten_input:0
┃ ━━━ 1,784,U8
┣ 1 DequantizeLinearF32 serving_default_flatten_input:0_dequant
┃ ━━━ 1,784,F32
┣ 2 QuantizeLinearI8 tfl.quantize
┃ ━━━ 1,784,I8
┣ 3 DequantizeLinearF32 sequential/flatten/Reshape_dequant
┃ ━━━ 1,784,F32