glow
glow copied to clipboard
pytorch
Fuse RescaleQuantized in the ChannelwiseQuantizedConv
In this case the RescaleQuantized node should be fused within the ChannelwiseQuantizedConv node output.
This fusing type should be refactored in GraphOptz in order to enable it more easily for other nodes as well (preferably with 1 line of code).
@mciprian13 Besides this case I can think of QuantizedConv and QuantizedNode. Is there any node else that seems obvious to add?
@842974287 For generality, I think the best approach here would be to fuse the RescaleQuantized in all the nodes with the exception of those nodes which have constraints regarding the in/out quantization parameters (which can be found in Quantization.cpp). Rule of thumb is that the final graph should not contain RescaleQuantize nodes at all since any requantization is detrimental to accuracy.
@mciprian13
those nodes which have constraints regarding the in/out quantization parameters (which can be found in Quantization.cpp)
Sorry I couldn't find the right constraints in Quantization.cpp. I did find a verify function in VerifierHelper.cpp https://github.com/pytorch/glow/blob/565e798d1b1c89d13a6dab06734dbc4acb9bff73/lib/Graph/VerifierHelper.cpp#L119 that verifies if the input and output are of same scale and offset if the node is quantized. But this verify function is called manually in Nodes.cpp to verify if the node is valid, which makes it less helpful. It would be super helpful if you can point me to the constraint in Quantization.cpp. Thanks a lot!
@842974287 Macro defined there used for nodes with should have same input/output quantization parameters: https://github.com/pytorch/glow/blob/cf3a95aed62bbe97391c2a925f07bbeefd4d0430/lib/Quantization/Quantization.cpp#L503-L504 and: https://github.com/pytorch/glow/blob/cf3a95aed62bbe97391c2a925f07bbeefd4d0430/lib/Quantization/Quantization.cpp#L398-L405 So what it means is that the RescaleQuantized node should be merged in all the nodes except those.
I want to help with this issue. Is there any clarity on whether this needs to be fixed or not? It would be great to have some guidance on how to proceed if needed. And could you please provide some insights into the significance of this issue as well?
