[circle-quantizer] QoL improvement: Warnings for poorly representable weights

[AShedko]

What?

It would be nice to issue a warning when a weight (this includes any learned parameters, such as bias values, mean and variance in batch norm, etc.) is poorly represented with a given quantization scheme.

Why?

Warnings could help debug accuracy problems in compiled models and shorten the time needed to find a bug or issue.

How?

We can check the range of orders of magnitude of values in a tensor and if it is significantly larger than the number of bits in a given quantization, issue a warning.

Example

Consider the following pseudocode

A = np.array([300000,30001,0.8e-10])  # Example weight
r = np.log2(A.max())-np.log2(A.min()) # = 51.73581201891895 range of binary orders of magnitude
thresh = 1.5 # Threshold to decrease the number of false-positive warnings
nbits = 16 # for int16, 8 for uint8
if (r > nbits * thresh):
    print("""Weight A is not faithfully represented in the quantized model,
             the resulting model's accuracy may suffer""")

This can be implemented as a generic checker which is run for all nodes with CircleConst type.

[seanshpark]

CC @jinevening