Incremental-Network-Quantization
Incremental-Network-Quantization copied to clipboard
AojunZhou
bit-width adjustment Question.
Is it possible to adjust the bit-width by changing the number 7?
In power2.cpp
template <typename Dtype> double weightCluster_zero( Dtype weight, int M) { double min=100; double ind=0; double flag=1.0; if(min>std::abs(weight)) { min=std::abs(weight); flag=0.0; }
for(int i=(M-7);i<=M;i++)
{
if(min>std::abs(weight-pow(2,i)))
{
min=std::abs(weight-pow(2,i));
ind=i;
flag=1.0;
}
if(min>std::abs(weight+pow(2,i)))
{
min=std::abs(weight+pow(2,i));
ind=i;
flag=-1.0;
}
}
return flag*pow(2,ind);
}
it looks like this equation [ Wl(i, j) = βsgn(Wl(i, j)) if(α+β)/2≤abs(Wl(i, j))<3β/2 ]
@TwistedfateKing yes, the number 7 (default) is corresponding to 5 bits in paper, you can modify it, 3 for 4 bits, 1 for 3 bits, 0 for 2 bits.
if i want to train 16 bit or 8 bit，how to set the number？Thanks i compute the number (eg. -7) by 1-2^(5-2), Is that correct?
n2 = n1 + 1 −2^(b−1)/2. For instance, if b = 3 and n1 = −1, it is easy to get n2 = −2, if b=5, n2=-1+1-(2^(5-1))/2=-8
I set “for(int i=(M-63);i<=M;i++)” for 8bit INQ model。 finally，i get weight value from 2^-12 to 2^-2 in 1st conv layer,any thing wrong with my setting?
@Zhouaojun
Hi, Thanks for sharing the awesome code. About the log2 represent of the filter, I have a problem. In the default 5 bit filter, does 1 bit for the sign of the value, one bit for whether a zero and the other 3 bits for exponent? If we use 1 represent negative number and 0 for positive; 0 for not zero, 1 for zero, then the following number is +2^7 5'b0(sign) 0(zero) 111 (exp)
and 5'bx1XXX is always zero, 5b'10101 represent -2^5.
Is my understanding correct?
