entity-fishing
entity-fishing copied to clipboard
Add a normalization after embeddings quantization
The quantisation of the embeddings (#46) wasn't correcting mapping the weight on values from -127 to +127, so it was choosen to store them into shorts instead of bytes.
With byte storage we would save more disk. It's anyway good idea to do a general review to understand why we have such problem with weight quantisation.
Quantization has no problem and is doing what is expected, there is requirement of having everything in the interval [-128, 127], what was not correct is that I thought everything was in this range and we could use byte for storing quantized embeddings.
What is needed is a projection on this range of the actual quantized embeddings as additional step, but I am not sure which distribution function to use.
I update the title of the issue to reflect this.