blog
blog copied to clipboard
Update 4bit-transformers-bitsandbytes.md
The value ends up being the same but this maybe clearer for the reader: mantisaa bits 1010 we have (2^-1 + 0 + 2^-3 + 0) = (0.5 + 0.125) = 0.625 instead of
mantisaa bits 1010 we have (0 + 2^-1 + 0 + 2^-3 ) = (0.5 + 0.125) = 0.625 Since the mantissa bits are read from left to right and the first bit is 1 it should correspond to 2^-1 and not 0