blog icon indicating copy to clipboard operation
blog copied to clipboard

Update 4bit-transformers-bitsandbytes.md

Open rekil156 opened this issue 2 years ago • 0 comments

The value ends up being the same but this maybe clearer for the reader: mantisaa bits 1010 we have (2^-1 + 0 + 2^-3 + 0) = (0.5 + 0.125) = 0.625 instead of mantisaa bits 1010 we have (0 + 2^-1 + 0 + 2^-3 ) = (0.5 + 0.125) = 0.625 Since the mantissa bits are read from left to right and the first bit is 1 it should correspond to 2^-1 and not 0

rekil156 avatar Jun 04 '23 22:06 rekil156