goldeneye icon indicating copy to clipboard operation
goldeneye copied to clipboard

possible bug in block fp implementation

Open a-maci opened this issue 1 year ago • 0 comments

https://github.com/ma3mool/goldeneye/blob/b9f53f1ee92845898b1c081edf7119f02a22aa98/src/num_sys_class.py#L351

After shifting the mantissa bits in the above line, the values might become < 1. So the exponents after reconstruction are no longer the same (one can check this after extracting the exponents after line 363 of the reconstructed tensor).

You probably need mant_adj[mant_adj < 1.0] = 1.0 after the above line.

a-maci avatar Aug 30 '23 23:08 a-maci