seamless icon indicating copy to clipboard operation
seamless copied to clipboard

Documentation: reproducibility and numeric precision

Open sjdv1982 opened this issue 8 months ago • 0 comments

For reproducibility, it is essential:

  • To perform Numpy calculations in float64. Matrix multiplication gives significant irreproducible errors otherwise.
  • To return results in float32. The least significant bits contain random rounding noise that will change checksums. Casting to float32 removes this.

sjdv1982 avatar Jun 17 '24 10:06 sjdv1982