xla
xla copied to clipboard
Express tests tolerances for exhaustive tests relative to the accuracy and subnormal boundary of the data type.
trafficstars
Express tests tolerances for exhaustive tests relative to the accuracy and subnormal boundary of the data type.
The following table shows the default absolute and relative error tolerances for each numeric datatype.
| Type | atol before | rtol before | atol after | rtol after |
|---|---|---|---|---|
| double | 1e-4 | 1e-4 | 4.45e-308 | 1.11e-14 |
| float | 1e-4 | 1e-4 | 2.35e-38 | 5.96e-6 |
| float16 | 1e-3 | 1e-3 | 1.22e-4 | 5e-4 |
| bfloat16 | 2e-3 | 2e-2 | 2.35e-38 | 1.6e-2 |
| complex |
1e-4 | 1e-4 | 4.45e-308 | 1.11e-14 |
| complex |
1e-4 | 1e-4 | 2.35e-38 | 5.96e-6 |