amber

amber copied to clipboard

When comparing pixel values the absolute errors you care about is strongly determined by the image format. E.g. with 32-bit floats per colour channel, you're in the range of 2**(-23) ULP (machine epsilon for a float). But with an 8-bits per channel, you're in the 1/256 range.

So when doing fuzzy comparisons on pixel values you want the test case to supply the absolute error bound, probably.

When comparing errors for compute purposes, e.g. error on sin(x) you express it as ULP which is a relative error. See https://www.khronos.org/registry/vulkan/specs/1.1/html/vkspec.html#spirvenv-precision-operation and https://en.wikipedia.org/wiki/Unit_in_the_last_place

Basically, the comparison we need to make is context dependent, and controlled by the person writing the test.