sRGB->linear differences between OCIO versions 1 and 2

[johnhaddon]

When upgrading to OCIO 2, we've hit some failures in our internal test suite, comparing images loaded with OIIO/OCIO vs expected results. This seems to be due to precision differences in the sRGB->linear conversion, with 1.0 now yielding 1.0000001192092896 rather than 1.0 exactly. Steps to reproduce :

1. Use the config from https://github.com/imageworks/OpenColorIO-Configs/blob/master/nuke-default/config.ocio.
2. Run the following Python snippet, which should fail with an assertion error :

import PyOpenColorIO

config = PyOpenColorIO.GetCurrentConfig()
processor = config.getProcessor( "sRGB", "linear" )
if hasattr( processor, "getDefaultCPUProcessor" ) :
	# OCIO 2
	processor = processor.getDefaultCPUProcessor()

assert( processor.applyRGB( [ 1.0, 1.0, 1.0 ] ) == [ 1.0, 1.0, 1.0 ] )

This fails with OpenColorIO 2.1.1 but passes with OpenColorIO 1.0.9.

Is this a bug, or can we just not expect the same output in the two different versions?

[doug-walker]

@johnhaddon , it generally is not good practice to use "==" to compare floating-point numbers. Float values should normally be compared to within some tolerance. Often this tolerance is based on the notion of an ULP which is basically the difference between adjacent floating point numbers. (There is lot of info elsewhere on the web about this topic.)

OCIO does not provide double-precision pixel processing, so the values 1.0 and 1.0000001192092896 are, for all practical purposes, identical.

[johnhaddon]

Hi @doug-walker, thanks for your reply. I'm aware of the general rule regarding floating point comparisons, but I must admit to being a little surprised at the results for this specific case. It's a simple transform, and I'm not aware of any other implementations with this result (although I've only recently tested OIIO's fallback and OCIO 1). We've added a little wiggle room to our tests now, so feel free to close if my expectations are unreasonable.

[SonyDennisAdams]

In 32-bit floating point 1.0000001192092896 (0x3F800001) is exactly (to double precision at least) one ULP from 1.0 (0x3F800000), which is another way of saying it is the next 32-bit floating point number after 1.0. Depending on what math functions are being used, 1 ULP is not unexpected.