Underflow bugfix & fix off-by-one error in the texture height calculation

[hannes-vernooij]

~~It should be max index / max size because 4096 / 4096 + 1 = 2, which means that two rows are allocated when a LUT1D with a size of 4096 fits in a 1D texture of 4096 pixels.~~

EDIT: In the initial commit I made the mistake of not taking into account that the last value of a row is repeated in the first value of the next row, this is now fixed. I validated the result by running a shaper LUT with a size of 4096, 4097, 8191 and 8192. I then applied the transform LUT on syntheticChart.01.exr and validated a 0 pixel difference.

Underflow bugfix

This also seems to solve an existing underflow bug that appeared with a 1D LUT with a size of 8191. When the height is calculated the value is not ceiled, which results in 8191 / 4096 = 2, while it should be 3.

This caused an underflow later in the code in: unsigned long missingEntries = width * height - (unsigned long)paddedChannel.size();

where width * height is less than (unsigned long)paddedChannel.size(); causing an underflow that results in a very high value that is iterated.

[linux-foundation-easycla[bot]]

The committers listed above are authorized under a signed CLA.

• :white_check_mark: login: hannes-vernooij / name: Hannes Vernooij (1cc758223ad7b7cd084ecd20d170b5f3073df991, 9005a123de00e4b78c6e94bf75c43e9f02948770)