Tiff color table component values seem wrong (minus one from what they should be)

[slesim]

What is the bug?

Using gdalinfo on an 8bits indexed tiff file. RGB component values returned (different from 0) have a minus one shift from what they should be :

0: 0,0,0,255 
1: 0,0,0,255 
2: 0,0,0,255 
3: 0,0,0,255 
4: 0,0,0,255 
5: 0,0,0,255 
6: 0,0,0,255 
7: 0,0,0,255 
8: 0,0,0,255 
9: 0,0,0,255 
10: 0,0,0,255 
11: 169,239,239,255 
12: 0,0,0,255 
13: 0,0,0,255 
14: 254,254,99,255 
15: 0,0,0,255 
16: 0,0,0,255 
17: 0,0,0,255 
18: 0,0,0,255 
19: 0,0,0,255 
20: 219,239,99,255 

Using other library like python tifffile, or .NET BitMiracle.LibTiff.Net :

0 : 0 0 0       
1 : 0 0 0       
2 : 0 0 0       
3 : 0 0 0       
4 : 0 0 0       
5 : 0 0 0       
6 : 0 0 0       
7 : 0 0 0       
8 : 0 0 0       
9 : 0 0 0       
10 : 0 0 0      
11 : 170 240 240
12 : 0 0 0      
13 : 0 0 0      
14 : 255 255 100
15 : 0 0 0      
16 : 0 0 0      
17 : 0 0 0      
18 : 0 0 0      
19 : 0 0 0      
20 : 220 240 100

Steps to reproduce the issue

Sample code to compare it with python tifffile :

from tifffile import TiffFile

with TiffFile('sample.tif', mode='r+') as tif:
    tag = tif.pages[0].tags['ColorMap']
    r, g, b = tag.value[0], tag.value[1], tag.value[2]
    for i in range(len(r)):
        print("%d : %d %d %d" % (i, int.from_bytes(r[i]), int.from_bytes(g[i]), int.from_bytes(b[i])))

Versions and provenance

Confirmed with 3.8.5, 3.7.2, 3.4.2. I don't have checked but we don't have noticed this issue in older version of our software using 1.9.2.

Additional context

No response

[rouault]

@slesim Can you attach such a file ? I believe the root cause is that TIFF color table entries are expressed as values in the [0,65535] range, and not [0,255], so software has to do remapping. GDAL uses division by 257 and truncation to do the conversion from [0,65535] to [0,255]. Other software might have slightly different approaches. The TIFF specification doesn't specify how to do that remapping

Current behaviour dates back to this changeset eeec5b62e385d53e7f2edaba7b73c7c74bc2af39 that went in GDAL 2.3.0. Previously division by 256 was done.

[cgohlke]

Tifffile does not do any conversion or remapping. tif.pages[0].tags['ColorMap'].value is a numpy array of dtype uint16.

The conversion in the example code is int.from_bytes(r[i]), which is wrong in general but happens to give expected results for multiples of 256.

[slesim]

Thanks for your answers, it helps my understanding. Here is a sample file : rugoIndex.zip