bitmap format big & little endian in image representations

[rmottola]

We should ensure we write & read the endian format property to all file types we support. If a specific library or format do not support a specific format, byteswapping should happen before.

Most importantly TIFF is ignoring the flag

http://www.simplesystems.org/libtiff/functions/TIFFOpen.html#byte-order

Reading is automatic, writing happens native if not otherwise specified.. Question: the byte is set for file, not image. Will they all be coherent?

[fredkiefer]

I don't understand what you mean by Most importantly TIFF is ignoring the flag. The documentation says otherwise or at least I read it differently. But it is really questionable what the tiff library will do if we specify that value. Most likely it will still expect that the given data is in host byte order and just write it differently, when the flag is given. This won't help us. What we should do is byte swap the data before handing it over to libtiff, when the NSBitmapFormatSixteenBitBigEndian flag is set. We also need to think about NSBitmapFormatThirtyTwoBitBigEndian for 32 bit data.

[rmottola]

I don't understand what you mean by Most importantly TIFF is ignoring the flag. The documentation says otherwise or at least I read it differently.

I just meant among our supported formats the most important to fix is TIFF because it is our "main" format and that GNUstep's TIFF handling is ignoring the flag - no implication for the TIFF library. I wrote the issue quickly lat at night not to forget our hacking session findings.

But it is really questionable what the tiff library will do if we specify that value. Most likely it will still expect that the given data is in host byte order and just write it differently, when the flag is given. This won't help us. I am unsure about that interpretation, while rereading it today. It is explicit that libtiff will write by default in the host byte order. It also specifies that by appending to an existing file (I think this can only happen if one is adding multiple images to a a multi-page file) it will "preserve" the file byte order. That can work reliably only if the library can do the byteswapping itself. So it might work!

What we should do is byte swap the data before handing it over to libtiff, when the NSBitmapFormatSixteenBitBigEndian flag is set. We also need to think about NSBitmapFormatThirtyTwoBitBigEndian for 32 bit data.

Maybe libtiff can do the swapping, from what I understand rereading the doc. However, there is a certain limitation while doing to: byte order is for the tiff file not for each tiff image. Also it would avoid a situation where a file with X byte order is read and saved by an application and automatically "converted" to Y if is running on host with byte order Y and not X. Not a big harm, but somehow unclean.

We should write some tests. I will start with some local ones and see if they can be made gui tests, depending on the implementation we follow

[rmottola]

@fredkiefer do you think we are done with just the TIFF fixes or are there other libraries to check? I don't think, since support applies only to > 8bit depths and PNG is big-endian by definition and sets it, and TIFF we fixed, I don't see something else left.