w3c
Add the Apple `iDOT` chunk to the PNG Extensions specification
Background
Apple have created a public chunk, iDOT, which is in widespread use on macOS, iPadOS and and iOS, particularly for screenshots. Despite being public, it was never registered.
The chunk has been reverse engineered with further details and an Apple engineer has confirmed that the analysis is correct:
I was involved in adding this to PNGs in ~2011 or so. I don't remember the details but you've got this pretty much right.
The reason was indeed performance: on the first retina iPads, decoding PNGs was a huge portion of the total launch times for some apps, while one of the two cores sat completely idle. My guess is that it's still at least somewhat useful today, even with much much faster single-thread performance, because screen sizes have also grown quite a bit.
Problems
The chunk has some design problems: it uses file offsets to point to other chunks, which are known to be problematic and means that it breaks if, for example, a text comment is inserted between iDOT and IDAT; it also breaks if the PNG image data is recompressed, for example by an image optimizer) so we should not add it to the main PNG specification, although the problem of multithreaded decoding that it tries to solve is under active consideration:
- https://github.com/w3c/png/issues/60
- https://github.com/w3c/png/issues/54
However, the chunk is in widespread use and, being ancillary, does not interfere with regular image decoding and viewing software. It is common for people to ask what it is:
- Unanswered question "iDOT PNG chunk" on Apple developer forums
- What is the iDOT chunk on Stack Overflow
Programs like pngcheck report it as an error, because it is an unregistered public chunk.
I confirm that "iDOT" is not a registered PNG chunk. – Glenn Randers-Pehrson Commented Nov 24, 2015
Solution - register it
We have a document listing chunks which are not a part of the main PNG specification:
and this includes chunks which were formally registered, like sTER Indicator of Stereo Image, chunks which were registered but never actually defined, like fRAc Fractal image parameters, chunks which were documented elsewhere, like dSIG Digital signature and even chunks which are deprecated, like gIFt GIF Plain Text Extension.
The non-registered status of iDOT has caused actual problems; recently pngcheck had an issue raised
- https://github.com/pnggroup/pngcheck/issues/49
and associated pull request to treat all unknown ancillary chunks as warnings, not errors specifically to silence errors caused by iDOT.
Therefore, a simple solution is to add iDOT to the PNG Extensions document.
I suggest adding it to the Chunks Not Described Here section, with a pointer to the reverse-engineered description.
Although I can also see the benefits of treating it as a regular registered chunk (the advantage being robustness if the external document goes away; the drawback being that the registration becomes the official definition).
Checkers like pngcheck (which already treat all the extension chunks as valid) can then simply report the existence of the iDOT chunk, rather than treating it as an error.
Note: The iDOT chunk has also been mentioned in passing in connection with another, more problematic Apple chunk, CgBI:
- https://github.com/w3c/png/issues/45
A pngcheck fork exists which has been modified to allow iDOT (and all other unknown ancillary chunks) in response to their issue
- https://github.com/metanorma/pngcheck-ruby/issues/17
@ronaldtse registering iDOT in the extensions document, and thus allowing pngcheck to not error on it, would I think solve the problem you had:
Originally we were using the return code from pngcheck to assert validity of a PNG file for the Metanorma standards publication suite. Since our software does have to accept Apple-created PNGs, we will have to post process the pngcheck error messages to know what kind of error it is.
An online tool, PNG Analyzer (source on GitHub) reports on iDOT-containing PNGs. (source for iDOT handling)
For example:
iDOT [28]
Parallel segments: 2
#1:
rows [0 - 139]
begins at IDAT 0 (file offset 752)
#2:
rows [140 - 279]
begins at IDAT 2 (file offset 28368)
Using this image:
CVE-2016-1811 identifies a bug where a bad iDOT chunk can crash older versions of Apple's ImageIO processor.
A pngcheck fork exists which has been modified to allow
iDOT(and all other unknown ancillary chunks) in response to their issue
@svgeesus yes that is actually our fork! It was worked into https://github.com/pnggroup/pngcheck/pull/51 that was closed.
registering iDOT in the extensions document, and thus allowing pngcheck to not error on it, would I think solve the https://github.com/pnggroup/pngcheck/pull/51#issuecomment-3049314993
Indeed, I guess the question is when would that happen? We install pngcheck as part of our stack on user machines to check for the validity of PNG files, so it is a key component that we have to maintain.
At @metanorma we basically have these choices:
- Upstream that patch into the official pngcheck repository and we can upgrade to pngcheck 4.0 (this culminated into https://github.com/pnggroup/pngcheck/pull/51)
- Maintain our fork using the new pngcheck 4.0 with our patch.
- Wait for pngcheck to officially support
iDOT.
So we have to figure out whether to do 2 or 3...
Using this image:
Ha! As a piece of background information, I screenshotted this particular image as part of the development workflow for ISO/DIS 19135, when I was using ImageOptim to shrink the PNG files of that document for submission to ISO/CS... The latest version is now all in SVG.
I suggest adding it to the Chunks Not Described Here section, with a pointer to the reverse-engineered description.
I would definitely support this suggestion. Thank you @svgeesus ! Let me know how I can help if there's anything I could do.
@svgeesus yes that is actually our fork!
I know. And what is now the official pngroup pngcheck was originally just my own fork, which I clumsily hacked to add support for third edition chunks like mDCV and cICP
Indeed, I guess the question is when would that happen?
Because this doesn't affect the official PNG specification, I expect that to be fairly fast.
And once it is done, it won't take me long to add it to pngcheck either.
As a datapoint, we resolved to upgrade cHRM last week to allow negative chromaticity values, and within two days I had a PR on the spec (which is merged) and a PR on pngcheck (which is being reviewed).
@svgeesus thank you for the amazing work on pngcheck and maintaining it.
Your support of recognizing the chunk and the consequent updates in pngcheck will be very much appreciated by the community 😉
It would definitely be easier for us to wait on the update than to update the fork again..
We also have an example of a four-band iDOT although not, I believe, evidence that any Apple platform will use it to decode on multiple cores.
iDOT [52]
Parallel segments: 4
#1:
rows [0 - 172]
begins at IDAT 0 (file offset 462)
#2:
rows [173 - 345]
begins at IDAT 9 (file offset 144049)
#3:
rows [346 - 518]
begins at IDAT 20 (file offset 321896)
#4:
rows [519 - 691]
begins at IDAT 30 (file offset 484329)
It is in the same boat as the other parallel decoding topics, which I think we'll cover in 5th edition. I'm assigning to that milestone.
We have been cleared to contribute details on iDOT. We'll go through the info you have and see if anything has been missed.
Thank you @svgeesus @leo-barnes @ProgramMax !! Now just need to wait for this to show up in pngcheck...
Now just need to wait for this to show up in pngcheck...
https://github.com/pnggroup/pngcheck/pull/60
Now just need to wait for this to show up in pngcheck...
@ronaldtse It landed today.