pcf_font: font larger than file?

[armijnhemel]

I tried the pcf_font.ksy with a few pcf fonts, but the last table consistently is reported as being too big for the file. I have tested with fonts from here:

https://github.com/python-pillow/Pillow/tree/master/Tests/fonts

as well as a few other .pcf fonts I downloaded from https://proggyfonts.net/download/ and the result is the same: bytes seem to be missing

The Web IDE gives me errors similar to:

Parse error: undefined
Call stack: undefined EOFError: requested 100 bytes, but only 72 bytes available

I tested a bit with pillow but it gets the same values as Kaitai, but it seems to ignore the PCF_BDF_ACCELERATORS table completely.

[generalmimon]

The explanation is here (https://github.com/freetype/freetype/blob/09195a82a4a39afb0f8281563f48ce4493455b4e/src/pcf/pcfread.c#L179-L198):

    /*
     * We now check whether the `size' and `offset' values are reasonable:
     * `offset' + `size' must not exceed the stream size.
     *
     * Note, however, that X11's `pcfWriteFont' routine (used by the
     * `bdftopcf' program to create PCF font files) has two special
     * features.
     *
     * - It always assigns the accelerator table a size of 100 bytes in the
     *   TOC, regardless of its real size, which can vary between 34 and 72
     *   bytes.
     *
     * - Due to the way the routine is designed, it ships out the last font
     *   table with its real size, ignoring the TOC's size value.  Since
     *   the TOC size values are always rounded up to a multiple of 4, the
     *   difference can be up to three bytes for all tables except the
     *   accelerator table, for which the difference can be as large as 66
     *   bytes.
     *
     */

This can be easily checked directly in the libXfont source code:

gitlab.freedesktop.org/xorg/lib/libxfont / src/bitmap/pcfwrite.c:271-278:

	case PCF_ACCELERATORS:
	    if (bitmapFont->bitmapExtra->info.inkMetrics)
		table->format = PCF_ACCEL_W_INKBOUNDS | format;
	    else
		table->format = PCF_DEFAULT_FORMAT | format;
	    table->size = 100;
	    table++;
	    break;

gitlab.freedesktop.org/xorg/lib/libxfont / src/bitmap/pcfwrite.c:333-340:

	case PCF_BDF_ACCELERATORS:
	    if (pFont->info.inkMetrics)
		table->format = PCF_ACCEL_W_INKBOUNDS | format;
	    else
		table->format = PCF_DEFAULT_FORMAT | format;
	    table->size = 100;
	    table++;
	    break;

Just as the comment explains, first a real table size is calculated and then it's always rounded up to the nearest multiple of 4. For example (src/bitmap/pcfwrite.c:311-318):

	case PCF_BDF_ENCODINGS:
	    table->format = PCF_DEFAULT_FORMAT | format;
	    nencodings = (pFont->info.lastRow - pFont->info.firstRow + 1) *
		(pFont->info.lastCol - pFont->info.firstCol + 1);
	    size = S32 + 5 * S16 + nencodings * S16;
	    table->size = RoundUp(size);
	    table++;
	    break;

(as you might have guessed, S32 = 4, S16 = 2 and S8 = 1 - see src/bitmap/pcfwrite.c:175-177)

RoundUp is defined like this (src/bitmap/pcfwrite.c:180):

#define RoundUp(s)  (((s) + 3) & ~3)

Rounding the size up essentially just ensures the 4-byte alignment of the start offset of each table. The relevant bit of code assigning the offsets is here (src/bitmap/pcfwrite.c:346-352):

    offset = header_size;
    for (cur_table = 0, table = tables;
	    cur_table < ntables;
	    cur_table++, table++) {
	table->offset = offset;
	offset += table->size;
    }

This is the only use of table->size in the serialization code (other than serializing the table->size itself to a u4le field). The parsing code even completely disregards table->size - it is read from the stream, but then there is not a single check on its value.

However, this routine of rounding the size up to a multiple of 4 creates a problem for the last table, because the ofs_body + len_body then ends up being beyond EOF due to that. So it will be needed to always adjust the last table size, just as with both types of accelerator tables.