Suboptimal BasisLZ Global Data Layout

[lexaknyazev]

The current layout of BasisLZ Global Data prevents eager decoder initialization because its variable-length imageDescs array is located in the middle and depends on the prior knowledge of the whole KTX configuration. We should revise it in the future revisions of BasisLZ.

By relocating imageDescs to the end of supercompressionGlobalData section, loaders would be able to do the following:

1. Upon encountering sgdByteOffset, immediately jump (e.g. using a separate thread) to it.
2. Use the contiguous block of data that would start there to initialize BasisLZ endpoints/selectors/tables even before the levels/faces/etc configuration is known.

[lexaknyazev]

Another rationale. Endpoints, selectors, and tables are internal implementation details of the BasisLZ codec and have no meaning in the broad KTX context. On the other hand, slice offsets are required for understanding KTX-wide layout, so those two blocks of data should not be interleaved.

As a workaround, optimized implementations could do the following:

1. Locate start / end offsets of supercompressionGlobalData and pass the whole block to the BasisLZ decoder.
2. Parse initial headers (endpointCount .. extendedByteLength ).
3. Subtract the sum of BasisLZ payload lengths from the end offset of the supercompressionGlobalData block.

[MarkCallow]

I don't understand what you mean by "eager decoder initialization." I also don't see how you could encounter sgdByteOffset before knowing the "levels/faces/etc configuration" as that information is in the header before sgdByteOffset.

The sliceOffsets are only needed for transcoding BasisLZ images. They are an index to the individual images of a level and are not necessary for understanding KTX-wide layout.

If we do come to an agreement on useful changes to the layout, these could be incorporated in a new supercompression scheme that we may introduce for global codebooks.

[lexaknyazev]

By "eager decoder initialization" I mean decompressing Huffman tables. They are completely self-contained and could be decoded in parallel even before anything else is ready.

By "knowing levels/faces/etc configuration" I mean computing imageCount to calculate the offset of endpointsData. In my opinion, this dependency gives no benefits as without it a single pointer to supercompressionGlobalData would be enough for the decoder to start its initialization.

They are an index to the individual images of a level and are not necessary for understanding KTX-wide layout.

Compared to an asset without supercompression, this index is necessary to understand where e.g. each cube face starts. So I'd argue that it contributes to KTX-wide layout. At the same time, this index in its completeness has lesser value for low-level Basis transcoder as the latter doesn't care much about semantics of each slice (except color/alpha designation and I/P flags).

Should we introduce global codebooks, this index would be entirely omitted from the asset containing only the codebooks.