KHR_mesh_quantization: Clarify dequantization transform

[abwood]

It took me a few reads of the KHR_mesh_quantization extension to wrap my head around how the quantization is defined for a mesh. If my understanding is correct, the external encoder will insert a dequantization transform to encode a scale and bias; in effect this transformation would map POSITION data into a unit (ish) cube.

If this is correct, I'd recommend this extension illustrates this upfront, with a glTF snippet or diagram showing the dequantization transform node. Here's a quick mockup of something in powerpoint, with made up numbers, that I personally would have found helpful.

[donmccurdy]

/cc @zeux

[zeux]

This is a great illustration! It would make sense to me to add something like this as an example.

Note that the extension is intentionally not specifying the exact method of providing a dequantization transform. This is because this leaves some amount of freedom to the encoder:

• The transform can be specified as a separate node (like in your illustration), or merged into an existing transform of an existing node
• For skinned meshes, the transform could be baked into skin binding matrices, or the bone hierarchy could be extended with extra nodes for dequantization purposes (although this is probably unwise)
• For some content, e.g. block voxel-based scenes, dequantization transform can be omitted when unnormalized accessors are used, if the scene is small
• For some content, if the mesh is already in -1..1 range, dequantization transform could be omitted as well when normalized accessors are used

So if we add this, we should say that this is "one possible example" of encoding quantized meshes or something along the same lines.

[prideout]

On a related note, I have a small comment about a possible ambiguity in the extension spec.

The spec permits non-normalized data for texture coordinates, and it is unclear to me what a non-normalized texture coordinate really means, from a sampling perspective.

For example, a non-normalized unsigned short value of 0x0002 for a U coordinate means either:

1. It maps to a U coordinate of 2.0. If you are wrapping with REPEAT, then interpolation between this and a neighboring 0.0 value would produce two tiles of the texture.

2. Texel coordinate of 2. So, if your texture width is 256, this means 2.0 / 255.0 aka ~0.008.

After thinking about it, I'm pretty sure (2) is what is intended, which I think this could be made more clear since it's a bit different from what "non-normalized" means for other types of data, such as position.

[zeux]

(1) is what is intended in the spec. You can use KHR_texture_transform to correct this to any desired range. I can submit a PR to clarify that (I think this is somewhat independent from this issue that suggests adding an illustration for positional data).

[zeux]

FWIW I understand that integer texture coordinates may seem weird but the motivation behind this extension in general is to decouple storage format from the interpretation a bit more. So when the data array carries integers, they are converted to floating point values according to the format width and the normalized boolean (this conversion can happen entirely on the GPU side), and then treated as a given attribute.

So there's no difference between position 1.0 (float), position 1 (unnormalized integer), position 32767 (normalized signed 16-bit integer), texture coordinate 1.0 (float), texture coordinate 1 (unnormalized integer) and texture coordinate 32767 (normalized signed 16-bit integer) - they are all treated as 1.0f for the purposes of further processing by geometry/texture pipeline (which involves transforming them and then using them for rasterization or interpolation + texture sampling).

[prideout]

That all makes sense! I was perhaps overthinking it wrt texture coordinates.

It probably wouldn't hurt for the spec to include a small warning for implementors like me, but it's not a big deal.