KhronosGroup
KHR_accessor_float64 and KHR_accessor_float16 Draft Proposals
Split from #2395.
As discussed in the Interactivity DTSG, it's important for the animation pointer data to be able to represent the same Object Model property values that could be set via behavior graphs.
How does this extension interact with mesh compression extensions like Draco or Meshopt?
For Draco, here's an (old) quote...
- https://github.com/google/draco/issues/360
... so it doesn't sound like f64 is supported. Draco's "edgebreaker" codec will change vertex count and order, so compressing only a subset of a mesh primitive's attributes must be avoided. The "sequential" codec does not change vertex count if deduplication is disabled using the "Expert Encoder API". So with some work, f64 attributes could be skipped during compression. Perhaps support for compressing f64 attributes could be added to the Draco library without changes to KHR_draco_mesh_compression, I'm not sure.
For Meshopt, I don't think there's support for f64 in the library today, but vertex count and order does not change during compression, so excluding specific vertex attributes from compression is easy. That's probably what I'd do in glTF Transform, just skip compression for f64 attributes. I assume that adding f64 support to EXT_meshopt_compression is technically possible, but not trivial:
- https://github.com/zeux/meshoptimizer/issues/2#issuecomment-806359065
Not to say that KHR_accessor_float64 should prohibit these compression methods. But there are technical obstacles.
In theory, if one is using f64 for precision, one should not switch to quantized, right? That's less precision than f32.
I tend to think of the component type's precision only as an upper bound on quantization precision. It's common to quantize POSITION accessors to 12–14 bits, stored in 16-bit integer accessors. HTTP-level compression will then reduce the transferred size. The same approach should work with f64, with accessors quantized to 32–64 bits. See gltfpack's -vpf flag, for an example with f32.
accessors quantized to 32–64 bits
Note that there are no 32- or 64-bit normalized data types.
@bghgary, @emackey, @javagl OK to move the draft to Release Candidate?
OK to move the draft to Release Candidate?
Should we add sample models? We usually want to include sample models if the intention is to get implementations so that people can test.
I agree that sample models are important (and I'd probably see whether I can quickly create one of these "minimal test models", like the Triangle storing everything with double or so). But I don't know whether they are required for this to go from 'draft' to 'release candidate'.
If RC means we want implementations (which I believe is the intention), then whoever implements the spec will need something to test. If we don't provide the sample, the implementors will have to create something instead which will make it less likely for the implementation to happen.
Don already covered the topic of generating test models via https://github.com/donmccurdy/glTF-Transform/pull/1417
In the meantime, if it matters, here's the AnimatedTriangle sample asset (embedded), using double/float64 values for the POSITION data and the rotation animation key frames/quaternions:
{
"extensionsUsed" : [ "KHR_accessor_float64" ],
"extensionsRequired" : [ "KHR_accessor_float64" ],
"accessors" : [ {
"bufferView" : 0,
"byteOffset" : 0,
"componentType" : 5123,
"count" : 3,
"type" : "SCALAR",
"max" : [ 2 ],
"min" : [ 0 ]
}, {
"bufferView" : 1,
"byteOffset" : 0,
"componentType" : 5130,
"count" : 3,
"type" : "VEC3",
"max" : [ 1.0, 1.0, 0.0 ],
"min" : [ 0.0, 0.0, 0.0 ]
}, {
"bufferView" : 2,
"byteOffset" : 0,
"componentType" : 5130,
"count" : 5,
"type" : "SCALAR",
"max" : [ 1.0 ],
"min" : [ 0.0 ]
}, {
"bufferView" : 2,
"byteOffset" : 40,
"componentType" : 5130,
"count" : 5,
"type" : "VEC4",
"max" : [ 0.0, 0.0, 1.0, 1.0 ],
"min" : [ 0.0, 0.0, 0.0, -0.7070000171661377 ]
} ],
"animations" : [ {
"channels" : [ {
"sampler" : 0,
"target" : {
"node" : 0,
"path" : "rotation"
}
} ],
"samplers" : [ {
"input" : 2,
"interpolation" : "LINEAR",
"output" : 3
} ]
} ],
"asset" : {
"generator" : "JglTF from https://github.com/javagl/JglTF",
"version" : "2.0"
},
"buffers" : [ {
"uri" : "data:application/gltf-buffer;base64,AAABAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADwPwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPA/AAAAAAAAAAAAAAAAAAAAAAAAAAAAANA/AAAAAAAA4D8AAAAAAADoPwAAAAAAAPA/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA8D8AAAAAAAAAAAAAAAAAAAAAAAAAgL6f5j8AAACAvp/mPwAAAAAAAAAAAAAAAAAAAAAAAAAAAADwPwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIC+n+Y/AAAAgL6f5r8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADwPw==",
"byteLength" : 280
} ],
"bufferViews" : [ {
"buffer" : 0,
"byteOffset" : 0,
"byteLength" : 6,
"target" : 34963
}, {
"buffer" : 0,
"byteOffset" : 8,
"byteLength" : 72,
"target" : 34962
}, {
"buffer" : 0,
"byteOffset" : 80,
"byteLength" : 200
} ],
"meshes" : [ {
"primitives" : [ {
"attributes" : {
"POSITION" : 1
},
"indices" : 0,
"mode" : 4
} ]
} ],
"nodes" : [ {
"mesh" : 0,
"rotation" : [ 0.0, 0.0, 0.0, 1.0 ]
} ],
"scene" : 0,
"scenes" : [ {
"nodes" : [ 0 ]
} ]
}
EDIT: The buffer/bufferView/accessor structure of this file:
Generator code: GeneratedAnimatedTriangleFloat64.zip
Here's MorphStressTest with f64 accessors:
I just picked this arbitrarily, as a scene containing both mesh data and animation. Converting other samples would be easy enough, happy to add whichever are wanted. The gist of the script (after other changes to support f64 in gltf transform) is ...
import { NodeIO } from '@gltf-transform/core';
import { KHRONOS_EXTENSIONS, KHRAccessorFloat64 } from '@gltf-transform/extensions';
import { dequantize } from '@gltf-transform/functions';
const io = new NodeIO().registerExtensions(KHRONOS_EXTENSIONS)
const document = await io.read('sample.glb');
function f64(options = {}) {
return (document) => {
document.createExtension(KHRAccessorFloat64).setRequired(true);
for (const accessor of document.getRoot().listAccessors()) {
if (accessor.getComponentType() === 5126 /* FLOAT */) {
accessor.setArray(new Float64Array(accessor.getArray()));
}
}
}
}
await document.transform(dequantize(), f64());
await io.write('sample_f64.glb', document);
So it's a direct conversion from f32 to f64, the additional precision is not required to render the asset correctly. Further changes will be required in glTF Transform to fully support f64 data, so I wouldn't expect features like mesh simplification or compression to work with f64 yet. See:
- https://github.com/donmccurdy/glTF-Transform/pull/1417
I assume that adding f64 support to EXT_meshopt_compression is technically possible, but not trivial
If filters aren’t used then it should just work. For translation type vectors filters are typically omitted or the exponential is used; the spec only allows f32 outputs for exponential filter so that wouldn't work. It isn’t hard to make it work but I doubt it’s super useful because that implies a precision tradeoff and f64 implies the opposite.
Without filters but with attribute codecs on f64 you will see some meaningful compression iff the values have less entropy than the format allows: for example I would expect reasonable results if f64 encodes f32 values, and minimal compression (10%?) for random values in 0..1 interval.
The coupling between animation usage and mesh/instance usage in this extension is unfortunate.
WebGL, WebGL2, WebGPU don't support double precision buffers or use of doubles in shaders. On desktop you do get some level of support through native APIs although it's conditional on hardware support and fairly sparsely represented in the API (eg in D3D12 there's no way to load 64-bit floats from a buffer directly, so you have to synthesize it from 2 32-bit halves manually).
Overall I would expect that the 99.99% path for a renderer to support this extension for accessors that require GPU access is to decode f64->f32 on load. This is obviously possible, and for some loaders this will not be hard because they don't work with buffer views directly and assume a "accessor -> GPU buffer" conversion model anyhow, but that does make life rather more difficult for renderers that map bufferViews to GPU directly, as you need to analyze accessors that refer to that.
On the flip side, choosing not to support this extension means you can't support f64 property animation, which might be undesirable. This extension could also help for some cases where you need eg instance positions to be f64 and you're using mesh_gpu_instancing extension purely as a compression technique (rendering instances one by one on CPU) or have special careful shader plumbing to actually support this via a GPU path...
I wish that this extension would at least not apply to mesh attributes but maybe there's some reasons to favor that.
that does make life rather more difficult for renderers that map bufferViews to GPU directly, as you need to analyze accessors that refer to that.
The sparse accessors feature from the base glTF 2.0 spec already requires implementations to be able to read/write individual attribute values.
@lexaknyazev We are in interest of having an EXT_accessor_float16 (or a KHR_) extension. We are also aware of this proposal for other data types: https://github.com/KhronosGroup/glTF/pull/2395
As @spnda worked on both extensions, can we somehow align what potential next steps could be?
From our side, our main interest is in having a half-float extension and I think it would make sense to have several additional data types extensions, however to be discussed at once to avoid fragmentation.
I agree with earlier concerns that several implementations may want to pick this up for the sake of 64-bit animation keyframes, but not be willing to take 64-bit vertex attributes at the same time. But I guess it's easy enough for an implementation to just report an error saying they don't support the latter when found. The draft looks good to me, and some sample models might be a good idea before moving to Release Candidate status.
I assume, there are no objections. Will share the pull request internally. Huawei is mainly interested in the fp16 one.
One more detail: Both extensions are still in "Draft" @emackey maybe we review this first?
Both extensions are still in "Draft"
We need sample assets and at least one implementation to update the status.
There are some tests shown in and linked from https://github.com/KhronosGroup/glTF/pull/2397#issuecomment-2118802415 above. But based on the points mentioned above, it could make sense to create models that explicitly use these new types in specific contexts (e.g. "only for animation but not for vertex attributes", or "both, animation and vertex attributes").
I have not yet looked at the updated state here (i.e. my review is still pending). During my first pass, there wasn't too much to say, and I assume that not sooo much has changed since that last pass, but I'll try to allocate some time to review the latest changes.
There are some tests shown in and linked from #2397 (comment) above. But based on the points mentioned above, it could make sense to create models that explicitly use these new types in specific contexts (e.g. "only for animation but not for vertex attributes", or "both, animation and vertex attributes").
I have not yet looked at the updated state here (i.e. my review is still pending). During my first pass, there wasn't too much to say, and I assume that not sooo much has changed since that last pass, but I'll try to allocate some time to review the latest changes.
To play god's advocate, you also create an example for half-float/float16?
@HuaweiNopper It probably would make sense to have the same basic test models for both extensions. I can try to create a float16 version (although this is usually not supported on the CPU, specifically not in Java, I can probably just manually encode some float into a short and shove it into a ShortBuffer...)
although this is usually not supported on the CPU, specifically not in Java
Java has helpers for half-float operations since JDK 20.
Iff there is something specific to review in
float16, drop me a note.
The f16 sections about animation samplers and compatibility are quite different than in f64.
@HuaweiNopper It probably would make sense to have the same basic test models for both extensions. I can try to create a
float16version (although this is usually not supported on the CPU, specifically not in Java, I can probably just manually encode somefloatinto ashortand shove it into aShortBuffer...)
Since C++23, half floats are also available in the C++ standard: https://en.cppreference.com/w/cpp/types/floating-point.html
Java has helpers for half-float operations since JDK 20.
I didn't have that on the radar. (I'm still mainly using Java 8 👴 )
The f16 sections about animation samplers and compatibility are quite different than in f64.
I had another look at this. The main differences seem to be
- the claim that the interpolation itself must be unaffected by the
float16type, - no aligment requirement issues
- the "Compatibility" is a bit simpler for the
float16case
(The latter two somewhat related to the inlined comments)
No specific comments about that.
About test models: I created some flavors of the AnimatedTriangle model. The file name suffix indicates whether it uses the new accessor for the POSITION, for the animation accessors, or for both. (No file suffix are the baselines that do not really use the extensions)
AnimatedTriangle-accessor_floatN-2025-10-23.zip
From the inlined comments:
It's important to have sample assets covering this issue.
As mentioned there, my tooling currently aims at getting the alignment right. For example, the one where float64 is used for the animations (but not the positions):
The animation data starts at byte 48 and not 44. But maybe I can create dedicated tests for this with the data starting at byte 44.
@HuaweiNopper It probably would make sense to have the same basic test models for both extensions. I can try to create a
float16version (although this is usually not supported on the CPU, specifically not in Java, I can probably just manually encode somefloatinto ashortand shove it into aShortBuffer...)Since C++23, half floats are also available in the C++ standard: https://en.cppreference.com/w/cpp/types/floating-point.html
Since the <stdfloat> header is optional it is barely supported by any compiler, even less support IEEE754 types that are not supported by the target hardware. GCC supports 32-bit and 64-bit since GCC 13, and MSVC supports 32-bit and 64-bit since 17.7. As far as I know nobody supports 16-bit anywhere, but I might be wrong.
I think your best bet is just to rely on x86/armv8 intrinsics for specific instruction set extensions that add support for binary16 data types, if you even want hardware accelerated processing over software floats.
@HuaweiNopper It probably would make sense to have the same basic test models for both extensions. I can try to create a
float16version (although this is usually not supported on the CPU, specifically not in Java, I can probably just manually encode somefloatinto ashortand shove it into aShortBuffer...)Since C++23, half floats are also available in the C++ standard: https://en.cppreference.com/w/cpp/types/floating-point.html
Since the
<stdfloat>header is optional it is barely supported by any compiler, even less support IEEE754 types that are not supported by the target hardware. GCC supports 32-bit and 64-bit since GCC 13, and MSVC supports 32-bit and 64-bit since 17.7. As far as I know nobody supports 16-bit anywhere, but I might be wrong.I think your best bet is just to rely on x86/armv8 intrinsics for specific instruction set extensions that add support for binary16 data types, if you even want hardware accelerated processing over software floats.
It is more about creating the examples. Furthermore, in most cases the data can be passed 1:1 to the GPU.
If needed to be processed on the CPU, e.g. GCC supports float16 since GCC 13: https://gcc.gnu.org/projects/cxx-status.html#cxx23
It is more about creating the examples. Furthermore, in most cases the data can be passed 1:1 to the GPU.
I understand that this is just for examples, I was just trying to say that it might not be feasible as an example since the support isn't quite there yet so some might have issues running it or testing it. Yes, a GPU does support this but a lot animation processing often still happens on the CPU in many renderers.
If needed to be processed on the CPU, e.g. GCC supports float16 since GCC 13: https://gcc.gnu.org/projects/cxx-status.html#cxx23
I was wrong about it outright not supporting it, yes. However, I checked and it can effectively only use it as a storage data type, since the x87 instructions don't support it. All processing converts them to binary32 first, then calls the instruction on those, and then converts the resulting binary64 float back to binary16: https://godbolt.org/z/EbvsTKh1f. So it is actually kinda usable to my surprise, but brings no real gain for most platforms.
It is more about creating the examples. Furthermore, in most cases the data can be passed 1:1 to the GPU.
I understand that this is just for examples, I was just trying to say that it might not be feasible as an example since the support isn't quite there yet so some might have issues running it or testing it. Yes, a GPU does support this but a lot animation processing often still happens on the CPU in many renderers.
If needed to be processed on the CPU, e.g. GCC supports float16 since GCC 13: https://gcc.gnu.org/projects/cxx-status.html#cxx23
I was wrong about it outright not supporting it, yes. However, I checked and it can effectively only use it as a storage data type, since the x87 instructions don't support it. All processing converts them to binary32 first, then calls the instruction on those, and then converts the resulting binary64 float back to binary16: https://godbolt.org/z/EbvsTKh1f. So it is actually kinda usable to my surprise, but brings no real gain for most platforms.
We mainly request the KHR_accessor_float16 extension to save the overall size of the glTF file and pass directly to the GPU e.g. using GL_HALF_FLOAT in an OpenGL context. Our purpose is not for e.g. interactivity and mainly for data size reduction.
brings no real gain for most platforms
It reduces storage (and potentially memory usage) so it's not that bad even for animations.
We mainly request the
KHR_accessor_float16extension to save the overall size of the glTF file
Do you have any specific use case (that could be shared) showing f16 benefits over 16-bit quantization? FWIW, unorm16 provides generally better precision than float16 at the same storage cost. The only drawback is that the actual mesh size would have to be handled via node transforms.