godotengine
Add audio support to the GLTF module
Implements this proposal https://github.com/godotengine/godot-proposals/issues/8814
This PR adds support for audio import and export in the GLTF module using the not-yet-finalized KHR_audio_emitter GLTF extension (https://github.com/KhronosGroup/glTF/pull/2137). This allows you to save audio inside of GLTF scenes, and load them back later. In the future this will also allow you to use GLTF as an interchange format between game engines.
Some example use cases: a fountain that makes water noises, a gun that makes custom sounds when fired, the radio from Portal that plays music on a loop, or a tree that includes bird chirping noises or rustling leaves or something. For more details about the intended use cases, see the proposal.
Freely licensed example file: https://github.com/omigroup/gltf-extensions/tree/main/extensions/2.0/KHR_audio_emitter/examples/boom_box This file contains a boom box, a short looping music clip, OMI physics, and licensing information via KHR_xmp_json_ld. The model is CC0, created by The Khronos Group, and the music is CC-BY 3.0, created by Kevin MacLeod.
The code in this PR is ready for review, but note that the extension is not yet finalized. Usually my approach is for us to be pioneers with Godot and implement extensions that may not be finalized, like with OMI physics. However, for this extension, since it is using the KHR_ namespace, we must tread carefully and avoid shipping features in Khronos's namespace into production without Khronos's approval. Don't put words in their mouth, so to speak.
Production edit: closes godotengine/internal-team-priorities#44
Marking as draft to prevent accidental merging before ratification. However, the code is complete and ready for review.
We're currently looking at adding KHR_audio_emitter support to UnityGLTF (for export and import), and I just tested some of our files on this Godot branch :)
Is it intentional that currently, embedded audio buffers don't seem to be supported? On playback of such a file, I get an error:
The file in question: 20250404-UnityGLTF-Audio.glb.zip
Godot seems to correctly see that there's an audio buffer embedded, its name and settings, but can't play it back. But our export implementation is still WIP – so it could also be that the file isn't valid in some way, I'm not sure if there's another player/implementation that supports the extension in its current form.
@hybridherbst Thank you for the test file. However, the problem is in the MP3 import code, not in the glTF importer.
As a test, I tried to save the data to a MP3 file and got this: test.mp3.zip. This correctly plays inside of VLC Media Player, but Godot is unable to import it:
The Godot MP3 import code is pretty trivial, it just calls into the MiniMP3 third-party library. The function returns an error code of 0 (so, no error), but the Hertz of the file is zero. I checked every function in MiniMP3 that contains hz = and none of the functions are even being called. I would open an issue on MiniMP3 but the library seems to be abandoned, maybe @godotengine/audio could chime in.
Anyway, it's not a bug in the glTF code. If Unity exports a glTF with an MP3 that Godot could normally read as a standalone file, then that same audio data should work inside of a glTF.
Thanks for looking at it! – turns out that the actual issue was a wrong mime type, the file specified mp3 but actually was a WAV. This made VLC and others that don't care so much play it back, but Godot (rightfully) rejected it. So the file you extracted would have needed to be renamed to .wav to work...
Here's an updated file with an mp3 and correct mime type: 20250408-UnityGLTF-Audio.glb.zip
I have critical concerns that WAV is in the spec and would cause portability issues with the Khronos specification.
@fire is it possible you've misread the spec? it's both mp3 and wav.
"Unless specified by another extension, the only supported mimeTypes are audio/mpeg and audio/wav." <-- this is in the spec of this particular glTF extension not the glTF spec itself etc.
@coderofsalvation what you refer to as "the spec" is an unofficial proposed pull request by OMI (which many of us participate in) against a draft KHR spec. Unfortunately Khronos has been extremely slow formally ratifying or publishing this particular extension, but that means while very solid and implementable, the draft extension on a non-khronos repository cannot be treated as canon.
I believe @fire is referencing some contention related to certain formats from some of the Khronos meetings, not all of which is visible publicly. I attended one such public meeting over two years ago where the subject of audio formats came up, but I am not sure about the current state so this is speculation on my part.
anyway I just wanted to illustrate some of the context here. The KHR audio spec has been in work for over 4 years so there is quite a long history...
@lyuma thanks for providing context @aaronfranke thanks for the new commit