Improve support for glTF → Model I/O conversion

[JaapWijnen]

Hey there, I'm trying to load a .glb file through ModelIO I'm using the following code:

let assetURL = Bundle.main.url(forResource: path, withExtension: nil)!
let gltfAsset = try! GLTFAsset(url: assetURL)
let bufferAllocator = MTKMeshBufferAllocator(device: device)
let mdlAsset = MDLAsset(gltfAsset: gltfAsset, bufferAllocator: bufferAllocator)

The asset I'm trying to load is part of this pack: https://kenney.nl/assets/mini-dungeon (free to download) I'm specifically loading the banner asset in this example. Models > GLB Format > banner.glb

The loaded gltf asset seems to contain some data, the mdl asset seems to be empty. In the screenshot the upper part is some of the content of the gltf asset according to the Xcode debugger, the bottom part is the mdl asset.

What could I be doing wrong here?

[warrenm]

Model I/O conversion is incomplete and not supported. However, as of 150183f, you should notice that you're at least able to load (and export) static models like the Kenney banner. Feel free to let me know if you encounter other issues with the MDL path or if you'd like to collaborate on improving it. It hasn't been exercised much until now. We'll use this issue to track progress on implementation.

[JaapWijnen]

Ah ok! I hadn't noticed it was incomplete thanks! I'll try that commit out somewhere this week! I've mainly been using it since I had some old MDL code lying around and wanted to load gltf assets through that path as well. What would you say is the easiest way to get Metal buffers out of the GLTF Asset? I'd like to play with animations too (some Kenney characters have animations for example). Would you recommend going through the MDL path for that? If so then I would like to try to contribute although my objC is a little rusty

[warrenm]

For getting glTF data into Metal buffers, you could just iterate the GLTFBuffers in the asset and copy them directly to Metal buffers that you allocate. It somewhat depends on what you want to do with the data from there. It's definitely possible to render a glTF asset with Metal without going through any of the conversion layers (SCN, MDL, RK), but it is a lot of work to do so, since it entails writing your own animation and rendering engine.

Regrettably, Model I/O is a buggy, almost totally neglected framework, so even rudimentary things like export of object hierarchies to USD is broken and probably has been since platform support for USD was introduced. In any case, the Model I/O converter in this repo currently doesn't have any animation support whatsoever, but again the question becomes what you want to do with assets once they're in Model I/O land. I have a hard time recommending Model I/O for any purpose whatsoever, because it's so poor at its stated purpose (3D asset storage and interchange). There's almost certainly a better way to achieve any use case you might have.

[JaapWijnen]

Thanks that is great advice! 😅 I'm working on a little render engine from scratch in Swift. Currently rendering a map with the Kenney Mini Dungeon tiles. Was previously doing this by loading the .obj assets he provides. I was loading them through Model I/O and then getting MTKMeshes from those instances to get vertex buffers etc. Since some of Kenney's assets also have animations I was hoping to make use of them. He has some mini characters with animations. But those aren't in the .obj format of course. So was looking at the other formats he provides but Model I/O isn't able to load any of those unfortunately. (FBX and GLB) I'm writing my own renderer atm so might as well get into the animation bit as well. Although I'm less experienced there and don't know how involved that is yet haha. The project is in Swift and Metal, no dependencies atm expect for me trying to use GLTFKit2 to load these through the Model I/O path I had already created for originally just using the .obj assets :)

Your feedback is really appreciated! Glad I'm not investing too much time in Model I/O given the context you gave above.