vpenades
File resolving issues.
For a project im working on, i need to handle texture resolvement myself. While working with this, i encountered two problems. One i could hack around, and another one which i am stuck with atm:
- i can use ReadContext.Create to define a custom loader for the files. If a .gltf references a .bin however, the function only gets passed the filename of that .bin and does not combine the gltf path with the .bin file. So when i load "foo/bar.gltf" and it references "baz.bin" i would expect to get a path "foo/baz.bin". So in the loading function i do not know where to look for that particular file. I could work around this by doing this, but thats pretty annoying, so i hope SharpGLTF adresses this by itself:
var gltfLoader = new GltfLoader(path);
var settings = ReadContext.Create(name => gltfLoader.LoadFile(name));
var modelRoot = ModelRoot.Load(path, settings);
public sealed class GltfLoader(string gltfPath)
{
public ArraySegment<byte> LoadFile(string path)
{
if (path != gltfPath)
path = Path.Combine(Path.GetDirectoryName(gltfPath), path);
using var stream = fileSystem.OpenRead(path);
return stream.ReadBytes((uint)stream.Length);
}
}
- Texture loading should be skippable, still providing the paths to those files. My project requires the framework to directly load images and stream them info gpu buffers, so i do not want sharpGLTF to also load those huge images into memory as i cannot use them from there. There is currently no way to disable auto texture loading, so i tried to simply return an empty array if the requested file is neither the gltf itself, nor the .bin file. While this correctly works, the path to those textures is nowhere stored. So i am unable to pass those paths into my framework which will load the image. It would be nice if the path (maybe even the resolved path, see issue 1) of a particular texture would be stored. In the PrimaryImage propety it is already possible to access the texture name, but the path is nowhere to be found.
for 1. I'll take a look
for 2. , see ReadSettings.ImageDecoder property, which is a callback to intercept texture loading
I do not want SharpGLTF to load the byte[] of the images into memory at all. The ImageDecoder gets an image parameter, which can be populated with the byte[] from the image, but it does not contain the resolve path to load the image yourself.
FileReaderCallback is expected to return the byte[] of the files the gltf references. In this case the .png file. For glb this makes sense, as the image is inside the .glb and i need to extract those data from the .glb. But for gltf files having external .png files, i want to omit the whole loading procedure at all, and simply access the path where it "should" be, so i can throw that path into my asset manager which uses a virtual file system underneath as well as a caching system. So if a texture is referenced from multiple .gltf models, only the first will cause a load, and the others will always return the same gpu texture.
ImageDecoder Can then parse the custom byte array, which is again not what i need. The only ugly hack i can think about is to write the path string as byte array in the FileReaderCallback, and then convert the byte[] back to a string from the data of the Image instance. There already is an Uri property which however is set to null as well as it being private. If it would be filles with the path to the file as well as have a public getter, i could resolve the textures from my caching system when parsing the scheme to populate my framework objects.
The rationale of ImageDecoder is this:
- glTF has three different ways of storing an image:
- Embedded (rarely used).
- Within the GLB binary blob.
- As an external file reference.
ImageDecoder tries to address texture preloading while supporting these three wildly different ways of storing a texture.
Also notice that the ImageDecoder callback returns a boolean that, if set to true it will free the buffers used by the image, so in a sense it loads the data for you while keeping compatibility with all texture storage mechanisms, allowing you to upload to gpu, and then releasing the buffers used by that image.
Also notice that when ImageDecoder callback is called, the Image object contains a MemoryImage object that not only contains the actual image bytes, but also a File Hint Path (I don't remember the propery name now), which has the path for the image. Of course this property is null when the image is embedded or contained within the GLB.
I am aware you want the path before the bytes are even loaded, but that would work only for externally stored images, which might be fine to your use case, but it can be potentially troublesome for others.
Possible solutions:
-
maybe yet another hook could be added, something like SatelliteImagePreload, which is passed a path and returns a byte array (or null) , but it would only work for external referenced files, which, from my point of view, introduces loop holes in the API.
-
Use the ReaderContext to capture image loading, use your cache system to load the images, _and then replace the incoming buffer with a tiny image to be passed to the library to keep things working. The library already defines a tiny PNG file in memory that is only a few bytes. That will ensure that the model passes validation after loading, while greatly reducing the memory footprint.