UnityGLTF
UnityGLTF copied to clipboard
KhronosGroup
Performance and Textures
I noticed the following while dynamically loading some GLTF models:
- Models loaded: 14
- Total primitives in the Scene: 78. Each consists of simple shapes, in some cases an assortment of flat planes, in other cases various boxes. Each Primitive (in the Unity scene sense) uses a single 2-4MB texture
- Memory profiler:
- Meshes: 151 / 27.6MB
- Textures: 573 / 1.30 GB
This seems horrifically wrong to me, and it indeed appears to be borne out by tragic performance and high loading times even for these very simple models. In particular, note that there are somehow 573 textures for 78 different Primitives (each of which uses a single texture) Is this an issue with UnityGLTF itself, or is there some flag I can provide or something in my models that is causing that? Is there any other information I can provide to help debug this?
By default Unity creates cpu copies of textures. This can be prevented by overriding ConstructTexture where we mark the texture as gpu only (though that means no samplers can be created). There is currently no config for that but we can make one.
My guess currently is that for some of the models there are samplers, which creates an additional texture, and that it is double counting the CPU and GPU texture.
Wrt to number of primitives, that is not surprising. We treat each MeshPrimitive as it's own Unity mesh due to constraints of Unity's system.
@blgrossMS thanks for that feedback. It makes sense that there might be double-counting between CPU and GPU textures, and it makes sense that each MeshPrimitive is its own Unity mesh. We're converting DAE files to GLB via KhronosGroup/COLLADA2GLTF, and since DAE requires images -> surface -> sampler -> texture, we certainly do have samplers (and we often reuse samplers, for the diffuse and ambient textures in an effect, for example). Does that explain how there can be 3.7x as many textures as meshes, though, especially when some of those meshes use non-textured colored effects?
If helpful, I can do a little more debugging to determine exactly the ratios between imported meshes and the numbers of images, samplers, and uses of samplers.
i havn't looked more closly at the converter but i suspect it has something to do with instancing and material assignment . is it possible to attach both the collada and the glb/gltf file ?
This is the code in that clones the meshes in the converter. Can you use the visual studio gltf plugin to open the gltf file and see if all the instances in the gltf are pointing to the same mesh, materials or they are getting a unique ID?
https://github.com/KhronosGroup/COLLADA2GLTF/blob/master/src/COLLADA2GLTFWriter.cpp for (GLTF::Primitive* primitive : primitiveMaterialMapping[materialBinding.getMaterialId()]) { if (primitive->material != NULL && primitive->material != material) { // This mesh primitive has a different material from a previous instance, clone the mesh and primitives GLTF::Mesh* cloneMesh = new GLTF::Mesh(); mesh = (GLTF::Mesh*)mesh->clone(cloneMesh); primitive = mesh->primitives[j]; } primitive->material = material; }
- I'll try to get you sample models later today or tomorrow.
- I'll check those IDs for you later today or tomorrow
In the meantime, here's some potentially helpful information derived from three different DAE models converted to GLB with embedded textures and then loaded, along with control cases. Please note that this is all computed in the editor, which along with the skybox I assumes accounts for the 380 textures and 16 meshes even with an empty scene.
- No objects: 380 textures (60.4MB), 16 meshes (1.3MB)
- Hidden GLTF Component: 380 textures (60.4MB), 16 meshes (1.3MB), 43 materials (67KB)
- With Model 1 (
terrain): 389 textures (129MB), 22 meshes (292 KB), 49 materials (79KB)- DAE: 3 JPEG images, of 407KB, 63.5KB, and 477.7KB (948.2KB total). (Apparently each image is counted three times?)
- DAE: 3 materials, each mapping to 1 effect
- DAE: 3 effects, each of which has a surface mapping to one image, a sampler, and uses that sampler twice, in the diffuse and ambient fields.
- With Model 2 (
trees): 376 textures (67.7MB), 18 meshes (217 KB), 49 materials (83KB)- DAE: 2 materials, each mapping to 1 effect
- DAE: 2 effects, each of which uses only colors
- With Model 3 (
struct): 402 textures (113.6MB), 281 meshes (8.5MB), 576 materials (1.6MB)- DAE: 5 JPEG images, of 35.7KB, 18.3KB, 13.6KB, 31.8KB, and 477.7KB (577.1KB total). (Appears to be 22 textures counted for 5 images)
- DAE: 10 materials, each mapping to 1 effect
- DAE: 10 effects, 5 of which use only colors, 5 of which use images. Each of the 5 using images uses a single image with a surface, a sampler, then uses that sampler in the diffuse and ambient fields.
- DAE also contains a lot of instanced geometry (4984 instance nodes) from 265 actual meshes. Related to the whopping 576 materials and 281 meshes listed? Sort of appears that each geometry, even with the same material, is creating a new material?
As promised, if it would help track this down:
- Input DAE: struct.dae.zip
- GLB generated with COLLADA2GLTF: struct.glb.zip (please excuse erroneous extra geometry, under discussion in KhronosGroup/COLLADA2GLTF/issues/236)
@blgrossMS thanks! Let me know if there's any additional evidence I can contribute.
@KermMartian I meant to look at this the last two weekends, but ended up fixing build and test issues that took priority. This is next on my list though! Sorry for the delay.
Sorry for the delayed response @KermMartian . I have looked into this a bit. Here are my findings:
Why is a 5.75 MB glb file taking up 71.8 MB of scene memory? 45.9 mb = texture data 8.6 mb = mesh data 17.3 mb = Unity objects (Transform, GameObject, MeshRenderer, Material, MeshFilter)
Texture: First of all, we are turning all textures into rgba32 which is making them fully sized uncompressed from a JPEG. We are also enabling mipmaps by default. Disabling mipmaps saves about 10 MB (down to 34.4 MB), so I'll make that a flag.
Next I disabled all texture copying for samplers and stopped copying textures to the CPU by default. This saved another 22.9 MB (down to 11.5 MB).
I finally tried to play around with moving it to different GPU compression formats, which should utilize even less memory, but had no success so far.
Mesh: Mesh size is double what it should be because we are currently doing read/write (which means its not a GPU only copy). In the models you provided we are using 8.6 MB of space instead of 4.3 MB, which is the size of the binary portion. We can add a flag here that will enable none CPU copies of mesh data. We already have a toggle for this (KeepCPUCopyOfMesh on GLTFSceneImporter). Disabling this flag did not actually change the numbers though, so it will require further investigation.
Unity Objects: Not much we can do here. There are 15377 objects that are generated from your hierarchy. We currently are matching the node structure of the file, so we need to generate that number of objects.
I also recommend using deep profiling and look at scene memory. The overall viewer gives an incorrect impression. For standalone with nothing in the scene I was getting: 2097 textures at 22.5 MB 23 mesh at 341 KB
This is great; thank you so much! I'll take your advice about looking at scene memory. In addition, how can I access the flag to disable texture copying for samplers, and is there anything I can do to support your adding of the flag to disable mipmaps and repair the flag to disable CPU copies of mesh data?
@blgrossMS Hope all is well, and I apologize for the bump. I wanted to check back to see where I can find the flag you mentioned, and if there's anything I can do to help with those other two missing/broken flags. Thanks!
To reduce GameObject count, in GLTFSceneImporter.ConstructMesh, I do this inside the loop:
// Only create the Primitive GameObject if multiple Primitives are used
//var primitiveObj = new GameObject("Primitive");
GameObject primitiveObj;
if (mesh.Primitives.Count == 1)
{
primitiveObj = parent.gameObject;
}
else
{
primitiveObj = new GameObject("Primitive");
}
For texture memory, in GLTFSceneImporter, change the RGBA32 to DXT1 to have the textures compressed on the GPU.
Even better for Textures is using DDS texture files. This takes a bit more modification of the code. I did it for our project and removes the frame drop when loading textures.
private static bool IsDDS(byte[] bytes)
{
return bytes[4] == 124;
}
private static Texture2D LoadTextureDXT(byte[] ddsBytes, TextureFormat textureFormat)
{
if (textureFormat != TextureFormat.DXT1 && textureFormat != TextureFormat.DXT5)
throw new Exception("Invalid TextureFormat. Only DXT1 and DXT5 formats are supported by this method.");
if (!IsDDS(ddsBytes))
throw new Exception("Invalid DDS DXTn texture. Unable to read"); //this header byte should be 124 for DDS image files
int height = ddsBytes[13] * 256 + ddsBytes[12];
int width = ddsBytes[17] * 256 + ddsBytes[16];
int DDS_HEADER_SIZE = 128;
byte[] dxtBytes = new byte[ddsBytes.Length - DDS_HEADER_SIZE];
Buffer.BlockCopy(ddsBytes, DDS_HEADER_SIZE, dxtBytes, 0, ddsBytes.Length - DDS_HEADER_SIZE);
Texture2D texture = new Texture2D(width, height, textureFormat, true);
texture.LoadRawTextureData(dxtBytes);
texture.Apply();
return (texture);
}
Where ever the terrible LoadImage is used, do this
byte[] buffer = memoryStream.ToArray();
if (IsDDS(buffer))
{
texture = LoadTextureDXT(buffer, TextureFormat.DXT1);
}
if (texture == null)
{
texture = new Texture2D(0, 0, TextureFormat.DXT1, true, isLinear);
texture.LoadImage(buffer, markGpuOnly);
}
I'm working on something similar. We are importing GLTF models with multiple pbrMetallicRoughness-based material choices (one material at a time) for a single mesh. How should that look in the gltf json file? Currently importing this model with 3 materials only puts texture/image/material id=2 into the unity mesh's material array.
How to fix? Should there be multiple "Primitives" arrays for the single mesh?
"meshes": [ { "name": "mesh_1", "primitives": [ { "material": 2, "mode": 4, "attributes": { "JOINTS_0": 46, "NORMAL": 44, "POSITION": 43, "TEXCOORD_0": 45, "WEIGHTS_0": 47 }, "indices": 42 } ] } ],
I feel this should be somewhere in the document, as Read/Write and Texture sizes tips are really helpful in building an application which needs multiple GLTF to be loaded in 1 place. Found this thread after a long time. Thanks all!
