Different boneWeights accuracy in edit mode and run mode

[ijinfeng]

Describe the bug I found a problem, the same glb, previewed in edit mode, and dragged into the scene, the model behaves normally, but the dynamically loaded glb behaves abnormally. Through tracking, I found that the boneWeights of the two are slightly different, so far it seems that the difference is in precision.

Files

Below is a comparison of boneWeights between the two glb loads.

And you can try this by use this glb file. uplimbs001.glb.zip

Code

async void Update()
    {
        if(Input.GetKeyDown(KeyCode.A))
        {
            string path = HaiSdkManager.GetPersistentPath();
            path = Path.Combine(path, "uplimbs001.glb");
            Dowork(Path.GetFullPath(path));
        }
    }

async void Dowork(string filePath)
    {
        var downloadProvider = new GLTFast.Loading.DefaultDownloadProvider();

        var m_Gltf = new GltfImport(
            downloadProvider,
            new UninterruptedDeferAgent(),
            null,
            null
            );


        var importSettings = new ImportSettings
        {
            // Avoid naming conflicts by default
            NodeNameMethod = NameImportMethod.OriginalUnique,
            GenerateMipMaps = true,
            AnimationMethod = AnimationMethod.Mecanim,
        };

        var instantiationSettings = new InstantiationSettings();

        bool success = await m_Gltf.Load(filePath, importSettings);

        if (success)
        {

            if (instantiationSettings.SceneObjectCreation == SceneObjectCreation.Never)
            {

                // There *has* to be a common parent GameObject that gets
                // added to the ScriptedImporter, so we overrule this
                // setting.

                instantiationSettings.SceneObjectCreation = SceneObjectCreation.WhenMultipleRootNodes;
                Debug.LogWarning("SceneObjectCreation setting \"Never\" is not available for Editor (design-time) imports. Falling back to WhenMultipleRootNodes.", this);
            }

            for (var sceneIndex = 0; sceneIndex < m_Gltf.SceneCount; sceneIndex++)
            {
                var scene = m_Gltf.GetSourceScene(sceneIndex);
                var sceneName = m_Gltf.GetSceneName(sceneIndex);
                var go = new GameObject(sceneName);
                var instantiator = new GameObjectInstantiator(m_Gltf, go.transform, null, instantiationSettings);
                var index = sceneIndex;
                success = await m_Gltf.InstantiateSceneAsync(instantiator, index);
            }
        }
    }

Looking forward to your reply!

[atteneder]

Thanks for the report.

Can you please check in your Projects settings under Quality how many skin weights bones you have set? It's possible you've set it to just two. In that case glTFast ignores bone weights 3 and 4 and re-normalizes the first two so that their sum is 1.0 again.

hth

[ijinfeng]

Thanks for the report.

Can you please check in your Projects settings under Quality how many skin weights bones you have set? It's possible you've set it to just two. In that case glTFast ignores bone weights 3 and 4 and re-normalizes the first two so that their sum is 1.0 again.

hth

In my project settings, Skin Weights is set to Unlimited. You are right, it works fine when I set it to 2 Bones, but the problem above occurs when it is set to Unlimited or 4 Bones.

I fixed this bug when there are more than 4 bones, and it works well now.

// In file `VertexBufferBones.cs`
public override void ApplyOnMesh(UnityEngine.Mesh msh, int stream, MeshUpdateFlags flags = PrimitiveCreateContextBase.defaultMeshUpdateFlags)
        {
            Profiler.BeginSample("ApplyBones");
            // FIX by jinfeng
            unsafe
            {
                for (int i = 0; i < m_Data.Length; i++)
                {
                    VBones bones = m_Data[i];
                    float total = bones.weights[0] + bones.weights[1] + bones.weights[2] + bones.weights[3];
                    if (total > 0)
                    {
                        float mult = 1f / total;
                        bones.weights[0] *= mult;
                        bones.weights[1] *= mult;
                        bones.weights[2] *= mult;
                        bones.weights[3] *= mult;
                        m_Data[i] = bones;
                    }
                }
            }
            msh.SetVertexBufferData(m_Data, 0, 0, m_Data.Length, stream, flags);
            Profiler.EndSample();
        }

[dalkia]

Hello! I had the same issue, and you solution work great. Thank you @ijinfeng !!

[atteneder]

@ijinfeng Thanks for looking into it and offering a solution.

The error comes from the input data, the glb file. It contains bone weights that are not normalized and thus not valid (run it through the glTF Validator to confirm). I suggest you figure out the source of that file and file an issue with whoever maintains that glTF generator (the meta data says it's hai me export plugin).

Given that recalculating bone weights requires additional computing resources, I won't fix it. If you input invalid data, don't expect correct output.

If I wanted to fix it, I'd use SortAndNormalizeBoneWeightsJob (in Jobs.cs) instead, since it is Burst compiled and runs in a thread (thus much faster). If you want your fix in place, I suggest to always run said job in VertexBufferBones.ScheduleVertexBonesJob.

One could argue, that this should be an option for the runtime (opt-in fixing weights). I'm hesitant.

hth