[rmodels] Potential problem with transforming normals during animations

[Bigfoot71]

• [x] I tested it on latest raylib version from master branch
• [x] I checked there is no similar issue already reported
• [x] I checked the documentation on the wiki
• [ ] My code has no errors or misuse of raylib

Issue description

There seems to be a potential error in the normal transformation process within the UpdateModelAnimation() function. I noticed that the shading looked strange when using my lighting shader during an animation, which led me to suspect the normals were being handled incorrectly. After some investigation, I checked the code responsible for transforming the normals in rmodels.c:2358:

// Normals processing
// NOTE: We use meshes.baseNormals (default normal) to calculate meshes.normals (animated normals)
if (mesh.normals != nullptr) {
    animNormal = (Vector3){ mesh.normals[vCounter], mesh.normals[vCounter + 1], mesh.normals[vCounter + 2] };
    animNormal = Vector3Transform(animNormal, surface.mesh.boneMatrices[boneId]);
    mesh.animNormals[vCounter] += animNormal.x * boneWeight;
    mesh.animNormals[vCounter + 1] += animNormal.y * boneWeight;
    mesh.animNormals[vCounter + 2] += animNormal.z * boneWeight;
}

I observed that the normals are being transformed like this:

animNormal = Vector3Transform(animNormal, surface.mesh.boneMatrices[boneId]);

But shouldn’t we be transforming the normals using only the rotational part of the matrix, by obtaining the inverse transpose?

animNormal = Vector3Transform(animNormal, MatrixTranspose(MatrixInvert(surface.mesh.boneMatrices[boneId])));

Or maybe I’m missing something else to properly solve this issue, which could explain why the function was written this way.

---

Here’s an example with the current function (look at the robot's hands):

And here’s an example with the inverse-transpose transformation applied:

[Bigfoot71]

Sorry for the title, I'm tired :fearful:

[raysan5]

But shouldn’t we be transforming the normals using only the rotational part of the matrix, by obtaining the inverse transpose?

@Bigfoot71 Interesting... sincerely, I don't know... but it seems to work ok with the transform... could you test with some other models to confirm the issue?

[Kirandeep-Singh-Khehra]

I have also faced similar issue with Mixamo y bot it was working before but then i updated to raylib5 or 5.5(can't recall) and messed up normals. I thought it was issue with model.

it looks like normals are overrotated or bone rotation also brightens the part. i am using UpdateModelAnimation() to update animation and lighting.vs and fs from raylib examples.

[Bigfoot71]

Unfortunately, I tried just under ten models from Sketchfab, all in GLB format. Most of the models either "disappear" during animation or are animated in a completely distorted way.

Regarding the deformation issue, I noticed there's already a related issue here: https://github.com/raysan5/raylib/issues/4569.

That said, I did manage to get one model working without visible differences between the two versions. However, the range of movement relative to the idle animation isn't very significant, which I think might have an impact.

For the robot, the limbs are noticeably misaligned between the idle and the animation used in my screenshots.

As for the solution I proposed, while it is mathematically correct for transforming normals, it does raise an efficiency concern. Performing a matrix inversion per vertex and per bone during each pose update (as I suggested) is quite expensive.

This is why I was wondering if this behavior might be intentional and handled differently, but I couldn't find any information about it.

I'll try again when I have time with formats other than GLB. I'll use Blender if necessary to attempt to make the models work in raylib.

---

Example of the only model I managed to get working:

Without invert transpose:

With invert transpose:

Edit: GitHub doesn't seem to display videos for me, so I'm adding the links here just in case

• https://github.com/user-attachments/assets/cb027546-39a2-463d-b4a9-7bda43a567ea
• https://github.com/user-attachments/assets/4df6ead7-bd45-419b-af7d-7c9bff53c28a

[Bigfoot71]

I just tested with a new model, and there is indeed an issue with the transformation of normals in the original function. Moreover, there seem to be additional animation problems...

Here it is with the current function:

https://github.com/user-attachments/assets/35e6b980-ad9e-42a2-808a-e4ab5ebeec73

Here it is with my quick fix:

https://github.com/user-attachments/assets/70fba5d5-b42b-4013-88e2-2c56da591533

Here is the original model: https://sketchfab.com/3d-models/animated-old-chest-4477ee9446b34d8e857906816ca5245a

[raysan5]

@Bigfoot71 Ok, I see. Please, could you send a PR with the fix?