mat3.normalFromMat4 seems to return unexpected values

[renekaesler]

The documentation of mat3.normalFromMat4() says, the method returns the transpose inverse of the given matrix. So I would expect the same result when i call transpose & invert on my own. In fact it is not:

const mv = [
  0.88,   -0.27,     0.39,     0.39,
  0.39,    0.88,    -0.27,    -0.27, 
 -0.27,    0.39,     0.88,     0.88, 
   1.5,       0,       -7,        1
];

const n1 = mat3.normalFromMat4(mat3.create(), mv);

const n2 = mat3.fromMat4(mat3.create(), mv);
mat3.invert(n2, n2);
mat3.transpose(n2, n2);

//  n1                    n2
//  0.8804924488067627    0.8804924488067627
// -0.27054348587989807  -0.27054348587989807
//  0.21384871006011963   0.3900510370731354
//  0.3900510370731354    0.3900510370731354
//  0.8804924488067627    0.8804924488067627
//  0.03931663557887077  -0.27054348587989807
// -0.27054348587989807  -0.27054348587989807
//  0.3900510370731354    0.3900510370731354
//  0.0593346506357193    0.8804924488067627

In my program, this results in wrong lightning calculations. Is there something I do miss?

[stefnotch]

When setting mv to

const mv = [
  0.88,   -0.27,     0.39,     0,
  0.39,    0.88,    -0.27,    0, 
 -0.27,    0.39,     0.88,     0, 
   0,       0,       0,        1
];

the results are equivalent.

That behavior is because those values at the "edges" affect the determinant of the matrix, which is used to calculate the inverse. https://github.com/toji/gl-matrix/blob/518d94fbcc0439c67b15958528a996574feaac0f/src/mat3.js#L531

Now, whether that is correct or not, I cannot say. I'd have to research it.

[renekaesler]

Thanks for the fast response,

Of course, if I eliminate the edges, I get equivalent results. In most cases, the matrix is dynamically generated and contains translations, rotations & scaling. Eliminating the edges is therefore not an option.

It feels to me, the method name implies the generation of the normal matrix directly from the model view matrix.

Kind Regards, Rene