bitshifter
`any_orthonormal_pair` and `cross` are not handedness-agnostic or documented
Describe the bug
Glam readme says it is handedness-agnostic, but Vec3::any_orthonormal_pair and Vec3::cross do not specify which handedness is produced and assume a handedness. (They are both right-handed)
To Reproduce
Read the docs on the methods
Expected behavior
Handedness is specified in either the docs or the function name as many functions in Mat3 do (_rh vs _lh)
System information (please complete the following information):
- Any
Additional context
I imagine cross_rh cross_lh would be a little bit undesirable. But any_orthonormal_pair_rh vs any_orthonormal_pair_lh could be manageable. How do we communicate this?
While I guess there would be two possible derivations of cross product that would produce orthogonal vectors with different signs, based on the derivation here https://math.libretexts.org/Bookshelves/Calculus/Calculus_(OpenStax)/12%3A_Vectors_in_Space/12.04%3A_The_Cross_Product. I have only ever come across one cross product formula, which is the one where the direction of u x v is determined by the right hand rule. I don't think that I have ever come across a math library or text that defines a left handed cross product. Typically this is taken into account when working with a left handed coordinate system, which is why there are _lh variants of methods that account for handedness before performing a cross.
I never thought about the existence of a left handed cross product before trying to understand this issue, I think documenting cross as right handed has the potential to add more confusion. I don't want to go into math textbook level of details in glam docs.
I think cross_rh and cross_lh would definitely be going against usual linear algebra conventions and would also cause more confusion.
The orthonomal vector methods were contributed but are based on this paper - https://graphics.pixar.com/library/OrthonormalB/paper.pdf, I'll need to read it (it's been a while since I last read it) and get back to you on that.
I agree about the cross product, cross_lh and cross_rh would be confusing. I do still think it worthwhile to point out that the vector returned by cross follows the right-hand-rule, though, simply because it is a deviation from "handedness agnostic"
Also, as I already read it and did the digging required to figure it out, some notes: the handedness is not documented anywhere in the JCGT/pixar paper or in Frisvad's original work, but it does talk about keeping the handedness consistent, fixing an issue with the original implementation where sometimes the opposite handedness is returned. I went to the original implementation source and saw that it was
- generating any orthogonal vector
- crossing it with the input vector for the third
and then I checked the cross product implementation, and it was the standard right-handed one. So it is a right-handed basis being constructed here.