Imath
Imath copied to clipboard
Check return value of extractSHRT in computeRSMatrix
extractSHRT() returns false if the input matrix is degenerate, and it leaves the output parameters uninitialized. When it is invoked from computeRSMatrix(), the return value is ignored, which means the subsequent code operates on uninitialized data and returns bogus results.
Since computeRSMatrix() returns a matrix, it can't easily return a boolean failure status, but a better solution would be to return a zero matrix Matrix<t>(T(0)), to indicate failure.
The existing code leads to a uninitialized variable reference warning while compiling testMiscMatrix.cpp because the as, ah, ar, and at variables might have failed to have been set:
src/Imath/ImathMatrix.h:4841:46: warning: ‘at.Imath_3_2::Vec3<float>::z’ may be used uninitialized in this function [-Wmaybe-uninitialized]
src/Imath/ImathMatrixAlgo.h:1127:25: note: ‘at.Imath_3_2::Vec3<float>::y’ was declared here
1127 | Vec3<T> as, ah, ar, at;