Imath icon indicating copy to clipboard operation
Imath copied to clipboard

Check return value of extractSHRT in computeRSMatrix

Open cary-ilm opened this issue 3 years ago • 0 comments

extractSHRT() returns false if the input matrix is degenerate, and it leaves the output parameters uninitialized. When it is invoked from computeRSMatrix(), the return value is ignored, which means the subsequent code operates on uninitialized data and returns bogus results.

Since computeRSMatrix() returns a matrix, it can't easily return a boolean failure status, but a better solution would be to return a zero matrix Matrix<t>(T(0)), to indicate failure.

The existing code leads to a uninitialized variable reference warning while compiling testMiscMatrix.cpp because the as, ah, ar, and at variables might have failed to have been set:

src/Imath/ImathMatrix.h:4841:46: warning: ‘at.Imath_3_2::Vec3<float>::z’ may be used uninitialized in this function [-Wmaybe-uninitialized]
src/Imath/ImathMatrixAlgo.h:1127:25: note: ‘at.Imath_3_2::Vec3<float>::y’ was declared here
 1127 |     Vec3<T> as, ah, ar, at;

cary-ilm avatar Mar 16 '22 01:03 cary-ilm