a possible bug for btMatrix3x3::inverse

[zhang-qiang-github]

The btMatrix3x3::inverse code is:

btScalar det = (*this)[0].dot(co);

btAssert(det != btScalar(0.0));
btScalar s = btScalar(1.0) / det;

And the dot code is:

return m_floats[0] * v.m_floats[0] +
			   m_floats[1] * v.m_floats[1] +
			   m_floats[2] * v.m_floats[2];

Sometime, m_floats[0] * v.m_floats[0] is inf, and m_floats[1] * v.m_floats[1] is -inf. inf-inf would return nan. In this way det=nan, and it would reports a bug for btAssert(det != btScalar(0.0));/btScalar s = btScalar(1.0) / det

A solution is: if det=nan, reset it to inf:

if (std::isnan(det))
	{
		det = INFINITY;
	}