panda3d
Collision detection is broken when transform cache is disabled
Description
On master branch, on macOS, collision detection between ray and polygon is broken when transform cache is disabled (on Linux with GCC this issue does not occur)
After some digging, I found that the problem is caused by get_wrt_mat() returning a singular matrix here :
https://github.com/panda3d/panda3d/blob/6274b478aad8bac262471c361e677d45804a2295/panda/src/collide/collisionPolygon.cxx#L655
This is because, in https://github.com/panda3d/panda3d/blob/6274b478aad8bac262471c361e677d45804a2295/panda/src/collide/collisionEntry.I#L296
A new temporary transform state is created by get_wrt_space() and a reference to its internal transform matrix is returned as reference. However, the temporary transform state is destroyed as we leave the scope of the method and the matrix becomes invalid.
Enabling transform cache, or switching the code in collisionEntry to return the matrix by value solve the problem.
(This issue is somewhat related to https://github.com/panda3d/panda3d/issues/1625 but occurring this time in the core code)
Steps to Reproduce
Add load_prc_file_data("", "transform-cache 0") to e.g. samples/chessboard/main.py, the pieces are no longer pickable.
Environment
- Operating system: macOS 14.1.1
- System architecture: x86-64
- Panda3D version: master / 6274b478aad8bac262471c361e677d45804a2295
- Installation method: built from source (using optimize level 3)
- Python version (if using Python): 3.9
- Compiler (if using C++): Xcode 15.3
