Solver Core - Solver Crashes When Interrupting Solve with Escape key

[david-cattermole]

Problem

While a solver is running (with the mmSolver command) the user can press the Escape key to stop the solve. Unfortunately, this often causes crashes, like the one on Linux below.

Expected behavior: No crash should happen, instead the Solver should exit the current evaluation and return access of Maya to the user.

Actual behavior: Sometimes/many times crashes happen, causing Maya to crash.

Steps to Reproduce

1. Open Solver UI
2. Set up basic solve.
3. Press "Solve" button to begin solving.
4. While solving, press the "Escape" keyboard key.
5. At the next frame, the solve will stop.

Error:

Solving... frames: 38 
Iteration 0001 | Eval 0001WARNING: User wants to cancel the evaluation!
Stack trace:
  std::string::assign(std::string const&)
  solve_3d_cminpack_lmder(SolverOptions&, int, int, std::vector<double, std::allocator<double> >&, std::vector<double, std::allocator<double> >&, std::vector<double, std::allocator<double> >&, SolverData&, SolverResult&)
  solve(SolverOptions&, std::vector<std::shared_ptr<Camera>, std::allocator<std::shared_ptr<Camera> > >&, std::vector<std::shared_ptr<Marker>, std::allocator<std::shared_ptr<Marker> > >&, std::vector<std::shared_ptr<Bundle>, std::allocator<std::shared_ptr<Bundle> > >&, std::vector<std::shared_ptr<Attr>, std::allocator<std::shared_ptr<Attr> > >&, Autodesk::Maya::OpenMaya20180000::MTimeArray&, std::vector<std::shared_ptr<StiffAttrs>, std::allocator<std::shared_ptr<StiffAttrs> > >&, std::vector<std::shared_ptr<SmoothAttrs>, std::allocator<std::shared_ptr<SmoothAttrs> > >&, Autodesk::Maya::OpenMaya20180000::MDGModifier&, Autodesk::Maya::OpenMaya20180000::MAnimCurveChange&, Autodesk::Maya::OpenMaya20180000::MComputation&, Autodesk::Maya::OpenMaya20180000::MString&, Autodesk::Maya::OpenMaya20180000::MStringArray&, bool, Autodesk::Maya::OpenMaya20180000::MStringArray&)
  MMSolverCmd::doIt(Autodesk::Maya::OpenMaya20180000::MArgList const&)
  THcommandObject::doIt(TargList&)
  TmetaCommand::doCommand(TargList&)
  TpythonInterpreter::dispatchMayaCommand(TmetaCommand&, _object*, _object*)
  TpythonInterpreter::genericDispatchFunction(_object*, _object*, _object*)
  PyObject_Call
  PyEval_CallObjectWithKeywords
  /usr/autodesk/maya2018/lib/libpython2.7.so.1.0(+0xf2f41) [0x7f83ec978f41]
  PyEval_EvalFrameEx
  PyEval_EvalCodeEx
  /usr/autodesk/maya2018/lib/libpython2.7.so.1.0(+0x793f8) [0x7f83ec8ff3f8]
  PyObject_Call
  PyEval_EvalFrameEx
  PyEval_EvalCodeEx
  PyEval_EvalFrameEx
  PyEval_EvalCodeEx
  PyEval_EvalFrameEx
  PyEval_EvalFrameEx
  PyEval_EvalCodeEx
  /usr/autodesk/maya2018/lib/libpython2.7.so.1.0(+0x792f1) [0x7f83ec8ff2f1]
  PyObject_Call
  /usr/autodesk/maya2018/lib/libpython2.7.so.1.0(+0x5c40f) [0x7f83ec8e240f]
  PyObject_Call
  PyEval_CallObjectWithKeywords
  PySide::SignalManager::callPythonMetaMethod(QMetaMethod const&, void**, _object*, bool)
  PySide::SignalManager::qt_metacall(QObject*, QMetaObject::Call, int, void**)
  QMetaObject::activate(QObject*, int, int, void**)
  QAbstractButton::clicked(bool)
  /usr/autodesk/maya2018/lib/libQt5Widgets.so.5(+0x25948a) [0x7f83f61e148a]
  /usr/autodesk/maya2018/lib/libQt5Widgets.so.5(+0x25a4fc) [0x7f83f61e24fc]
  QAbstractButton::mouseReleaseEvent(QMouseEvent*)
  /usr/autodesk/maya2018/lib/python2.7/site-packages/PySide2/QtWidgets.so(+0x41e403) [0x7f82d2076403]
  QWidget::event(QEvent*)
  /usr/autodesk/maya2018/lib/python2.7/site-packages/PySide2/QtWidgets.so(+0x41b6fb) [0x7f82d20736fb]
  QApplicationPrivate::notify_helper(QObject*, QEvent*)
  QApplication::notify(QObject*, QEvent*)
  /usr/autodesk/maya2018/lib/libExtensionLayer.so(+0x1edc8c) [0x7f84014eac8c]

Software Versions

This problem was reported on the below system, but the problem may affect other systems.

• mmSolver version: v0.3.11
• Maya version: 2018
• Operating System (OS): Linux