Terminate action did not worked out as expected

[dimaxano]

Very rarely I'm getting Cholesky failure error. As I see from verbose information, chi2 increases a lot from previous iteration, so gain should be negative. I added Terminate action with gain threshold to stop when gain appears to be negative:

terminateAction = new ::g2o::SparseOptimizerTerminateAction;
terminateAction->setGainThreshold(1e-4);
optimizer.addPostIterationAction(terminateAction);
optimizer.initializeOptimization();
optimizer.optimize(num_iter);

Here is the verbose info printed (truncated it a bit to make post less bulky):

iteration= 10	 chi2= 14733.445557	 time= 0.00963067	 cumTime= 0.120209	 edges= 15199	 schur= 1	 lambda= 0.005710 levenbergIter= 1
iteration= 11	 chi2= 12210.277196	 time= 0.00939389	 cumTime= 0.129603	 edges= 15199	 schur= 1	 lambda= 0.003807 levenbergIter= 1
iteration= 12	 chi2= 10712.759619	 time= 0.00956701	 cumTime= 0.13917	 edges= 15199	 schur= 1	 lambda= 0.002538 levenbergIter= 1
iteration= 13	 chi2= 10248.724055	 time= 0.0214634	 cumTime= 0.160634	 edges= 15199	 schur= 1	 lambda= 0.000993 levenbergIter= 1
iteration= 14	 chi2= 10137.211749	 time= 0.00931742	 cumTime= 0.169951	 edges= 15199	 schur= 1	 lambda= 0.000364 levenbergIter= 1
iteration= 15	 chi2= 10122.276286	 time= 0.0100389	 cumTime= 0.17999	 edges= 15199	 schur= 1	 lambda= 0.000243 levenbergIter= 1
int g2o::csparse_extension::cs_cholsolsymb(const cs_di*, number_t*, const cs_dis*, number_t*, int*): cholesky failed!
Cholesky failure, writing debug.txt (Hessian loadable by Octave)
iteration= 16	 chi2= 166971700.062054	 time= 0.134642	 cumTime= 0.314631	 edges= 15199	 schur= 1	 lambda= 0.000081	 levenbergIter= 1
int g2o::csparse_extension::cs_cholsolsymb(const cs_di*, number_t*, const cs_dis*, number_t*, int*): cholesky failed!
Cholesky failure, writing debug.txt (Hessian loadable by Octave)
iteration= 17	 chi2= 833675183.716477	 time= 0.134195	 cumTime= 0.448827	 edges= 15199	 schur= 1	 lambda= 0.000027	 levenbergIter= 1
int g2o::csparse_extension::cs_cholsolsymb(const cs_di*, number_t*, const cs_dis*, number_t*, int*): cholesky failed!
Cholesky failure, writing debug.txt (Hessian loadable by Octave)
iteration= 18	 chi2= 250457092.500930	 time= 0.134763	 cumTime= 0.58359	 edges= 15199	 schur= 1	 lambda= 0.000009	 levenbergIter= 1

As you see, on iteration 16 chi2 increased a lot (gain is negative) but optimization still continues further

Are there any piece of advice to make it stop after negative gain?

g2o version (pretty old btw): commit

[sjulier]

Had a quick look, but not sure. Can you instrument up the post iteration action and see if it's called and what the flow of logic is within it?