About 'setMarginalized' and marginal covariance computation

[HeatherHu]

I wonder if the optimization result is affected by setMarginalized(true). Or is this just related with computeCovariance()? I tried set it different to see the time cost. But the results seem to be same whatever it is true or false. I don't totally understand the algorithm for computing marginal convariance in g2o. What I want to do is to obtain camera pose covariance part. I did in following steps: (1) add the current camera pose as a vertex setId(0) ; setEstimate(initial value); setMarginalize(false); setFixed(false); (2) add points setId(i+1) ; setEstimate(initial value); setMarginalize(true); setFixed(true); (3) add edge: setMeasurements(position in image); setInformation(Identity); setRobustKernel(Huber);setDelta(sqrt(5.991)); set camera params(fx,fy,cx,cy) (4) Get covariance after optimization by using the following code: g2o::SparseBlockMatrixXd spinv; optimizer.computeMarginals(spinv,optimizer.vertex(0)); The pose covariance is spinv.block(0,0)->eval(). I did not understand marginalize very well. I am not sure that setMarginalize as true affect the result of optimization or not. I just worried I did something wrong. Is anyone familiar with this? Could you have a look to see if it is a problem anywhere? Thanks very much for your help.

[HeatherHu]

I checked the code again. It seems that setMarginalize(false) related with pose and setMarginalize(true) related with landmark. I am so confusing about this.

[IQ17]

actually this is required by G2O. If you check the G2O source code, you will find out that G2O thinks everything with setMarg TRUE as pose, and everything with setMaeg FALSE as landmark

[IQ17]

sorry, TRUE as landmark, and FALSE as pose

[HeatherHu]

@IQ17 Thanks a lot for your reply. Yeah. I found this in the G2o source code, does this mean that we must setMarginalize as false if it is a pose, otherwise set as true? However, I found the comment saying that 'this node should be marginalized out during the optimization when it set as true'. If I still want the landmark being optimized, can I set it as false? It seems that if false the structure will be change to pose (6dof). It is confusing to me. Also, do you know the marginal covariance algorithm in G2o? Which paper does present this? Thanks again.

[ferreram]

Hi, The setMarginalize() option allows you to make use of the Schur Complement trick to speedup the Bundle Adjustment optimization. By separating the camera poses and the landmarks in your optimization problem you can take advantage of the problem's matrix structure. Usually you have many more landmarks than camera poses in Bundle Adjustment problems so, by setting every landmarks as setMarginalize(true), g2o will first estimate the camera poses in a forward pass and then use the resulting poses to optimize the landmarks in a backward pass. The Schur Complement uses the problem sparsity to speedup its computation and implicitly computes the approximated Hessian to solve the optimization problem. It is this approximated Hessian that is given and computed by computeMarginals(). So, about your first question, if you set your landmarks as fixed it won't make a change in terms of speed to set them as setMarginalize(true). However, this will indeed impact the computeMarginals() function as this one depends on the problem structure and g2o will setup the problem by separating states being set as setMarginalize(true) and ones set as setMarginalize(false). If you google "Schur Complement" and look at its formulation it should be clearer.

Hope it helps!