ethzasl_ptam icon indicating copy to clipboard operation
ethzasl_ptam copied to clipboard
verified publisher icon ethz-asl

PTAM Nan crash

Open icoderaven opened this issue 12 years ago • 6 comments

Hi I've noticed this issue quite a few times, and wanted to know if others face this as well. I'm running PTAM on the Pandaboard running Ubuntu 12.04 and Fuerte, and if I try to initialize a map with very little movement between consecutive keyframes, I get a bunch of error messages You sucker, tried to give me a nan and then the node becomes unresponsive. I set the BundleDebugMessages parameter to be true, and then I see Nuked 0 measurements. printed in an infinite loop. My main worry is that I see this error sometimes when AutoInit is set to be true, and PTAM crashes mid flight. Should I just handle it with an exception, or are worse things happening? Thanks!

icoderaven avatar Jun 03 '13 23:06 icoderaven

Update: I just pulled the latest branch (after seeing some commits that seemed relevant) and ptam no longer compiles. I'm on it right now. (I was on b1eb83797092f95309299cce8f888ea5d83a7628 - arch native)

icoderaven avatar Jun 04 '13 23:06 icoderaven

Hi icoderaven,

the odroid (ARM) branch is not officially supported and we might not keep this branch always upstream compatible. So it might take a while until we find the time to make it compile with the newest commits. I therefore suggest to keep the b1eb83797092f95309299cce8f888ea5d83a7628 commit for now.

As to your previous message: This is a BundleAdjustment crash. The initial baseline for the stereo init is used internally as a measure of scale for the global map. Therefore if this distance is very low the triangulation of the map is poorly constrained. Later many keyframes get added to a map putting high load on the BA thread which then will not manage to keep the map clean from bogus map-point measurements. I assume this is then leading to wrong convergence in the non-linear optimization. Handling this with an exception will not help if your map is corrupted in the first place.

Try to have sufficient baseline for the init.

Simon

simonlynen avatar Jun 05 '13 08:06 simonlynen

Thanks for the excellent explanation Simon! Well, I often see this issue when I enable autoInit, so maybe I could put in a small check for the distance of the baseline and throw it if it's too small? Alternately, I could maybe increase the AutoInitPixel parameter? What would you recommend? I think I see this issue generally when I am moving over low texture regions.

icoderaven avatar Jun 05 '13 15:06 icoderaven

Hi,

If AutoInit is enabled we do the following during the initialization process

  • take a first frame as soon as you enable AutoInit
  • on every next camera frame we de-rotate the feature tracks using ESM (i.e. the SmallBlurryImage rotation estimation) and check if the mean disparity is larger than AutoInitPixel with respect to the first frame. (This is essentially the check for distance you are suggesting)
  • a larger value in AutoInitPixel gives you a larger initial base line. A value between 20 and 40 pixels is reasonable.

If you already have AutoInitPixel between 20 and 40 pixels, then your issue might arise from:

  • too much rotation during initialization (the ESM de-rotation method is all but precise)
  • bad camera calibration (did you verify your calibration?)
  • We included a more conservative check on the map health and require at least the number of TrackingQualityFoundPixels to be tracked successfully in each camera frame. In low-texture regions you might end up below this threshold and trigger continuously a map re-init. If you lower this value, remember that PTAM is robust mainly because errors are averaged over many features, tracking only few features inevitably lowers the overall performance.

I hope this helps. Best Stephan

From: Kumar Shaurya Shankar [[email protected]] Sent: Wednesday, June 05, 2013 8:38 AM To: ethz-asl/ethzasl_ptam Subject: Re: [ethzasl_ptam] PTAM Nan crash (#20)

Thanks for the excellent explanation Simon! Well, I often see this issue when I enable autoInit, so maybe I could put in a small check for the distance of the baseline and throw it if it's too small? Alternately, I could maybe increase the AutoInitPixel parameter? What would you recommend? I think I see this issue generally when I am moving over low texture regions.

— Reply to this email directly or view it on GitHubhttps://github.com/ethz-asl/ethzasl_ptam/issues/20#issuecomment-18985074.

stephanweiss avatar Jun 05 '13 17:06 stephanweiss

Hi Stephan So, the only way I could get it to reliably work was to run PTAM offboard and use the compressed image stream from the camera on board. I also set the TrackingQualityGood to 0.4 and QualityLost to 0.3 so that we reinitialize the map whenever the quality drops and I can now get good pose estimation of over 12 meters loops on grass, which is very impressive. :+1: I think it was the low framerate on the raw image that was causing the ESM to stutter; I haven't seen the 'You sucker ... ' messages again after the change. The only hiccup I have happens sometimes when PTAM thinks that it is doing a good job, but its estimated plane is messed up. I shall investigate it a bit more though and let you know exactly when that happens.

icoderaven avatar Jun 24 '13 23:06 icoderaven

Hi,

Thanks for this feedback. Indeed, a slow frame rate and large motion at the same time is not good for PTAM. What happens in this case is the following:

  • at each frame we have sufficient base-line such that PTAM creates a new KeyFrame (nothing wrong with that so far)
  • Creating a new KeyFrame is THE computational expensive part in PTAM so a slow processor (like the pandaboard) takes a while to process it)
  • since the base line is large (large motion, slow framerate) many new features are visible and have to be triangulated: this takes time
  • in the meantime the MAV flies further generating already a new large base-line
  • once the KF is finally generated, PTAM already finds itself with having to create another one because the base-line it gained in the meantime.

To overcome this issue you can use a more powerful CPU, reduce the cam resolution (calibrate the cam afterwards) or: Reduce "MaxPatchesPerFrame", enable "UseKFPixelDist" (requests a new KF already after detecting a pixel disparity of "AutoInitPixel" reducing the amount of newly added features per KF), enable "NoLevelZeroMapPoints" and augment the FAST thresholds (Thres_lvl[0-3]) to reduce the corners to be processed (careful with that, do sufficient tests after changing these params).

Best Stephan

From: Kumar Shaurya Shankar [[email protected]] Sent: Monday, June 24, 2013 4:25 PM To: ethz-asl/ethzasl_ptam Cc: stephanweiss Subject: Re: [ethzasl_ptam] PTAM Nan crash (#20)

Hi Stephan So, the only way I could get it to reliably work was to run PTAM offboard and use the compressed image stream from the camera on board. I also set the TrackingQualityGood to 0.4 and QualityLost to 0.3 so that we reinitialize the map whenever the quality drops and I can now get good pose estimation of over 12 meters loops on grass, which is very impressive. I think it was the low framerate on the raw image that was causing the ESM to stutter; I haven't seen the 'You sucker ... ' messages again after the change. The only hiccup I have happens sometimes when PTAM thinks that it is doing a good job, but its estimated plane is messed up. I shall investigate it a bit more though and let you know exactly when that happens.

— Reply to this email directly or view it on GitHubhttps://github.com/ethz-asl/ethzasl_ptam/issues/20#issuecomment-19943234.

stephanweiss avatar Jun 28 '13 21:06 stephanweiss