best PTAM parameters if unlimited computation power is available

[stephanweiss]

On 16.5.13 17:20 , "Kieran" wrote:

[...]

There are lots of settings for the ptam algorithm and I wondered if you could advise the best settings if computation power was unlimited. i.e I don't mind if the computational delay is increased but I want the best pose estimate possible.

Also, is there any way of manually setting the orientation of the ground plane? I would like the ptam 'world' axes to be aligned to the initial image axes. One way I can think of doing this is to take the first reported arbitrary pose(immediately after the baseline movement) and use this as an offset for all following orientations.

Thanks again

Kieran

On 22/05/2013 23:26, Weiss, Stephan wrote:

Hi Keiran,

Thanks for using our framework. If you have unlimited computation power you could use the following (all in the dynamic reconfigure):

• MaxPatchesPerFrame: some high value (e.g. 1000) the more feature you track, the more errors get "averaged"
• UseKFPixelDist = TRUE: this creates a new KF if the new image has a disparity of at least "AutoInitPixel"
• AutoInitPixel=20
• NoLevelZeroMapPoints=TRUE: lvl0 map point usually are very unreliable (see my thesis chapter 2 for stats)
• Only if you work on a planar surface: EpiDepthSearchMaxRange=1 (or smaller): this is the sigma-bound of the existing point cloud within which new features are added. This is an extremely strong rejection criteria for outliers/features not lying in a plane.
• MaxKF=0: do not delete distant KFs, keep them all. NOTE: with UseKFPixelDist=true and MaxKF=0 you will create many KFs - a global bundle adjustment will even bring a very powerful machine to its limits soon => do not change all the other parameters. Furthermore, you want to have the camera rate as fast as possible with no motion blur and sufficient, strong, and evenly distributed features.

As for the handling:

• a clean initialization is crucial. Only translate the camera, try to not apply any rotation in the initialization phase. Many feature tracks improve the initialization, find a good trade-off between disparity of the first two KFs and number of feature tracks (a disparity of about 20-40 pixels for init is ok). You may set AutoInit=true for this - it does some sanity checks and initializes only if some conditions are met. Uncheck this during the run, so you see better when the mapping fails.
• As soon as some KFs are created, stop at each new creation of a KF until the global bundle adjustment has finished.

Aligning the (initial) map: your best bet is to do this in System.cc in the ImageCallback function using

• mpMapMaker->ApplyGlobalScaleToMap() // scale the whole map
• mpMapMaker->ApplyGlobalTransformationToMap() //transform the whole map you can verify if the map is already there/initialized with "mpMap->IsGood()" You probably want to record some PTAM poses first with which you can determine the transformation to your desired coordinate system and only then apply the global map transformation.

Best Stephan

From: Kieran Wood Sent: Thursday, May 23, 2013 4:08 AM

Hi,

Thanks for this. I will be trying it soon (within the next 2 weeks) and will let you know the result.

I am expecting to bring the machine to its limits in order to see if increased accuracy can be gained at the expense of lower rates and increase delay.

Kieran

Kieran,

Great. Any experience and feedback on this from your side will be much appreciated.

[drkieranwood]

Hi, I am still waiting to try these settings. Unfortunately my write-up has taken me away from the lab for a while. Thanks again.

[ank700]

Hello @stephanweiss , I tried ethzasl-ptam with the above mentioned parameters. I would say that I have both good and bad results on the same path. I initialize from a planar surface , then I am able to move randomly till some distance(~5m in my case) i.e. new features are detected all the time. After some time, it seems as if the algorithm stops detecting new features in the image. Also, the number of features detected, in a scene full of features, reduces drastically.

The following is the link to the screen capture for my case. Duration - 3:27 https://www.dropbox.com/sh/mafrba1k2pq2gpd/AABD9Tv7PdGS06xk36_3IKdoa?dl=0

I initialize from a planar surface and then start moving linearly at first. It is a forward looking camera. The problem arises when I rotate 180 deg to go back to the starting point. It does not work, instead I have to move backwards only. Please see the video when you have time. Maybe you might see some mistake that I have made. The above link also contains the params file that I used.

Thanks

[stephanweiss]

Hi,

Thanks for the video, that helps a lot to understand the issue. It seems that you are not deleting old KFs so when you turn at around 2:35 and look from behind a the starting place, PTAM thinks it should see these features (it has no notion of occlusion). The tracking quality of PTAM is defined by the ratio of (features seen)/(features potentially visible) so if you look from behind at the starting poster then this ratio drops remarkably (this is also true, by the way, if you have a lot of clutter and thus occlusion in your scene). There are two ways around that:

a) limit the number of stored key frames such that the front of your starting poster is not mapped any more by the time you "look back". This is, of course, a hack.

b) change the definition of a "good tracking quality" in the PTAM code. (e.g. if either the ratio is ok OR at least N features are found, or something more meaningful and robust...)

Implementing b) is on my list since long. So if you find the time to improve this definition of "good tracking quality" please send us a pull request.

I hope this helps.

Best,

Stephan

---

Von: ank700 [email protected] Gesendet: Montag, 06. Juni 2016 16:31 An: ethz-asl/ethzasl_ptam Cc: Stephan Weiss; Mention Betreff: Re: [ethz-asl/ethzasl_ptam] best PTAM parameters if unlimited computation power is available (#19)

Hello @stephanweisshttps://github.com/stephanweiss , I tried ethzasl-ptam with the above mentioned parameters. I would say that I have both good and bad results on the same path. I initialize from a planar surface , then I am able to move randomly till some distance(~5m in my case) i.e. new features are detected all the time. After some time, it seems as if the algorithm stops detecting new features in the image. Also, the number of features detected, in a scene full of features, reduces drastically.

The following is the link to the screen capture for my case. Duration - 3:27 https://www.dropbox.com/sh/mafrba1k2pq2gpd/AABD9Tv7PdGS06xk36_3IKdoa?dl=0

I initialize from a planar surface and then start moving linearly at first. It is a forward looking camera. The problem arises when I rotate 180 deg to go back to the starting point. It does not work, instead I have to move backwards only. Please see the video when you have time. Maybe you might see some mistake that I have made. The above link also contains the params file that I used.

Thanks

You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ethz-asl/ethzasl_ptam/issues/19#issuecomment-223976438, or mute the threadhttps://github.com/notifications/unsubscribe/AAtHt2vGNZ1QCgQvxP47TnHClPEBY6Viks5qJC9LgaJpZM4ArYeH.

[ank700]

Thank you for your reply. I had set maxkf to 1, so it was saving all the keyframes. I did some experiments with a, it is still not very effective. I will try to define "tracking quality" using number of features found/seen and check for the changes.

After creating a keyframe, does ptam detect other features as well or does it just looks for same features which were detected at the previous kf? I will try with the changes and let you know.

[stephanweiss]

PTAM projects all available 3D map points into the current camera view. Those which end up being projected to the image (i.e. not outside it or behind it) will be searched for (i.e. the process is not related to the previous kf only), let's name this amount N. PTAM will then find M features with M <= N. The tracking quality is then assessed as M/N if this is smaller than a threshold p, the tracking quality is set to "poor" and no additional KFs are generated. If M/N<r (with r<p) then the tracking quality is set to bad, 3 to 5 bad tracking events trigger a map loss and re-localization. So, this tracking quality assessment using M/N should be replaced since it is (obviously) very sensitive to occlusion.

Best,

Stephan

---

Von: ank700 [email protected] Gesendet: Dienstag, 07. Juni 2016 15:19 An: ethz-asl/ethzasl_ptam Cc: Stephan Weiss; Mention Betreff: Re: [ethz-asl/ethzasl_ptam] best PTAM parameters if unlimited computation power is available (#19)

Thank you for your reply. I had set maxkf to 1, so it was saving all the keyframes. I did some experiments with a, it is still not very effective. I will try to define "tracking quality" using number of features found/seen and check for the changes. After creating a keyframe, does ptam detect other features as well or does it just looks for same features which were detected at the previous kf? I will try with the changes and let you know.

You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ethz-asl/ethzasl_ptam/issues/19#issuecomment-224277542, or mute the threadhttps://github.com/notifications/unsubscribe/AAtHt6ix4Fh2BHSQTS1YVxoNJq6CyOnTks5qJW_dgaJpZM4ArYeH.

[ank700]

Hello, Your explanation helped me to understand the problem quickly. I have the following results/questions:

1. Calculating TrackingQuality using just nTotalFound. I have uploaded a bag file and the files that I changed at

https://mega.nz/#F!9VgHkLTD!xqnNgKunTqyxMofejwB0pA

With this set-up, I am able to complete the whole dataset with good number of features, but there is drift of about 1m (if initialization is proper!) Please run the bag file at half the speed otherwise it is unable to detect features. I want to know what you think about it. I am sorry for the inconvenience. The changes are made in the following function- MakeKeyFrame_Lite in keyfrme.cc and assesstrackingquality in tracker.cc

2. When I use OAST16, should I to train the decision tree using test image in order to improve the corner detection in my dataset.

3. Increase the number of pyramid levels - I increased it to 8 and accordingly made changes in the Keyframe.cc file. Currently, the image is scaled by a factor of 2 which renders the last 2-3 pyramid levels (that I added) useless. So, I tried to change the scale value to something between 1 & 2. It gives the following error:

pure virtual method called terminate called without an active exception [ptam-2] process has died [pid 21810, exit code -6, cmd ......

ptam crashes because of image scaling(?). The image size values are rounded off to the nearest integer so a decimal size value is not the problem.

4. Am I thinking right - If I scale the image by a factor smaller than 2, then , ptam should be able to project more features on the image plane (and also find them). This will increase the number of features detected and saved for further computation.

5. If I am wrong above, would it be possible to add another feature detector which is able to detect more features? For example, I am able to detect a lot of features using ORB feature detection. It also detects the features very fast, consistently and even in irregularly changing light conditions.

Best, Ankit

[ank700]

Please excuse my previous very long post. For '1', I have made the following video to show the results -

https://mega.nz/#!FZ5DSTaQ!ClDbDWk2tZ2gh1YyKvr7XFj96ZgxOzj2tdXTCIoHMAM

I set maxkf to 1. Now, it does track throughout the dataset but with some drift. The drift in this video is the least of all the trials that I made. So, I will have to still improve the definition of tracking quality. Currently, I have set tracking quality to good if total features found is more than 20, else poor and left the 2 checks for bad as it is.
I also increased the pyramid levels to 8, although only first 5 are able to detect features. Therefore, I was trying (3) in previous post thinking that ptam could be able to detect more features.

[stephanweiss]

Hi,

the video looks actually very good. PTAM optimizes the camera poses and found features over the whole trajectory/map (bundle adjustment (BA)). In particular at the end of the video it seems that there were some tracking issues (see thoughts about this below). This likely introduces some outliers in the BA and causes the misalignment (i.e. drift) you see. One way to mitigate this would probably be to augment the cornerness threshold at which PTAM decides to consider a FAST corner so it only includes very distinctive (i.e. easy to re-find) features.

As for the motion: At the end of the video you are looking to the left of the initial target. That is area that has not been mapped before I think. At the same time, you do a large rotation with little translation in this new area. Thus, PTAM has only a very small base-line to triangulate the new features, which in turn reduces the accuracy of this new part in the map (you see the issue by the jitter of the grid in the image). Always try to move such that the translational part is large with respect to your rotational part for a given time step.

Hope this helps.

Best, Stephan

---

Von: ank700 [email protected] Gesendet: Freitag, 10. Juni 2016 11:49 An: ethz-asl/ethzasl_ptam Cc: Stephan Weiss; Mention Betreff: Re: [ethz-asl/ethzasl_ptam] best PTAM parameters if unlimited computation power is available (#19)

Please excuse my previous post. For '1', I have made the following video to show the results -

https://mega.nz/#!FZ5DSTaQ!ClDbDWk2tZ2gh1YyKvr7XFj96ZgxOzj2tdXTCIoHMAM

I set maxkf to 1. Now, it does track throughout the dataset but with some drift. The drift in this video is the least of all the trials that I made.

You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ethz-asl/ethzasl_ptam/issues/19#issuecomment-225141195, or mute the threadhttps://github.com/notifications/unsubscribe/AAtHt628frsJjo3syaFYOqMDs1zml8oxks5qKTMcgaJpZM4ArYeH.

[stephanweiss]

I would be surprised if changing the number and steps of pyramidal levels would change anything... I recommend to not change anything there.

---

Von: ank700 [email protected] Gesendet: Donnerstag, 09. Juni 2016 14:03 An: ethz-asl/ethzasl_ptam Cc: Stephan Weiss; Mention Betreff: Re: [ethz-asl/ethzasl_ptam] best PTAM parameters if unlimited computation power is available (#19)

Hello, Your explanation helped me to understand the problem quickly. I have the following results/questions:

1. Calculating TrackingQuality using just nTotalFound. I have uploaded a bag file and the files that I changed at

https://mega.nz/#F!9VgHkLTD!xqnNgKunTqyxMofejwB0pA

With this set-up, I am able to complete the whole dataset with good number of features, but there is drift of about 1m (if initialization is proper!) Please run the bag file at half the speed otherwise it is unable to detect features. I want to know what you think about it. I am sorry for the inconvenience. The changes are made in the following function- MakeKeyFrame_Lite in keyfrme.cc and assesstrackingquality in tracker.cc

2. Increase the number of pyramid levels - I increased it to 8 and accordingly made changes in the Keyframe.cc file. Currently, the image is scaled by a factor of 2 which renders the last 2-3 pyramid levels (that I added) useless. So, I tried to change the scale value to something between 1 & 2. It gives the following error:

pure virtual method called terminate called without an active exception [ptam-2] process has died [pid 21810, exit code -6, cmd ......

ptam crashes because of image scaling(?). The image size values are rounded off to the nearest integer so a decimal size value is not the problem.

3. Am I thinking right - If I scale the image by a factor smaller than 2, then , ptam should be able to project more features on the image plane (and also find them). This will increase the number of features detected and saved for further computation.

4. If I am wrong in 3, would it be possible to add another feature detector which is able to detect more features? For example, I am able to detect a lot of features using ORB feature detection. It also detects the features very fast (~33ms as mentioned in this paper http://webdiis.unizar.es/~raulmur/MurMontielTardosTRO15.pdfhttp://webdiis.unizar.es/%7Eraulmur/MurMontielTardosTRO15.pdf). I am able to detect a lot of features consistently and even in irregularly changing light conditions.

You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ethz-asl/ethzasl_ptam/issues/19#issuecomment-224875153, or mute the threadhttps://github.com/notifications/unsubscribe/AAtHtytegpqgxXmfeiCmpbe1z71dMd7Pks5qKAEmgaJpZM4ArYeH.