Downward looking camera

[Gingol]

Hi Christian, I'm looking at your SVO work and I was wondering which are the characteristics of the algorithm that make it suitable only for downward looking camera. I made some researches and I found that it's related to the keyframes selection, but I don't get in which way. Can you please explain it to me? Thanks in advance

Mauro

[cfo]

Hi Mauro, there are two main issues:

• Keyframe selection: Currently, a keyframe is only selected when we travel 15% of the average scene depth. if your camera is facing forwards, the scene depth is very large and no new keyframes are selected.
• Seed update: When moving forward the seeds are not converging quick enough because the parallax is very small. The problem here is that we are updating the seeds only with frames that come after the seed's creation. With a downlooking camera this is fine because previous frames have not observed that surface before. However, with a forwardfacing camera we should also update the seeds with older keyframes.

If somebody has time to implement this, i gladly accept pull requests.

best, christian

[Gingol]

Thanks for the reply, now it's more clear. So tell me: if I use the svo with a forwardfacing camera looking at a wall and I make only translation movements, it should work?

[cfo]

yes, that should work

[inspirit]

regarding seeds update: what are the difficulties adding N closest/overlapping keyframes to DepthFilter together with New Frame currently being processed? https://github.com/uzh-rpg/rpg_svo/blob/master/svo/src/frame_handler_mono.cpp#L191 here we can call first: setCoreKfs(Config::coreNKfs()); 3 closest overlapping Keyframes will be stored in core_kfs_ i assume new_frame_ that is currently being processed will be there as well and then pass this set to DepthFilter... of course u need to maintain the Depth Filter queue... here i wonder what frames have the priority? (currently u limit queue size to 3 frames)

[taogashi]

@inspirit coreNKfs() only finds nearby keyframes, while this may improve a little, but I think it's better to store a set of last common frames.

[focs]

Does it make sense to use old keyframes to update new seeds? New seed will belong to a new keyframe and by definition keyframes shouldn't have much overlap between them. Wouldn't it make more sense to keep a "smart" list of frames from the recent past? somewhere between the current keyframe (where new seeds were extracted from) and the last/previouse keyframe.

[hashim19]

can i use SVO on a car with camera facing forward? From the above discussion it seems that i cant. If someone has implemented it forward motion ?

[Veerachart]

Hi Christian,

I wonder if there is any update on this issue.

I saw your paper in RSS 2015 (IMU Preintegration on Manifold for Efficient Visual-Inertial Maximum-a-Posteriori Estimation) which integrated IMU with SVO and used it in forward-looking camera. I wonder if there were any changes in SVO part to make that work, or it was the IMU who caused the improvement.

Thanks, Veerachart

[zanazakaryaie]

Hi guys

Thanks for sharing your knowledge and experience

Is it possible to use SVO for a camera installed on a wheeled robot looking the left or right side of the robot?