Inserting custom objects into map

[mhkabir]

Hi guys,

We're working on a project where we want to insert certain custom objects into a map and then perform the map optimisation (i.e loop closure).

Essentially, we are running in VIO mode and observing certain objects in the environment (e.g road signs) which are perceived and classified by a separate pipeline. At the end of a run, we have a set of objects in the mission frame from this pipeline. We want to insert these into the VI-map before it is optimised and loop closed, and then perform the optimisation including these objects (so as to finally end up with consistent object poses in the map frame.)

Any suggestions on how to best achieve this would be appreciated!

Regards, Kabir

[mfehr]

Hi @mhkabir Here are some very general pointers:

• There are functions to attach arbitrary sensor measurements to the VIMap, we use it for GPS, but I guess you could also add some transformations or constraints from object matching. Have a look at this.
• There is a plugin to attach external loop closure constraints to the map (just simple 6DOF constraints between parts of the map), maybe this would be useful if you want to go for a more loosely coupled approach. However, this feature currently is not pushed to the public version. If this sounds like something you need, we could put in some effort and release it early.
• If you want the objects to be part of the map, I guess you would need to introduce a new type of landmark of sorts and tie it to the vi_map, however, since I don't know exactly what you plan is it's hard to make any recommendations on how to do this. I have one suggestion though: if the objects contains a lot of data, e.g. dense models or lot's of heavy descriptors. I would look into adding a new resource type to the resource system to store and retrieve this data from the harddrive on demand instead of having it all im memory. Have a look here and here to see what the different options are to attach such data to the VI-map.

Hope this helps, otherwise feel free to contact us and we can discuss it in more detail.