Awesome_Dynamic_SLAM

The following paper are the papers that focuses on the SLAM in dynamic Environments and Life-long SLAM. In the dynamic environment, there are two kinds of robust SLAM: First is detection & removal. Another is detection & tracking. Although the mapping part in dynamic environment is not my focus, but I will also put some articles yet also very interesting.

Vision means that the pipeline is built with camera. Others are the same, such as lidar, radar, sensor fusion.

Related survey papers:

• A survey: which features are required for dynamic visual simultaneous localization and mapping?. Zewen Xu,CAS. 2021

• State of the Art in Real-time Registration of RGB-D Images. Stotko, Patrick. University of Bonn. 2016

• Visual SLAM and Structure from Motion in Dynamic Environments: A Survey. University of Oxford. 2018

• State of the Art on 3D Reconstruction with RGB-D Cameras. Michael Zollhöfer. Stanford University. 2018

Dynamic Object Detection and Removal

• (IEEE RA-L'22)DynaVINS: A Visual-Inertial SLAM for Dynamic Environments, code:https://github.com/url-kaist/dynaVINS, 非深度学习结合的方案，而是使用约束对运动对象上的特征点进行去除

• DeFlowSLAM: Self-Supervised Scene Motion Decomposition for Dynamic Dense SLAM

  • 浙江大学，貌似是DROID-SLAM得部分，将光流分成了动态物体和静态，分别进行后续的操作。效果看起来也很不错。page，code

• Efficient Spatial-Temporal Information Fusion for LiDAR-Based 3D Moving Object Segmentation

  • 毫末智行，code，动态检测

• [2203.03923] ROLL: Long-Term Robust LiDAR-based Localization With Temporary Mapping in Changing Environments IROS 2022

  • code：https://github.com/HaisenbergPeng/ROLL

• POCD: Probabilistic Object-Level Change Detection and Volumetric Mapping in Semi-Static Scenes

  • RSS 2022，半静态场景中的地图更新

• J. Schauer and A. Nuchter, “The Peopleremover—Removing Dynamic Objects From 3-D Point Cloud Data by Traversing a Voxel Occupancy Grid,” IEEE Robot. Autom. Lett., vol. 3, no. 3, pp. 1679–1686, Jul. 2018, doi: 10.1109/LRA.2018.2801797.

  • 基于体素遍历的方法的运动对象去除，虽然这种方法有很多的缺点，但是论文提出了很多的trick，来解决这些问题，看起来效果还是不错的。
  • code，video

• N. Rufus, U. K. R. Nair, A. V. S. S. B. Kumar, V. Madiraju, and K. M. Krishna, “SROM: Simple Real-time Odometry and Mapping using LiDAR data for Autonomous Vehicles.” IV 2020

  • 较为粗暴的去除可能的运动对象，去除地面后再取出

• M. Schorghuber, D. Steininger, Y. Cabon, M. Humenberger, and M. Gelautz, “SLAMANTIC - Leveraging Semantics to Improve VSLAM in Dynamic Environments,” ICCV 2019 workshop

  • 视觉SLAM，动态环境。通过语义对点的置信度进行计算，高置信度的点来辅助低置信度的点，最终确定用于定位和建图的部分。

• S. Gu, S. Yao, J. Yang, and H. Kong, “Semantics-Guided Moving Object Segmentation with 3D LiDAR,” arxiv 2022.05

  • 动态物体的分割网络，也是rangenet的思想那套。

• Y. Pan, B. Gao, J. Mei, S. Geng, C. Li, and H. Zhao, “SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instances,” IV 2020

  • 动态物体室外数据集，北大，网站

• S. Pagad, D. Agarwal, S. Narayanan, K. Rangan, H. Kim, and G. Yalla, “Robust Method for Removing Dynamic Objects from Point Clouds,” ICRA 2020

  • video，动态去除

• L. Sun, Z. Yan, A. Zaganidis, C. Zhao, and T. Duckett, “Recurrent-OctoMap: Learning State-Based Map Refinement for Long-Term Semantic Mapping With 3-D-Lidar Data,” RAL

  • life long slam

• P. Egger, P. V. K. Borges, G. Catt, A. Pfrunder, R. Siegwart, and R. Dubé, “PoseMap: Lifelong, Multi-Environment 3D LiDAR Localization,” IROS 2018

  • lifelong slam，ETH SAL组

• DynamicFilter: an Online Dynamic Objects Removal Framework for Highly Dynamic Environments，ICRA 2022

  • IJRR大佬，可惜不开源。。，港大，南科大

• X. Ma, Y. Wang, B. Zhang, H.-J. Ma, and C. Luo, “DynPL-SVO: A New Method Using Point and Line Features for Stereo Visual Odometry in Dynamic Scenes.” arXiv, May 17, 2022

  • 使用点和线特征的双目视觉里程计，动态去除的论文
  • 东北大学，也还没有开源

• M. T. Lázaro, R. Capobianco, and G. Grisetti, “Efficient Long-term Mapping in Dynamic Environments,” IROS 2018

  • 高效的ICP方案，并且实现了地图实体的合并。由于处理的是2D地图，因此也就没有那么多的需要处理的东西。可以直接用点可视化来去除运动的点云。
  • code，

• T. Krajník, J. P. Fentanes, J. M. Santos, and T. Duckett, “FreMEn: Frequency Map Enhancement for Long-Term Mobile Robot Autonomy in Changing Environments,” TRO 2017

• G. Kurz, M. Holoch, and P. Biber, “Geometry-based Graph Pruning for Lifelong SLAM.” IROS 2021

  • 我们提出了一种新的方法，该方法考虑了几何准则来选择要剪枝的顶点。这是有效的，易于实现，并导致具有均匀分布的顶点的图形，这些顶点仍然是机器人轨迹的一部分。此外，我们提出了一种新的边际化方法，与现有方法相比，该方法对错误的循环闭包具有更强的鲁棒性。 主要设计到SLAM后端的优化，当地图或者是因子图更新时，如何对因子图进行剪枝的问题。

• Quei-An Chen and Akihiro Tsukada，“Flow Supervised Neural Radiance Fields for Static-Dynamic Decomposition”, ICRA 2022

  • AI葵，code，nerf+光流的动态物体，去除并修复，video。

• W. Ding, S. Hou, H. Gao, G. Wan, and S. Song, “LiDAR Inertial Odometry Aided Robust LiDAR Localization System in Changing City Scenes,” ICRA 2020

  • 百度出品的使用激光和IMU，在运动场景的定位，并且在之前构建的地图中，针对场景新增加的东西，将会新建相关的地图。
  • life-long SLAM

• G. D. Tipaldi, D. Meyer-Delius, and W. Burgard, “Lifelong localization in changing environments,” IJRR 2013

  • life-long的定位

• S. Zhu, X. Zhang, S. Guo, J. Li, and H. Liu, “Lifelong Localization in Semi-Dynamic Environment,” ICRA 2021

  • 清华，life-long的定位

• F. Pomerleau, P. Krüsi, F. Colas, P. Furgale, and R. Siegwart, “Long-term 3D map maintenance in dynamic environments,” ICRA 2014

  • 动态环境中，地图更新

• D. J. Yoon, T. Y. Tang, and T. D. Barfoot, “Mapless Online Detection of Dynamic Objects in 3D Lidar.” Conference on Computer and Robot Vision (CRV) 2019

  • 点云动态检测

• M. Zhao et al., “A General Framework for Lifelong Localization and Mapping in Changing Environment.” IROS 2021

  • 高仙机器人的life-long 定位的论文
  • 多session的地图表示和一种高效的在线地图更新策略，子系统组成：局部激光雷达里程计（LLO）、全局激光雷达匹配（GLM）和位姿图优化（PGR），LLO的作用是构建一系列局部一致的子地图，GLM子系统负责计算传入扫描点云和全局子地图之间的相对约束，并将子映地图和约束插入PGR，PGR是系统中最重要的部分，它从LLO和GLM收集子地图和约束关系，修剪并保存在历史地图中的旧的子地图，并执行姿势图稀疏化和优化。

• D. Henning, T. Laidlow, and S. Leutenegger, “BodySLAM: Joint Camera Localisation, Mapping, and Human Motion Tracking,” *arXiv:2205.02301

  • 人体构建和SLAM相结合，与AirDOS有点类似

• Pfreundschuh, Patrick, et al. “Dynamic Object Aware LiDAR SLAM Based on Automatic Generation of Training Data.” (ICRA 2021)

  • ETH ASL,code, video, dataset, Lidar
  • 作者基于deep-learning（3D-MiniNet网络）进行实时3D动态物体检测，滤除动态物体后的点云被喂给LOAM，进行常规的激光SLAM。提供了学习的方法是属于无监督的方法。

• Canovas Bruce, et al. “Speed and Memory Efficient Dense RGB-D SLAM in Dynamic Scenes.” (IROS 2020)

  • GIPSA-lab, code, video

• Yuan Xun and Chen Song. “SaD-SLAM: A Visual SLAM Based on Semantic and Depth Information.” (IROS 2020)

  • USTC, code, video

• Dong, Erqun, et al. “Pair-Navi: Peer-to-Peer Indoor Navigation with Mobile Visual SLAM.” (ICCC 2019)

  • Tsinghua, Code, Video, Slides.

• Ji Tete, et al. “Towards Real-Time Semantic RGB-D SLAM in Dynamic Environments.” (ICRA 2021)

• Palazzolo Emanuele, et al. “ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals.” (IROS 2019)

  • code,video.University of Bonn.

• Arora Mehul, et al. Mapping the Static Parts of Dynamic Scenes from 3D LiDAR Point Clouds Exploiting Ground Segmentation. p. 6.

• Chen Xieyuanli, et al. “Moving Object Segmentation in 3D LiDAR Data: A Learning-Based Approach Exploiting Sequential Data.” IEEE Robotics and Automation Letters, 2021

  • code. video. University of Bonn.

• Zhang Tianwei, et al. “FlowFusion: Dynamic Dense RGB-D SLAM Based on Optical Flow.”(ICRA 2020)

  • code. video.

• Zhang Tianwei, et al. “AcousticFusion: Fusing Sound Source Localization to Visual SLAM in Dynamic Environments.”,IROS 2021

  • video. 结合声音信号

• 1. Liu Yubao and Miura Jun. “RDS-SLAM: Real-Time Dynamic SLAM Using Semantic Segmentation Methods.” IEEE Access 2021

  2. Liu Yubao and Miura Jun. “RDMO-SLAM: Real-Time Visual SLAM for Dynamic Environments Using Semantic Label Prediction With Optical Flow.” IEEE Access, vol. 9, 2021, pp. 106981–97. IEEE Xplore, https://doi.org/10.1109/ACCESS.2021.3100426.

  • code, video.

• Cheng Jiyu, et al. “Improving Visual Localization Accuracy in Dynamic Environments Based on Dynamic Region Removal.” IEEE Transactions on Automation Science and Engineering, vol. 17, no. 3, July 2020, pp. 1585–96. IEEE Xplore, https://doi.org/10.1109/TASE.2020.2964938.

• Soares João Carlos Virgolino, et al. “Crowd-SLAM: Visual SLAM Towards Crowded Environments Using Object Detection.” Journal of Intelligent & Robotic Systems 2021

  • code, video

• Kaveti Pushyami and Singh Hanumant. “A Light Field Front-End for Robust SLAM in Dynamic Environments.”.

• Kuen-Han Lin and Chieh-Chih Wang. “Stereo-Based Simultaneous Localization, Mapping and Moving Object Tracking.” IROS 2010

• Fu, H.; Xue, H.; Hu, X.; Liu, B. LiDAR Data Enrichment by Fusing Spatial and Temporal Adjacent Frames. Remote Sens. 2021, 13, 3640.

• Qian, Chenglong, et al. RF-LIO: Removal-First Tightly-Coupled Lidar Inertial Odometry in High Dynamic Environments. p. 8. IROS2021, XJTU

• K. Minoda, F. Schilling, V. Wüest, D. Floreano, and T. Yairi, “VIODE: A Simulated Dataset to Address the Challenges of Visual-Inertial Odometry in Dynamic Environments,”RAL 2021

  • 动态环境的数据集，包括了静态，动态等级的场景，感觉适合用来作为验证。
  • 东京大学，code

• W. Dai, Y. Zhang, P. Li, Z. Fang, and S. Scherer, “RGB-D SLAM in Dynamic Environments Using Point Correlations,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1, 2020

  • 浙大，使用点的关联进行去除。

• C. Huang, H. Lin, H. Lin, H. Liu, Z. Gao, and L. Huang, “YO-VIO: Robust Multi-Sensor Semantic Fusion Localization in Dynamic Indoor Environments,” in 2021 International Conference on Indoor Positioning and Indoor Navigation (IPIN), 2021.

  • 使用yolo和光流对运动对象进行判断，去除特征点后进行定位
  • VIO的结合

• Dynamic-VINS：RGB-D Inertial Odometry for a Resource-restricted Robot in Dynamic Environments.

  • 分割+运动点置信度，code,video

Dynamic Object Detection and Tracking

• Visual-Inertial Multi-Instance Dynamic SLAM with Object-level Relocalisation (2022)

  • IROS 2022，实验室网址：https://mlr.in.tum.de/research/semanicobjectlevelanddynamicslam

• Learning to Complete Object Shapes for Object-level Mapping in Dynamic Scenes (2022)，与上面是同一个作者，

  • 都是基于MID-Fusion做的东西。

• T. Ma and Y. Ou, “MLO: Multi-Object Tracking and Lidar Odometry in Dynamic Environment.” arXiv, Apr. 29, 2022

  • SLAM+MOT了

• Z. Wang, W. Li, Y. Shen, and B. Cai, “4-D SLAM: An Efficient Dynamic Bayes Network-Based Approach for Dynamic Scene Understanding,” IEEE Access

  • 语义识别动态后，使用UKF之类的进行动态追踪，但是图的效果不好。

• T. Ma and Y. Ou, “MLO: Multi-Object Tracking and Lidar Odometry in Dynamic Environment.” , ArXiv 2022

  • 基于LOAM的目标追踪，分别对运动对象和自身进行估计，之后进行融合。属于松耦合的感觉

• R. Long, C. Rauch, T. Zhang, V. Ivan, T. L. Lam, and S. Vijayakumar, “RGB-D SLAM in Indoor Planar Environments with Multiple Large Dynamic Objects,”

  • 先做了动态移除，这是动态追踪的。在结构化环境（面）中进行SLAM+MOT

• “AirDOS: Dynamic SLAM benefits from Articulated Objects,” Qiu Yuheng, et al. 2021(Arxiv)

  • code(TBA), Paper, Video. CMU RI, Vision.

• “DOT: Dynamic Object Tracking for Visual SLAM.” Ballester, Irene, et al.(ICRA 2021)

  • code, video, University of Zaragoza, Vision

• Liu Yubao and Miura Jun. “RDMO-SLAM: Real-Time Visual SLAM for Dynamic Environments Using Semantic Label Prediction With Optical Flow.” IEEE Access.

• Kim Aleksandr, et al. “EagerMOT: 3D Multi-Object Tracking via Sensor Fusion.” (ICRA 2021)

  • TUM, code, video, Sensor Fusion.
• 1. Shan, Mo, et al. “OrcVIO: Object Residual Constrained Visual-Inertial Odometry.” (IROS2020)
  2. Shan, Mo, et al. “OrcVIO: Object Residual Constrained Visual-Inertial Odometry.” (IROS 2021)
  • code, video, Project page.

• Rosen, David M., et al. “Towards Lifelong Feature-Based Mapping in Semi-Static Environments.” (ICRA 2016)

• 1. Henein Mina, et al. “Dynamic SLAM: The Need For Speed.” (ICRA 2020)
  2. Zhang Jun, et al. “VDO-SLAM: A Visual Dynamic Object-Aware SLAM System.” (ArXiv 2020)
  3. Robust Ego and Object 6-DoF Motion Estimation and Tracking,Jun Zhang, Mina Henein, Robert Mahony and Viorela Ila. IROS 2020(code)
  • code, video, Vision

• Minoda, Koji, et al. “VIODE: A Simulated Dataset to Address the Challenges of Visual-Inertial Odometry in Dynamic Environments.” (RAL 2021)

  • code_and dataset, video.

• Vincent, Jonathan, et al. “Dynamic Object Tracking and Masking for Visual SLAM.”, (IROS 2020)

  • code, video,

• Huang, Jiahui, et al. “ClusterVO: Clustering Moving Instances and Estimating Visual Odometry for Self and Surroundings.” (CVPR 2020)

  • Tsinghua, code, video.slides.

• Liu, Yuzhen, et al. “A Switching-Coupled Backend for Simultaneous Localization and Dynamic Object Tracking.” (RAL 2021)

  • Tsinghua

• Yang Charig, et al. “Self-Supervised Video Object Segmentation by Motion Grouping.”(ICCV 2021)

  • Project page.

• Long Ran, et al. “RigidFusion: Robot Localisation and Mapping in Environments with Large Dynamic Rigid Objects.” ,(RAL 2021)

  • project page.code, video,

• Yang Bohong, et al. “Multi-Classes and Motion Properties for Concurrent Visual SLAM in Dynamic Environments.” IEEE Transactions on Multimedia, 2021

• Yang Gengshan and Ramanan Deva. “Learning to Segment Rigid Motions from Two Frames.” CVPR 2021

  • Project page.

• Thomas Hugues, et al. “Learning Spatiotemporal Occupancy Grid Maps for Lifelong Navigation in Dynamic Scenes.”

  • code.

• Jung Dongki, et al. “DnD: Dense Depth Estimation in Crowded Dynamic Indoor Scenes.” (ICCV 2021)

  • code, video.

• Luiten Jonathon, et al. “Track to Reconstruct and Reconstruct to Track.”, (RAL+ICRA 2020)

  • code, video. Reconstruct.

• Grinvald, Margarita, et al. “TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction.”(ICRA 2021)

  • code, video.

• Wang Chieh-Chih, et al. “Simultaneous Localization, Mapping and Moving Object Tracking.” The International Journal of Robotics Research 2007

• Ran Teng, et al. “RS-SLAM: A Robust Semantic SLAM in Dynamic Environments Based on RGB-D Sensor.”

• Xu Hua, et al. “OD-SLAM: Real-Time Localization and Mapping in Dynamic Environment through Multi-Sensor Fusion.” * (ICARM 2020)* https://doi.org/10.1109/ICARM49381.2020.9195374.

• Wimbauer Felix, et al. “MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera.” (CVPR 2021)

  • Project page. code. video. video 2.

• Liu Yu, et al. “Dynamic RGB-D SLAM Based on Static Probability and Observation Number.” IEEE Transactions on Instrumentation and Measurement, vol. 70, 2021, pp. 1–11. IEEE Xplore, https://doi.org/10.1109/TIM.2021.3089228.

• P. Li, T. Qin, and S. Shen, “Stereo Vision-based Semantic 3D Object and Ego-motion Tracking for Autonomous Driving,” arXiv 2018

  • 沈邵颉老师组

• G. B. Nair et al., “Multi-object Monocular SLAM for Dynamic Environments,” IV2020

• M. Rünz and L. Agapito, “Co-fusion: Real-time segmentation, tracking and fusion of multiple objects,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), May 2017, pp. 4471–4478.

  • code,

• TwistSLAM: Constrained SLAM in Dynamic Environment,

  • S3LAM的后续，使用全景分割作为检测的前端

Object SLAM & Application

• CubeSLAM: Monocular 3D Object SLAM, IEEE Transactions on Robotics 2019, S. Yang, S. Scherer PDF

  • code. video. Vision

• Salas-Moreno Renato F., et al. “SLAM++: Simultaneous Localisation and Mapping at the Level of Objects.” (CVPR 2013)

  • code, video,

• Nicholson Lachlan, et al. “QuadricSLAM: Dual Quadrics From Object Detections as Landmarks in Object-Oriented SLAM.” (RAL-2018)

  • Project page, code.

• Wu Yanmin, et al. “EAO-SLAM: Monocular Semi-Dense Object SLAM Based on Ensemble Data Association.” 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2020, pp. 4966–73. arXiv.org, https://doi.org/10.1109/IROS45743.2020.9341757.

  • Project page. code.

• H. Osman, N. Darwish, and A. Bayoumi, “LoopNet: Where to Focus Detecting Loop Closures in Dynamic Scenes,” IEEE Robotics and Automation Letters, pp. 1–1, 2022, doi: 10.1109/LRA.2022.3142901.

  • 动态环境中的回环检测，网络。code，video

• M. N. Finean, L. Petrović, W. Merkt, I. Marković, and I. Havoutis, “Motion Planning in Dynamic Environments Using Context-Aware Human Trajectory Prediction,” arXiv:2201.05058 [cs], Jan. 2022.

  • 含有人的动态环境的导航，code，video。类似避障和规划类的

Researchers

🥼1. Berta Bescos

主页：谷歌学术 | 个人主页 | GitHub

博士学位论文： Visual slam in dynamic environments

代表性工作

• [1] Bescos B, Fácil J M, Civera J, et al. DynaSLAM: Tracking, mapping, and inpainting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4): 4076-4083.
• [2] Bescos B, Neira J, Siegwart R, et al. Empty cities: Image inpainting for a dynamic-object-invariant space[C]//2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019: 5460-5466.
• [3] Bescos B, Cadena C, Neira J. Empty cities: A dynamic-object-invariant space for visual SLAM[J]. IEEE Transactions on Robotics, 2020, 37(2): 433-451.
• [4] Bescos B, Campos C, Tardós J D, et al. DynaSLAM II: Tightly-coupled multi-object tracking and SLAM[J]. IEEE Robotics and Automation Letters, 2021, 6(3): 5191-5198.

🥼2. Yubao Liu

Walk Into AI World

主页：谷歌学术 | GitHub

代表性工作：

1. RDS-SLAM: real-time dynamic SLAM using semantic segmentation methods
2. KMOP-vSLAM: Dynamic Visual SLAM for RGB-D Cameras using K-means and OpenPose
3. RDMO-SLAM: Real-time Visual SLAM for Dynamic Environments using Semantic Label Prediction with Optical Flow

3. Guoquan Huang(SLAMMOT)

4. Shenshao Jie(MOT)