:sectnums: :sectnumlevels: 1 :toc: macro :toclevels: 2

image::doc/images/UnionLogo.png["UnionLogo", width=800px]

toc::[]

== Overview In traditional Multi-Agent Exploration (MAE) methods, all agents start in the same place and start with a shared coordinate system; and once the relative transformation missed due to localization failures, the MAE system will be very fregial and easily crash. This framework provide a novel Autonomous Exploration paradigm, all the agents can start in the darkness, without knowing anything about where they are, where each other are, and what the environment looks like. As the MAE system explorer individually, MetaUnion helps to find associated overlaps among different agents and solve the above three challenges gradually; and most importantly, MetaUnion can incrementally update the feature learning ability for new observed areas. In a summary, our key contributions includes:

• Multi-Agent Localiztion without Initial Transformation;
• Multi-Agent Hierarchical Exploration;
• Incremental Lifelong Localization.

=== System Structure

image::doc/images/MetaUnion.png["MetaUnion", width=850px]

MetaUnion mainly contains three key modules within the framework: AutoMerge, MrExplorer, and BioSLAM. In the begining, each agent is randomly assigned at a new envionrment, and has neither information about other agents' localization, nor place recognition ability for the new areas, and conduct single-agent exploration individually. When the trajectries of different agents have overlaps, AutoMerge module can automatically detect the data association between agents, and estimate the relative positions for them. Based on the relative transformation, MrExplorer can divide the agent into different groups based on their connections, and apply hierarichal exploration for individual agent. Parallel with the above procedure, BioSLAM constructs a dual-memory system to memorize the large-scale and long-duration place features. As the MAS system explorer to bigger area, their cooperation ability and place recognition ability are improved gradually.

== Software

To quickly install all the necessary softwares for MetaUnion, please following the instructions, [source,shell]

git clone --recurse-submodules -j8 https://github.com/MetaSLAM/MetaUnion.git

On Server

sh scripts/install_server.sh

On Client

sh scripts/install_client.sh source ~/.bashrc

Please keep in mind, the scripts will automatically generate PATH into the bashrc, and you can use [source,bash]

METASLAM=/home/maxtom/codespace/MetaUnion source /home/maxtom/codespace/MetaUnion/stack/devel/setup.bash

on client

alias enclient='sh /home/maxtom/codespace/MetaUnion/scripts/start_client.sh'

on server

alias enserver='sh /home/maxtom/codespace/MetaUnion/scripts/start_server.sh'

general command

alias cdw='cd /home/maxtom/codespace/MetaUnion' alias killmeta='sh /home/maxtom/codespace/MetaUnion/scripts/killall.sh'

In the following sections, we will investigate the details of each module.

=== MrExplorer

image::doc/images/MrExplorer.png["System", width=800px]

Since all agents begin from unknown darkness area within an given map, each agent are starting from a local exploration policy based on its observations. And based on the AutoMerge module, small group agents will find their relative transformations, then such agents are grouped into one set, and the single agent exploration policy are transfermed into multi-agent exploration policy. Gradually, all the agents are converted into one group. This hierarchical strategy help crowd agents quickly understand their coopeartion status with each others, and can balance the exploration efficiency and robustness without any initial position and relative transformation information.

.API: [white blue-background]#Input# and [white red-background]#Output# [NOTE]

[options="interactive"]

• [white blue-background]#Odometry for each agent#
• [white blue-background]#Grouping status from AutoMerge#
• [white red-background]#Waypoints for each agent# ====

=== AutoMerge

image::doc/images/AutoMerge.png["System", width=800px]

AutoMerge can provide an automatic data-association for agents without any knowledge about initial position and each others. When agents upload the observations to the cloud, the agents can detect their associated overlaps (if exists), and send back the grouping results among different agent groups.

.API: [white blue-background]#Input# and [white red-background]#Output# [NOTE]

[options="interactive"]

• [white blue-background]#Odometry from individual agents#
• [white blue-background]#SubMaps from individual agents#
• [white red-background]#Grouping Indexes#
• [white red-background]#Relative Transformations# ====

.TODO List [NOTE]

[options="interactive"]

• [ ] [red yellow-background]#Onine SubMap Generation#
• [ ] [red yellow-background]#Onine Feature Evaluation#
• [ ] [red yellow-background]#Onine Cache Checking#
• [ ] [red yellow-background]#Onine Position Estimation# ====

=== BioSLAM

image::doc/images/BioSLAM.png["System", width=800px]

We present BioSLAM, a lifelong SLAM framework for learning various new appearances incrementally and maintaining accurate place recognition for previously visited areas.Unlike humans, artificial neural networks suffer from catastrophic forgetting and may forget the previously visited areas when trained with new arrivals. For humans, researchers discover that there exists a memory replay mechanism in the brain to keep the neuron active for previous events. Inspired by this discovery, BioSLAM designs a gated generative replay to control the robot's learning behavior based on the feedback rewards.Specifically, BioSLAM provides a novel dual-memory mechanism for maintenance: 1) a dynamic memory to efficiently learn new observations and 2) a static memory to balance new-old knowledge. When combined with a visual-/LiDAR- based SLAM system, the complete processing pipeline can help the agent incrementally update the place recognition ability, robust to the increasing complexity of long-term place recognition.

.API: [white blue-background]#Input# and [white red-background]#Output# [NOTE]

[options="interactive"]

• [white blue-background]#Dual Memory Assistance#
• [white blue-background]#Long-short Memorization#
• [white red-background]#Regular Updated Models# ====

.TODO List [NOTE]

[options="interactive"]

• [ ] [red yellow-background]#Online Feature Updating#
• [ ] [red yellow-background]#Strategy Selection#
• [ ] [red yellow-background]#Group Waypoints Estimation# ====

== Multi-Agent Exploration

=== Run MrExplorer

[source, bash]

cdw enserver #on server enclient #on client

.TODO List [NOTE]

[options="interactive"]

• [ ] [red yellow-background]#Change the name of the project#
• [ ] [red yellow-background]#Integrate the configuration of the client into the config file#
• [ ] [red yellow-background]#Debug the global planning#
• [ ] [red yellow-background]#Integrate the multi-master into this# ====

=== FakeMerge

[source, bash]

git clone https://github.com/MetaSLAM/fakemerge.git fakemerge cd fakemerge catkin_make source devel/setup.bash roslaunch fakemerge transformPose.launch

== Multi-Agent Localization

[source,yaml]

DATA: OFFLINE_LENGTH: 100 # Set for desire testing length

Use OFFLINE_LENGTH to set the length for each agent.

[source,bash]

python src/offline_merging.py

In the current global_rough_align step, we will use spectral clustering method to divide agents into different groups based on their connections.

=== Visualization

Outside the docker, subscibe /global_map with world frame with rviz. Different un-merged maps will be visualized along the z-axis (index*30).

== Lifelong Localization

See the demo tutorial in the link:tests/demo.ipynb[jupyter] and link:tests/test.py[python] version.

=== Configure Datasets

Download Pittsburgh datasets for online map merging. https://drive.google.com/drive/folders/19AK8jc6yZpKN6Ub_ILGJaceZixpbwKcV?usp=sharing

Download pre-trained models and set path [source,bash]

cd data && sh download.sh echo "export ROS_IP='172.17.0.1' " >> ~/.bashrc echo "export MASTER_IP='172.17.0.2' " >> ~/.bashrc echo "export ROS_MASTER_URI=http://$MASTER_IP:11311/ " >> ~/.bashrc echo "export BAG_PATH='$PATH_TO_PITT_ROS_BAGS' " >> ~/.bashrc source ~/.bashrc

And the following difference matrix in the data/results/. Then set ROS_IP to enable communication between docker and host computer. Finally, export dataset path, and replace PATH_TO_PITT_ROS_BAGS to the Pittusbrugh rosbags.

== TODO List