Arena-BuildingToolkit

Single/Multi-agent intelligence is still at the stage where many problems remain unexplored. As a part of Arena project, this repository is to provide a building toolkit for researchers to easily invent and build novel single/multi-agent problems. More resources can be found in Arena Home. If you use this repository to conduct research, we kindly ask that you cite the paper as a reference.

Features

Following prefabs can be easily combined with each other, creating your own learning environments.

[	Agent Prefabs	]
BasicAgent	ArenaCrawler	ArenaWalker
BlowBlowAgent	BoomerAgent	RollingAgent
SnakeAgent	TankAgent	TennisAgent

[	Playground Prefabs	]
Playground	PlaygroundWithBarrier	PlaygroundWithDeadWalls
PlaygroundWithLight	PlaygroundWithMaze	PlaygroundWithRooms
PlaygroundWithSlopes	PlaygroundWithWalls	PlaygroundWithDestroyables

Status: Release

We are currently open to any suggestions or pull requests from the community to make Arena a better platform. To contribute to the project, joint us in Slack, check To Do list in Trello as well as check issues opened in the repo. If you have a game in mind you want to study, you are wellcome to contact us via Slack or open an issue. We are happy to implement it for you, if it's helpful for the community.

Table of Contents

Note: bare with me on my typos and open up a pull request to fix them, I appreciate it.

• Arena-BuildingToolkit
  • Status: Release
  • Table of Contents
  • Get Started
    • Folder Structure
    • Scene Structure
      • Essential Scene Structure
      • Suggested Scene Structure
    • Set NodeID
    • Switching Between Cameras
    • Compile Game to Binary File
  • Utilities
    • Visualize Vector Observation
    • Visualize Visual Observation
    • Lidar Sensor
    • Use Transform Reinitializers
    • Dynamically Change Social Tree Structure
    • Log Information
    • UI Percentage Bar
  • Reward Function
    • Reward Function Based on Distance
    • Use Dense Reward Functions
    • Use More Agent Prefabs
  • Agent Prefabs
    • Use Gun to Attack
    • Show Aim Line
    • Gun Attack with Raycast
    • Snake Agent
  • Playground Prefabs
    • Use Eatable Prefabs
    • Playground with Maze
    • Playground with Barriers
  • Randomization
    • Light Randomization
    • Physical Properties Randomization
  • Advanced
    • Upgrade/Change Version of ML-Agents
    • Make Your Game More Neat with Prefabs
    • Handling Variants of Your Games Efficiently
  • Citation
  • License
  • Acknowledgement
  • References

Get Started

This section will take you through the entire pipline of making an Arena game.

Clone the project to your computer, download and install Unity Editor 2018.4.13f1 Personal, download here. Open the project with Unity Editor.

Folder Structure

• Assets
  • ML-Agents: supports from Unity ML-Agents
  • ArenaSDK
    • Brains: where we put shared brains for agents
    • Code Format: where we put a formating config file that can be used if you are using Atom with package atom-beautify. It can be used to format code as we did in the project
    • GameSet: where we put all the games
    • Images: images we are using in the project
    • Materials: materials we are using in the project
    • PhysicMaterial: physic material we are using in the project
    • ThirdPartyAssets: third-party assets used in the project
    • Prefabs: all the prefabs including that of the agents, playgrouds and etc.
    • Scripts: all the scripts

Scene Structure

Note that we will not be demonstrating how to make an Arena game scene from scretch, we will explain how an Arena game is structured. After this, by looking at and playing with all the example games, you will find it easy to create your own variants or a new one from scretch.

Before you start, we are expecting you to have basic knowledge on Unity. Thus, you are recommended to finish the Roll-a-ball tutorial to learn some basic concepts of Unity.

A typical game scene in Arena looks like the following picture, where agents are orginized in a tree hierarchy. At each node, you can change the relationship of the agents under this node by selecting different reward functions. The example below shows that

• under the global node, the two team nodes are competitive, such as which box reaches the target first;
• under the team node, the two agent nodes are collaborative, such as the distance that the box being moved forward;
• at the agent node, the agent receive independent rewards, such as that encouraging moving forward.

The above example game can be found in ./Assets/ArenaSDK/GameSet/ArenaCrawlerPush/Dense-2T2P.

The tree structure can be easily altered by dragging, duplicating, or deleting nodes in the GUI interface. The relationship defined by each node can be easily altered by selecting another reward function from the dropdown list at each node. An demonstration of quick creation of social structures can be found in this [Video].

Essential Scene Structure

The above is achieved by attaching scripts to the scene in the following manner:

• GlobalManager.cs ArenaNode.cs
  • ArenaNode.cs
    • ArenaNode.cs ArenaAgent.cs
    • ArenaNode.cs ArenaAgent.cs
  • ArenaNode.cs
    • ArenaNode.cs ArenaAgent.cs
    • ArenaNode.cs ArenaAgent.cs

Note that there can be hierarchies of more than 3 levels, asymmetry and dynamic hierarchies.

Suggested Scene Structure

The above scene structure is essential to make an Arena environments. However, additional components are suggested:

• GlobalManager.cs ArenaNode.cs
  • TopDownCamera: this is a prefab, looking down at the playgroud top-down. In this camera, the ID of each agent is visable so that you can see how agents of different IDs interact with each other.
  • VisualizationCamera: this is a prefab, looking at the playgroud from nice angle, where you have the best view of the game field.
  • World: this is where you put game objects that belongs to the global world. For example, in the Tennis game, put the ball here.
  • ArenaNode.cs
    • World: this is where you put game objects that belongs to each team. For example, in the game of collaboratively pushing a box, put the box here.
    • ArenaNode.cs ArenaAgent.cs
    • ArenaNode.cs ArenaAgent.cs
  • ArenaNode.cs
    • World
    • ArenaNode.cs ArenaAgent.cs
    • ArenaNode.cs ArenaAgent.cs

To gain more understanding of games under Arena framework and work with them, take a look at more games in the ./Assets/ArenaSDK/GameSet/. Also, contact us if you find any difficulties, we will add documentations accordingly. (For now, the feedback is that it is quite easy to understand and get one's hands on.)

Set NodeID

[Video]

Set valid NodeID.

Switching Between Cameras

One camera takes the whole window at one time, press F1 and F2 to navigate across different cameras.

Compile Game to Binary File

[Video]

Compile your game to a binary file, and train with Arena-Baselines. You may encounter a warning message as follows:

just ignore it. It is promoted because with a plus/pro versions of Unity or if you are an member of this project registered on Unity you get access to things like cloud build and other services, but this isn't needed at all.

If you intend to run your built game on Arena-Baselines, naming the compiled game file should follow specific rules as described in Common Naming Rules. And register the ID of the game in Arena-Baselines according to Register New Games.

Utilities

In the above section, you went through the entire pipline of making an Arena game, hope you enjoy it. Go on with this section to explore more handy utilities in Arena.

Visualize Vector Observation

[Video]

After you add some vector observations to your agent, you may want to check if they make sense for the agent.

• All ArenaAgent are equipped with this visualization tool by default.
• It provides flexible ways to investigate/preview the vector observation.
• It is only enabled in Unity editor, and will not slow down your game when it is compiled for the target platform.

Visualize Visual Observation

[Video]

After you add some visual observations via AgentCamera prefab to your agent, you may want to check if the actual observation for agent (smaller and maybe in grayscale) makes sense.

• All ArenaAgent are equipped with this visualization tool by default.
• It provides flexible ways to investigate/preview the visual observation.
• It is also supported in a compiled game, though not recommended, as it slows down your game.
• The baseline side of Arena project provides better visualization tool for the compiled game.

Lidar Sensor

[Video]

Lidar sensor is widely used in robots to get precise low dimensional representation of the surroundings. Lidar sensor is suitable for robotic SLAM applications. It provides 360 degree scan field, certain rotating frequency with certain ranger distance. Lidar is the ideal sensor in cost sensitive areas like robots consumer and hardware hobbyist. It is also recently used as a representation for game AI, see OpenAI Hide-and-seek. We also included a series of features for visualizing and testing lidar sensor. We would recommend using the game ./Assets/ArenaSDK/GameSet/Tennis as an example to play with lidar sensor.

• Note: When set ScanFramePerSecond to a positive number, you will still get all the data from lidar at each step, the difference is that some of the data is not refreshed, i.e., only part of data is refreshed at each step. This corresponding to the limited scan frequency of a real lidar sensor.

Use Transform Reinitializers

[Video]

Use transform reinitializers to re-initialize objects at the reset of each episode.

Dynamically Change Social Tree Structure

Arena-BuildingToolkit now supports dynamically change social tree structure, during an episode, or at the reset of an episode.

This could open up an interesting research direction of multi-agent learning with agents dies or newly born dynamically. We temporally refer to this kind of problem as "social lifelong learning", in contrast to "lifelong learning" in the context of single agent learning. Say: Can the agent learns to give birth to a specific number of babies, based on the resources it has and the stage of the society / population.

Other interesting directions under this setting could be ad-hoc teamwork where you need to train an agent (or a group of them) to complement an existing set of teammates.

Log Information

[Video]

UI Percentage Bar

[Video]

Identify where the log information comes from.

Reward Function

Reward Function Based on Distance

[Video]

Use reward function based on distance.

Use Dense Reward Functions

[Video]

Use multiple off-the-shelf dense reward functions.

Use More Agent Prefabs

[Video]

Use multiple off-the-shelf agent prefabs in Arena-BuildingToolkit.

Agent Prefabs

Use Gun to Attack

[Video]

Use gun to fire a bullet, which will kill an agent.

Show Aim Line

[Video]

Show an aim line for visualizing your gun attack.

Gun Attack with Raycast

[Video]

Use Raycast for gun attack, which is safer and more efficient than firing a bullet.

Snake Agent

[Video]

Snake is the common name for a video game concept where the player maneuvers a line which grows in length, with the line itself being a primary obstacle. The concept originated in the 1976 arcade game Blockade, and the ease of implementing Snake has led to hundreds of versions (some of which have the word snake or worm in the title) for many platforms. See Wiki Page.

Playground Prefabs

Use Eatable Prefabs

[Video]

Add eatable objects to the game.

Playground with Maze

[Video]

A playground with randomly regenerated maze at the every episode.

Playground with Barriers

[Video]

A playground with randomly regenerated barriers at the every episode.

Randomization

Light Randomization

[Video]

Randomize all lights in the game.

Physical Properties Randomization

[Video]

Randomize physical properties in the game.

Advanced

Upgrade/Change Version of ML-Agents

How to upgrade or change the version of the Unity ML-Agent dependence. Override "Assets/ML-Agents" with the the recent ML-Agent folder here. Apply several changes in ML-Agent:

• change "public BroadcastHub broadcastHub = new BroadcastHub();" in "Assets/ML-Agents/Scripts/Academy.cs" to "[HideInInspector] public BroadcastHub broadcastHub = new BroadcastHub();"
• change "AgentInfo info;" in "Assets/ML-Agents/Scripts/Agent.cs" to "protected AgentInfo info;"
• change "AgentInfo m_Info;" in "Assets/ML-Agents/Scripts/Agent.cs" to "protected AgentInfo info;"

Make Your Game More Neat with Prefabs

[Video]

Suggestions on how to prefab you game so that you can enjoy cleaner project.

Handling Variants of Your Games Efficiently

[Video]

How to handle different configurations of your games with prefabs and prefab variants.

• Prefab - GlobalManager in "Assets/ArenaSDK/Prefabs/" handles settings of all games.
• Prefab Variant - GlobalManager Variant in "Assets/ArenaSDK/GameSet/[cool_game]/" handles settings of the specific game [cool_game].

Citation

If you use Arena to conduct research, we ask that you cite the following paper as a reference:

@inproceedings{song2020arena,
  title={Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence.},
  author={Song, Yuhang and Wojcicki, Andrzej and Lukasiewicz, Thomas and Wang, Jianyi and Aryan, Abi and Xu, Zhenghua and Xu, Mai and Ding, Zihan and Wu, Lianlong},
  booktitle={AAAI},
  pages={7253--7260},
  year={2020}
}

as well as the engine behind Arena, without which the platform would be impossible to create

@article{juliani2018unity,
  title={Unity: A general platform for intelligent agents},
  author={Juliani, Arthur and Berges, Vincent-Pierre and Vckay, Esh and Gao, Yuan and Henry, Hunter and Mattar, Marwan and Lange, Danny},
  journal={arXiv preprint arXiv:1809.02627},
  year={2018}
}

License

Apache License 2.0

Acknowledgement

We give special thanks to the Whiteson Research Lab and ML-Agents team, with which the discussion shaped the vision of the project a lot. Also, we use some free packages from Unity Assets Store list in the reference section, the code of which is publically available in the Unity Assets Store. We thank them for their generosity and contribution to the community.

We also use two packages that is not free (list in the reference section), thus, the code of these two packages is not included in the code release. But the compiled binary files are released in Arena-Baselines.

References

• Tower Defense Template
• Low Poly Crates
• Low Polygon Soccer Ball
• Space Robot Kyle
• Ragdoll and Transition to Mecanim
• Free Little Games Asset Pack
• Video Capture
• SimplePoly Stadium Kit
• Do not shoot Aliens - mobile game
• Space Shooter FREE
• Third Person Controller - Basic Locomotion FREE
• Photon Unity Networking Classic - FREE
• Sun Temple
• House On The Lake
• Sci-Fi Styled Modular Pack
• Nature Starter Kit 2
• Mountain Race Track - Night
• Race Tracks
• Windridge City
• Flooded Grounds
• Rule Engine
• Forge Networking Remastered
• SmartFoxServer2X Multiplayer SDK
• FPSME
• Full Body FPS Controller
• Polygon City Pack - Environment and Interior [Free]
• Easy FPS
• Sport Car - 3D model
• SBP Vehicle Physics
• The Virus war
• Simple Pinball
• Lake Race Track
• Free Barcade Asset Pack
• Snake 3D cartoon with Pads Android Ready
• Chainlink Fences
• Post Processing Stack
• Low Poly Boxing LITE
• Fighting Motions Vol.1
• Fighter Interceptor
• Mixed Motion
• T95 Super Heavy Tank
• Fantasy Defensive Structures
• Tileable Bricks Wall
• Terrain Textures - Snow - Free Samples
• 15 Original Bricks Textures
• Tank - Mark V drivable
• Simple Health Bar FREE
• SciFi Enemies and Vehicles
• Free Trees
• Tree Randomizer
• Rock Pack
• Substance in Unity
• Hand Painted Grass Texture
• Stone Floor Texture
• Unity Particle Pack 5.x
• Maze Generator