ML-Agents-Fighting-Animations

Overview

Unity project with the results of the training of an artificial intelligence (AI) with the objective of creating simple humanoid procedural animations of defensive combat movements with a sword against an enemy attack using inverse kinematics (IK) and ML-Agents.

Software Versions

• Unity 2020.1.13f1
• ML-Agents Release 12

Previous Knowledge

Useful links to learn some aspects of this project:

• Inverse kinematics
• Machine learning and reinforcement learning
• Deep learning and deep reinforcement learning
• ML-Agents

Demo Explanation

The demo consists of a list of attack animations that can be selected from the drop-down menu that appears on the screen. When one is selected, it is reproduced and the samurai AI performs the procedural defense animation.

https://user-images.githubusercontent.com/62213937/178051228-0c21dc84-9eff-4415-a53e-a23f6d09b6e7.mp4

Execute demo

Download the following ZIP (build) and execute "TFGProject.exe".

Project Main Components

• Attacker: A humanoid (a knight with a shield and sword) that has premade attack animations that hit the defender.

• Defender (AI): A humanoid (a samurai with his katana) that defense himself (moving the katana and blocking the enemy blow) using the AI.

Creating the Animations

The animations are created by two components:

• An AI that applies a force to the katana in order to move it and make it collide with the attacker's sword at both midpoints of its blades.
• IK applied to the defender's arms so that they follow the movement of the sword created by the AI.

The AI

This artificial intelligence is trained by deep reinforcement learning, an artificial neural network (ANN) with a reinforcement learning algorithm, both implemented by the ML-Agents project.

AI Configuration

The agent in this training environment can move and rotate the sword as if the samurai was moving it with his arms. In more technical terms, the actions that the agent can perform are 5 discrete actions ("move" in the X, Y, and Z axes, and "rotate" in the X and Z axes) with three options each (apply negative value, apply positive value, or not apply any new value).

As for his observations, from the environment, the agent will know the following.

3D Position and rotation:

• Both swords
• Full body of the samurai divided in Head, Spine, Left and Right Arm, Hip, and Left and Right Leg

Velocity:

• Both swords

The next image shows how it is represented in Unity.

Training Configuration

All the configuration data (reinforce algorithm used, ANN settings and hyperparameters, etc) is listed in the next image.

behaviors:
  DefendSamurai:
    trainer_type: ppo
    hyperparameters:
      batch_size: 512
      buffer_size: 12000
      learning_rate: 0.0003
      beta: 0.005
      epsilon: 0.2
      lambd: 0.95
      num_epoch: 3
      learning_rate_schedule: linear
    network_settings:
      normalize: true
      hidden_units: 512
      num_layers: 3
      vis_encode_type: simple
      memory: null
    reward_signals:
      extrinsic:
        gamma: 0.99
        strength: 1.0
    init_path: null
    keep_checkpoints: 5
    checkpoint_interval: 500000
    max_steps: 30000000
    time_horizon: 1000
    summary_freq: 50000

See Training Configuration File in order to understand the previous image.

Explore Project

Software Requirements

• Unity 2020.1 or later
• Python 3.6.1 or later
• ML-Agents release 12 or later *

*See ML-Agents: Installation

Installing

• Download the project and add it to Unity Hub.

Acknowledgments

Third party resources:

• Samurai and katana model
• Knight model and attack animations
• Tatami structure
• Japanes leaf
• Skybox