godot_rl_agents_plugin icon indicating copy to clipboard operation
godot_rl_agents_plugin copied to clipboard

AIController refactor

Open Ivan-267 opened this issue 2 years ago • 0 comments

This branch was made to show one possible way to refactor AIController to address some ideas mentioned on Discord. It hasn't been well tested yet. Feedback, suggestions, alternatives to the approach, and/or contributions are all welcome.

What it tries to address:

  • Removes the need for having separate AIController2D/3D node with duplicated code, making it simpler to add future changes.
  • Adds a default implementation for get_reward() so it does not have to be implemented in the extended controller (but still can if needed).
  • Makes the controller reset itself when done = true, and it emits a signal reset_started, which the player and/or any other script can connect to to take care of resetting its own state as needed.
  • Removes the dependency on _player from AIController. The extended AIController can then add the reference to Player or any other node as needed using the correct type / class for better code completion.

This changes the way that the AIController would be used a bit.

As an example, here are some changes I've made when testing this with RingPong (which uses the AIController) to controller.gd:

extends AIController
@export var player: Node3D

(I'm adding the node in the inspector, but node name reference can be used as well)

	var ball_pos = player.to_local(player.ball.global_position)
	var ball_vel = player.to_local(player.ball.linear_velocity)

to_local is now called on the player object since AIController is now a Node (does not have access to to_local), this should be fine since theoretically AIController does not need to have a position itself or move with the player.

and the following is removed from the extended controller since there's a default implementation now:

- func get_reward() -> float:	
-	return reward

And in Player.gd resetting the controller can be simplified to:

func game_over():
	ai_controller.done = true

and the init call is no longer necessary so this part was removed:

- func _ready():
-	ai_controller.init(self)

and also this part was changed since AIController (base) resets itself:

-	if ai_controller.needs_reset:
-		ai_controller.reset()
-		ball.reset()
-		return

To handle resetting the rest of the environment (the ball in this case), player connects to the AIController's signal and uses:

Godot_v4 1-stable_mono_win64_3zif1ofYiu

Alternatively, the ball could connect to the signal and reset itself as needed (the same goes for any other node that needs resetting). The exact implementation would depend on the environment designer's choice.

Note: The name is AIController3D in the screenshots since I didn't rename the node, but it would be called AIController by default with these changes.

This should also address: https://github.com/edbeeching/godot_rl_agents_plugin/issues/17

Ivan-267 avatar Jul 27 '23 15:07 Ivan-267