[EXE] Cartpole reinforcement learning exercise

[Macro1027]

Learning Goals

1. Understand the basics of reinforcement learning and Q-learning.
2. Learn how to set up and use the Gymnasium environment.
3. Implement a simple Q-learning algorithm from scratch.
4. Train a Q-learning agent to solve a specific task in the Gymnasium environment.
5. Evaluate the performance of the trained Q-learning agent.

Exercise Statement

In this exercise, you will create a reinforcement learning (RL) exercise that trains a simple Q-learning agent from scratch using the Gymnasium environment. The goal is to implement a Q-learning algorithm that allows an agent to learn and improve its performance in a given task through interactions with the environment.

You will:

1. Set up the Gymnasium environment.
2. Implement the Q-learning algorithm.
3. Train the Q-learning agent.
4. Evaluate the agent's performance.

Prerequisites

1. Basic understanding of Python programming.
2. Familiarity with reinforcement learning concepts.
3. Knowledge of Gymnasium (formerly OpenAI Gym) environments.
4. Understanding of Q-learning and its key components (states, actions, rewards, Q-table).

Data Source/Summary

This exercise uses the Gymnasium environment, a toolkit for developing and comparing reinforcement learning algorithms. The specific environment used in this exercise is the CartPole-v1 environment, where the goal is to balance a pole on a moving cart.

(Optional) Suggest/Propose Solutions

1. I have a solution using PyTorch and Gymnasium, and I can create a pull request to include the exercise statement and solution.
2. Chapter 6 of "Reinforcement Learning: An Introduction" by Sutton and Barto provides a detailed explanation of Q-learning.
3. The tutorial on Q-learning from DataCamp (https://www.datacamp.com/tutorial/introduction-q-learning-beginner-tutorial) offers a step-by-step guide to implementing Q-learning.

(Optional) Further Links/Credits to Relevant Resources

1. This exercise and solution proposal are inspired by the tutorial "Reinforcement Q-Learning from Scratch in Python with OpenAI Gym" (https://www.learndatasci.com/tutorials/reinforcement-q-learning-scratch-python-openai-gym/).
2. The PyTorch tutorial on Deep Q-Learning (https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html) provides additional insights into implementing Q-learning with neural networks.
3. The GitHub repository "Q-Learning-Algorithm-in-Python-with-Cart-Pole-OpenAI-Gym" (https://github.com/AleksandarHaber/Q-Learning-Algorithm-in-Python-with-Cart-Pole-OpenAI-Gym--Gymnasium-Environment) contains a practical implementation of Q-learning.