Q-learning implementation for Taxi-v3 environment

[ArshiaIlaty]

Proposal

Code Overview

1. Q-Learning Agent (QLearningAgent class):

   • Implements a Q-learning algorithm with epsilon-greedy exploration
   • Maintains a Q-table to learn state-action values
   • Features include:
     • Epsilon decay for reducing exploration over time
     • Handling of action masks (valid actions)
     • Learning rate and discount factor configuration

2. Training Function (train_taxi()):

   • Trains the agent for a specified number of episodes
   • Uses a progress bar to track training
   • Tracks and stores episode rewards
   • Periodically reports average reward and current epsilon value

3. Testing Function (test_agent()):

   • Evaluates the trained agent in the Taxi environment
   • Renders the environment for visual demonstration
   • Prints total reward for each episode

Environment Details

The Taxi-v3 environment is a grid-world problem where an agent must:

• Pick up a passenger at one of four locations
• Drop the passenger at another specified location
• Navigate efficiently while avoiding invalid moves

Motivation

Training agents improvement and I can expand it to the other agents, such as Cliff Walking Agent

Pitch

No response

Alternatives

No response

Additional context

No response

Checklist

• I have checked that there is no similar issue in the repo

[pseudo-rnd-thoughts]

How is this unique for the tutorials that already exist? Other than being for a difference environment?

[ArshiaIlaty]

The uniqueness of this work lies in the specific focus on the Taxi and CliffWalking environments, for which comprehensive tutorials and code implementations for Q-learning are currently lacking. While tutorials for environments like FrozenLake and Blackjack with Q-learning are readily available and were referenced, I noticed the gap in resources for these other environments.

To address this, I created and provided the necessary code for the Taxi and CliffWalking environments to help fill that gap and make it easier for others to explore and learn. Please find the attached files for your reference.

Screen.Recording.2024-12-18.at.11.15.50.AM.mov