Why?: Keras implementation of a blog post (Deep Reinforcement Learning: Pong from Pixels) that originally used Python's numpy library for neural network operations.
During testing, the trained model was executed using pure Python in Jupyter Notebook:
Next, this model was implemented using Keras and TensorFlow. Here, I show the model and its one-run reward
The Model that performed the best:
Model 1:
Model 2:
This repository contains two implementations of a Pong game agent - one in pure Python and one using Keras/TensorFlow. The goal is to contrast different approaches to this reinforcement learning problem.
The Python implementation in Pong_game_python_implementation.py follows a policy gradient method using RMSProp and a simple 2-layer neural network model. Key aspects:
State Representation: The Pong screen is preprocessed into an 80x80 1D vector Model Architecture: 2 fully-connected hidden layers, using ReLU activations and Xavier initialization Training: Policy gradient using discounted rewards and RMSProp parameter updates Keras Implementation The Keras implementation in Pong_game_The_keras_model.py uses a Deep Q Learning (DQN) agent with a Convolutional Neural Network (CNN) model. Key aspects:
State Representation: Stores last 4 frames as 84x84 grayscale images Model Architecture: 3 Conv2D layers for feature extraction, followed by fully-connected layers Training: DQN agent with experience replay, target network, and ε-greedy exploration strategy The CNN automatically extracts spatial features from the game screen, instead of manual engineering of the input. Training is done through Q-learning updates based on memories of (state, action, reward) transitions.