Reinforcement Learning · AMS 691
Comparing GRPO and PPO on Traditional Reinforcement Learning Tasks
A comparison of PPO and GRPO in Atari Galaxian, examining training stability and performance when learning from visual observations.
Overview
The group project compares Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO) in the Atari Galaxian environment. It examines the behavior of value-based and group-relative advantage estimates in this visual control task.
Topics & methods
- Train PPO and GRPO with 16 parallel actors over 150 iterations.
- Use a learning rate of 0.0001 and discount factor of 0.99.
- Compare training returns, learning stability, and example gameplay.
Discussion
PPO trained more steadily and achieved higher training returns than GRPO in this configuration.