Proximal Policy Optimization is Easy with Tensorflow 2 | PPO Tutorial

Proximal Policy Optimization (PPO) has emerged as a powerful on policy actor critic algorithm. You might think that implementing it is difficult, but in fact tensorflow 2 makes coding up a PPO agent relatively simple.

We're going to take advantage of my PyTorch code for this, as it serves as a great basis to expand on. Simply go to my github and copy the code, and then follow along.

Code for this video is here:
https://github.com/philtabor/Youtube-Code-Repository/tree/master/ReinforcementLearning/PolicyGradient/PPO/tf2

A written crash course to PPO can be found here:
https://www.neuralnet.ai/a-crash-course-in-proximal-policy-optimization/

Learn how to turn deep reinforcement learning papers into code:

Get instant access to all my courses, including the new Prioritized Experience Replay course, with my subscription service. $29 a month gives you instant access to 42 hours of instructional content plus access to future updates, added monthly.


Discounts available for Udemy students (enrolled longer than 30 days). Just send an email to [email protected]

https://www.neuralnet.ai/courses

Or, pickup my Udemy courses here:

Deep Q Learning:
https://www.udemy.com/course/deep-q-learning-from-paper-to-code/?couponCode=DQN-JUNE-22

Actor Critic Methods:
https://www.udemy.com/course/actor-critic-methods-from-paper-to-code-with-pytorch/?couponCode=AC-JUNE-22

Curiosity Driven Deep Reinforcement Learning
https://www.udemy.com/course/curiosity-driven-deep-reinforcement-learning/?couponCode=ICM-JUNE-22

Natural Language Processing from First Principles:
https://www.udemy.com/course/natural-language-processing-from-first-principles/?couponCode=NLP-JUNE-22

Reinforcement Learning Fundamentals
https://www.manning.com/livevideo/reinforcement-learning-in-motion

Here are some books / courses I recommend (affiliate links):
Grokking Deep Learning in Motion: https://bit.ly/3fXHy8W
Grokking Deep Learning: https://bit.ly/3yJ14gT
Grokking Deep Reinforcement Learning: https://bit.ly/2VNAXql

Come hang out on Discord here:
https://discord.gg/Zr4VCdv

Need personalized tutoring? Help on a programming project? Shoot me an email! [email protected]

Website: https://www.neuralnet.ai
Github: https://github.com/philtabor
Twitter: https://twitter.com/MLWithPhil

Time stamps:
0:00 Intro
01:17 Code restructure
01:57 PPO Memory
03:05 Network classes
08:41 Agent class
24:39 Main file
25:54 Moment of Truth