Deepmind Course Notes

In this series of blog posts, I want to share what I’ve learned from the Deepmind course on Advanced Deep Learning & Reinforcement Learning and some extra things that I’ve discovered from exploring the topics that have been presented in the course but not fully explained. The course slides could be found here.

Introduction

Creating Artificial General Intelligence is of type of thing that could transform our world wildly once again. I think the wonder of this could be expressed precisely by the Deepmind’s slogan: What if solving one problem could unlock solutions to thousands more?
But why Reinforcement Learning? well, there is a belief that Deep Reinforcement Learning could be a foundation for Artificial General Intelligence, so maybe because of that, there is an incredible upward trend towards Reinforcement Learning throughout past years. See this tweet from Chris Manning!

The amazing rise of reinforcement learning!
(With graph neural networks and meta-learning in hot pursuit. ConvNets? Tired.) Based on #ICLR2021 keywords HT @PetarV_93 pic.twitter.com/ozKpNUVH1i

— Christopher Manning (@chrmanning) November 28, 2020

Deepmind is one of the most prominent companies (if not the most!) working in this field. Deepmind’s AlphaGo was a significant breakthrough in the history of this field. AlphaGo succeeded in defeating a professional go player a decade before expected!
“This landmark achievement was a decade ahead of its time.” —Deepmind’s website
Fortunately, Deepmind has released many great courses on different aspects of artificial intelligence literature; one of them is a course on advanced RL and DL, presented by Thore Graepel and some other great guest lecturers.

Before reading

My main goal of writing this series of posts is to provide a complemantory resource for the main course, not to present that once again; So:

• I assume that the reader has basic knowledge about Reinforcement Learning concepts e.g. agent, reward, action, … (if dont, I strongly suggest this Specialization on coursera)
• I also assume that the reader knows about neural networks and deep learning (once again, if don’t, I suggest this Specialization on coursera)
• This posts mainly discuss what not fully explained on the videos, so they have to be read after watching the course classes on youtube.
• I may add some content that has not been mentioned on the course but I found them helpfull to make better understanding.

If you are ready to go, let’s start with first session on Introduction to Machine Learning Based AI!