The following notes contain Lesson 6 of the David Silver's lecture [1] and Chapter 8 of Shiyu Zhao's Mathematical Foundation of Reinforcement Learning [2].

Reference

[1] https://www.davidsilver.uk/teaching/

[2] https://github.com/MathFoundationRL/Book-Mathmatical-Foundation-of-Reinforcement-Learning