Handouts
All handouts will be posted on this page.
 The following textbook is the excellent introduction to real life
probability concepts:
Data Management 12 Student Edition by Wayne Erdman, Maria Rosa Cruiscuolo,
Roland Meisel, David Petro, Jacob Speijer, Wendy Telford, McGrawHill Education,
ISBN 9781259256363. In particular, read
Chapter 1 "Introduction to Probability". The following two chapters
Permutations (Chapter 2) and
Combinations (Chapter 3) are also useful. It is recommended to read all 3 chapters.
Do all exercises from sections 1.4 and 1.5 in Chapter 1 to make sure you understand well
mutually exclusive events, independent/dependent events and conditional probability.
 Basic facts from probability (a PDF file).

Probabilities and Expectations, a handout prepared by Ashique Mahmood, Sep 9, 2015.
 More information about Normal (Gaussian) distribution:
from MathWorld.
 If you would like to read more about expected value and variance of
discrete random variables, then
this 6th chapter of a textbook written by
Charles Grinstead and J. Laurie Snell can be useful. A
complete textbook is freely available
for download thanks to the American Mathematical Society.
 Selecting an outcome according to a given probability
distribution: this handout will help you to complete Assignment 1.
 A
solution of the gambler's problem using the policy iteration algorithm:
Example 4.3 from the textbook.
This C program was written by Wee Pham.
 A
solution of the Blackjack game using the onpolicy Monte Carlo algorithm:
This C program was written by Wee Pham. Warning: there are a few minor
errors in this program, but nevertheless it demonstrates well the method
of MonteCarlo onpolicy control using epsilongreedy policies.

Reinforcement learning, a brief review paper written by
Peter Dayan & Watkins, CJCH (2001). Published in:
Encyclopedia of Cognitive Science London, England: MacMillan Press.

Reinforcement Learning: A Survey,
by Kaelbling, L.P., Littman, M.L., and Moore, A.W.
Journal of Artificial Intelligence Research (1996), Volume 4, pages 237285.

Reinforcement Learning by Peter Dayan. A detailed overview of
the basic theory underlying reinforcement learning, and its links
with neuroscience, psychology, statistics and engineering. Published in
CR Gallistel (editor), Steven's Handbook of Experimental Psychology, New
York, NY: Wiley.
cps841 (Reinforcement Learning)