Course Description

• This course will provide a comprehensive introduction to reinforcement learning, a powerful approach to learning from interaction to achieve goals in stochastic and uncertain environments. Reinforcement learning has adapted key ideas from machine learning, operations research, control theory, psychology, and neuroscience to produce some strikingly successful engineering applications. The focus is on algorithms that learn what actions to take, and when to take them, so as to optimize long-term performance. This may involve sacrificing immediate reward to obtain greater reward in the long-term or just to obtain more information about the environment. The course will cover Markov decision processes, dynamic programming, temporal-difference learning, Monte Carlo reinforcement learning methods, and the integration of learning and planning.
• Prerequisites: Interest in learning approaches to artificial intelligence; basic probability theory (MTH304 should be OK); computer programming ability (one of the modern programming languages: C, Java, Prolog).
• Text Book:  Andrew Barto and R. S. Sutton Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998. (Clicking on the title will take you to a full description of the book, from which you can obtain a detailed look at what will be covered in this course.).

Course Policies

• Examinations: The quizzes, midterm test, and the final exam may include short essay and yes/no questions, as well as problem solving (but not programming questions). The duration of these examinations will be 15, 1h30min, and 2h30 minutes, respectively. To pass this course you must get a grade of 40% or better on your final exam. There will be no supplemental examination. Brief quizzes may be given without notice (marks will contribute to class participation).


• Assignments should be submitted on or before the deadline specified in the assignment (you are encouraged to submit assignments earlier). Your assignment is considered late if any part of the assignment is late. Late penalties are as follows: 10% off for each day late. No assignments will be accepted if more than 2 days late. Exception: Late submissions will be accepted without a penalty only if you provide documented medical reasons (or other emergencies). The same policy applies if you miss a test (fill in an official form).
• From time to time, I will hand out exercises. The students are expected to solve the exercises, but they will not be graded. However, working on exercises will improve your understanding of this course (and will help you to get better marks on tests).
• Up to 5% extra credit may be assigned for class participation (a student attends the entire class, participates actively by asking/answering questions, works on problems in class, submits a correct solution and signs the participation sheet). Unexplained lack of attendance can negatively affect one's grade.
• Handouts and assignments will be made available on the Web only. You are responsible for visiting the course Web pages regularly and reading assignments and tests related information that is provided or linked from these Web pages. Before sending your questions by e-mail to the instructor, check these Web pages whether similar questions have been already answered. Grades for tests and assignments will be posted on the Grades Web page no later than two weeks after the due date (test date).

Remarking Policy

1. Read carefully the marking guide for the assignment or test you'd like to be remarked.
2. Fill in this remarking form (available online).
3. Give the form and your assignment/test to TA who marked your work or to the instructor (at lecture time or scheduled office hour), who will forward it to a TA.
4. If you are not satisfied with the TA's remarking, you can appeal to the instructor.
5. You may not submit a remarking request later than ONE WEEK from the date on which the assignments/tests were returned in class. It's your responsibility to pick up your work ASAP.
6. Your mark can decrease if TA sees something that was incorrectly awarded too high a mark.

Tentative Course Calendar (all changes of dates will be announced)

 

Course Work	Due Date	Grade Value (%)
Assignment  1	September 26, Monday	10
Assignment   2	October 24, Monday	10
Midterm	October 28, Friday, 4-6pm	20
Assignment  3	November 21, Monday	10
Class participation & quizzes	In assigned class time	5
Final Exam	Exam period	45
		100