CS 277: Control and Reinforcement Learning
Winter 2021
Note: This course was previously offered as CS 295.
Course logistics
- When: Tuesdays and Thursdays at 5–6:20pm
- Lectures will be recorded and added to this playlist with access for uci.edu accounts.
- Where: zoom
- Announcements and forum: piazza
- Important course announcements will be made on the forum.
- Please post on the forum, publicly or privately, all course-related questions (no emails please).
- Assignments: gradscope
- Published on this page biweekly.
- Instructor: Prof. Roy Fox
- Office hours: calendly
- Enrolled students are welcome to:
- schedule 15-minute slots (more than once if needed);
- give at least 4-hour notice;
- attend individually or with friends.
Grading policy
- Assignments: 72%
- 4 best of 5 assignments count for 18% each.
- No late submission.
- Project: 23%
- Participation: 5%
Schedule
(subject to changes)
Assigments
- Assignment 1; due Friday, January 22 2021 (Pacific Time).
Resources
Courses
- Sergey Levine (Berkeley)
- Pieter Abbeel (Berkeley)
- Dimitri Bertsekas (MIT; also available in book form; also see 2017 book)
- David Silver (UCL)
Books
- Vincent François-Lavet et al., An Introduction to Deep Reinforcement Learning
- Richard Sutton & Andrew Barto, Reinforcement Learning: An Introduction (2nd edition)
- Csaba Szepesvári, Algorithms for Reinforcement Learning
RL libraries
More resources
Further reading
Imitation Learning
- Behavior Cloning with Deep Learning Bojarski et al., CVPR 2016
- DAgger Ross et al., AISTATS 2011
- Goal-conditioned imitation learning Ding et al., NeurIPS 2019
- DART Laskey et al., CoRL 2017
- HVIL Fox et al., Infer2Control @ NeurIPS 2018
Temporal-difference methods
- DQN Mnih et al., Nature 2015
- Double Q-learning van Hasselt et al., NeurIPS 2010
- Double DQN van Hasselt et al., AAAI 2016
- Clipped double Q-learning Fujimoto et al., ICML 2018
- Dueling networks Wang et al., ICML 2016
Policy-gradient methods
- REINFORCE Williams, ML 1992
Academic honesty
Don’t cheat. Academic honesty is a requirement for passing this class. Compromising the academic integrity of this course is subject to a failing grade. The work you submit must be your own. Academic dishonesty includes, among other things, copying answers from other students or online resources, allowing other students to copy your answers, communicating exam answers to other students during an exam, or attempting to use notes or other aids during an exam. If you do so, you will be in violation of the UCI Policy on Academic Honesty and the ICS Policy on Academic Honesty. It is your responsibility to read and understand these policies, in light of UCI’s definitions and examples of academic misconduct. Note that any instance of academic dishonesty will be reported to the Academic Integrity Administrative Office for disciplinary action, and may fail the course.