Course Outline
The course deals with various issues related to online learning and decision making under uncertainty:
- Intro: online learning
- Online convex optimization:
- Follow the Leader, Follow the regularized leader,
- Gradient Descent, Mirror Dscent
- Convex duality
- Online classificatrion
- Partial Information (Multi-Arm Bandits)
- Regret and Equilibrium (zero-sum games, correlated equilibrium)
- Stochastic MAB: UCB, Explore/Expoit, Best Arm identification
- Lower bounds: adversarial and stochastic (intro to Info Theory)
- Special types of MAB:
- Contextual
- linear
- Markovian (Gittins Index)
- switching cost
- MAB and pricing
- Portfolio section
Formalities
Staff
li.ca.uat|ruosnam#ruosnaM yahsiY .forP
Prerequisites
- Formal prerequisite: Introduction to Machine Learning
Scribe notes
explanation about latex pdf tex
Home Assignments
Final Project
Grades
30% - writing a scribe notes for one of the lectures
30% - Home works
40% - final project
Reading
Books and surveys
Learning, Regret minimization, and Equilibria, A. Blum and Y. Mansour
Online Learning and Online Convex Optimization. Shai Shalev-Shwartz
Introduction to Online Convex Optimization, Elad Hazan
Introduction to Multi-Armed Bandits, Alex Slivkins
Bandit Algorithms, Tor Lattimore and Csaba Szepesvari
Related courses
Advanced Topics in Theory of Computing: Bandits, Experts, and Games, Alex Slivkins, University of Maryland at College Park, Fall 2016
Bandit Algorithms, Csaba Szepesvari, University of Alberta, Fall 2016
Algorithms and Uncertainty, Nikhil bansal, UC Berkeley Fall, 2016
Algorithms and Uncertainty, Nikhil Devanur and Anna Karlin, University of Washington, Spring 2017
Prediction and Learning: It's Only a Game, Jacob Abernethy, University of Michigan, Fall 2013
Mathematics of Machine Learning, Prof. Philippe Rigollet, MIT Fall 2015