General Information

Course Outline

The course deals with various issues related to online learning and decision making under uncertainty:

  • Intro: online learning
  • Online convex optimization:
    • Follow the Leader, Follow the regularized leader,
    • Gradient Descent, Mirror Dscent
    • Convex duality
  • Online classificatrion
  • Partial Information (Multi-Arm Bandits)
  • Regret and Equilibrium (zero-sum games, correlated equilibrium)
  • Stochastic MAB: UCB, Explore/Expoit, Best Arm identification
  • Lower bounds: adversarial and stochastic (intro to Info Theory)
  • Special types of MAB:
    • Contextual
    • linear
    • Markovian (Gittins Index)
    • switching cost
  • MAB and pricing
  • Portfolio section



Homepage|ruosnam#ruosnaM yahsiY .forP


  • Formal prerequisite: Introduction to Machine Learning

Scribe notes

explanation about latex pdf tex

template file pdf tex

scribe list

Home Assignments

Final Project


30% - writing a scribe notes for one of the lectures
30% - Home works
40% - final project


Books and surveys

Learning, Regret minimization, and Equilibria, A. Blum and Y. Mansour

Online Learning and Online Convex Optimization. Shai Shalev-Shwartz

Introduction to Online Convex Optimization, Elad Hazan

Introduction to Multi-Armed Bandits, Alex Slivkins

Bandit Algorithms, Tor Lattimore and Csaba Szepesvari

Related courses

Advanced Topics in Theory of Computing: Bandits, Experts, and Games, Alex Slivkins, University of Maryland at College Park, Fall 2016

Bandit Algorithms, Csaba Szepesvari, University of Alberta, Fall 2016

Algorithms and Uncertainty, Nikhil bansal, UC Berkeley Fall, 2016

Algorithms and Uncertainty, Nikhil Devanur and Anna Karlin, University of Washington, Spring 2017

Prediction and Learning: It's Only a Game, Jacob Abernethy, University of Michigan, Fall 2013

Mathematics of Machine Learning, Prof. Philippe Rigollet, MIT Fall 2015

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License