Key Sequence Notation New Concepts Restricted Gradient and Actor-Critic Exploration and Exploitation with Binary Bandit Undirected Exploration (Explore-then-commit) Directed Exploration (Softmax Method, Quantile Exploration, UCB 1, Posterior Sampling) Important Results / Claims Bayesian Model Estimation and greedy action epsilon-greedy exploration with decay Questions Interesting Factoids