Big day. Policy Gradient. New Concepts Approximate Policy Evaluation and Roll-out utility Policy Optimization methods: Local Policy Search (aka Hooke-Jeeves Policy Search) Genetic Policy Search Cross Entropy Method Policy Gradient, Regression Gradient and Likelyhood Ratio Gradient Reward-to-Go Important Results / Claims monte-carlo policy evaluation Finite-Difference Gradient Estimation Linear Regression Gradient Estimate Questions for next office hour