Machine Learning Theory

The course covers basic theories of modern machine learning: 1. statistical learning theory: generalization, uniform convergence, Rademacher complexity, VC theory, reproducing Hilbert kernel space and their applications on simple classification/regression models; 2. optimization theory: gradient descent, stochastic gradient descent and their convergence analyses for convex functions, nonconvex functions 3. deep learning theory: basic approximation, optimization and generalization results for deep neural networks; 4. reinforcement learning theory: MDP, Bellman equations, planning, and sample complexity results for value iteration/Q-learning.