📔 Statistical Learning Theory

📌 Logistics: These notes have been compiled from E2 237 (Fall 2025), Statistical Learning Theory, taught by Prof. Parimal Parag at IISc (course page). My handwritten notes are in preparation and will be made available in due course.

• Lec-1: Introduction
• Lec-2: Review — linear algebra
• Lec-3: Review — convexity
• Lec-4: Review — constrained optimization
• Lec-5: Support vector machines — separable case
• Lec-6: Support vector machines — non-separable case
• Lec-7: Kernel methods — PDS kernels
• Lec-8: Kernel methods — reproducing kernel Hilbert space
• Lec-9: Sample complexity — probably approximately correct learning
• Lec-10: Sample complexity — Rademacher complexity
• Lec-11: Sample complexity — VC dimensions
• Lec-12: Complexity theory — upper bounds on separating hyperplanes
• Lec-13: Complexity theory — margin based generalization bounds
• Lec-14: Complexity theory — lower bounds
• Lec-15: Statistical decision theory — setting
• Lec-16: Statistical decision theory — Bayes and minimax risk
• Lec-17: Statistical decision theory — minimax theorem
• Lec-18: Statistical decision theory — normed loss and log-concavity
• Lec-19: Divergence — definitions
• Lec-20: Divergence — local behavior
• Lec-21: Fisher information — parametrized family
• Lec-22: Fisher information — local behavior of divergence
• Lec-23: Large scale asymptotics — Minimax lower bounds
• Lec-24: Large scale asymptotics — Bayesian lower bounds
• Lec-25: Information theoretic methods — Mutual information and channel capacity
• Lec-26: Information theoretic methods — Mutual information method
• Lec-27: Reduction to hypothesis testing — Le Cam method
• Lec-28: Reduction to hypothesis testing — Examples
• Lec-29: Reduction to hypothesis testing — Assouad’s Lemma
• Lec-30: Reduction to hypothesis testing — Fano’s method