Information Theoretic Learning
Graduate Course, IISc, 2023
This is a self-taught course. The topics are designed by myself mostly following the book: Information Theory: From Coding to Learning by Y. Polyanskiy and Y. Wu..
- Lec-1: Introduction and Probability review.
- Lec-2: Uncertainty Modeling and Motivating Examples
- Lec-3: Entropy and Conditional Entropy
- Lec-4: Submodularity, Han’s inequality and Shearer’s lemma
- Lec-5: Divergence
- Lec-6: Differential Entropy and Markov Kernels
- Lec-7: Conditional Divergence
- Lec-8: Divergence: local behavior and Fisher information
- Lec-9: Mutual Information: intro and examples - coming soon
- Lec-10: Conditional Mutual Information - coming soon
- Lec-11: Probability and Estimation erros - coming soon
- Lec-12: Mutual Information: geometric interpretation - coming soon
- Lec-13: Variational Charaterization of divergence - coming soon
- Lec-14: Gibb’s variational principle - coming soon
- Lec-15: Continuity of divergence - coming soon
- Lec-16: PAC-Bayes - coming soon
- Lec-17: Convexity of information measures - coming soon
- Lec-18: Saddle point and Capacity - coming soon
- Lec-19: f-divergence: definition and properties - coming soon
- Lec-20: Total Variational and Heelinger distances - coming soon
- Lec-21: Inequalities between f-divergences - coming soon
- Lec-22: Comparisions with Total Variational distance - coming soon
- Lec-23: f-divergence: examples and topics - coming soon
- Lec-24: f-divergence: local analysis - coming soon
- Lec-25: Fisher information and Renyi Divergence - coming soon
- Lec-26: f-divergence: Variational representation - coming soon
- Lec-27: Neyman-Pearson formulation and Likelihood test - coming soon
- Lec-28: Bounds of $R(P,Q)$ - coming soon
- Lec-29: Aymtotic analysis - coming soon
- Lec-30: Basics of Large deviation theory - coming soon
- Lec-31: Information Projections - coming soon
- Lec-32: Sanov’s Theorem and further topics - coming soon
- Lec-33: ($E_0,E_1$) trade-off - coming soon
- Lec-34: Sequential Hypothesis testing - coming soon
- Lec-35: Decision Thoery and Gaussian Location Model - coming soon
- Lec-36: Bayes risk and Mini-max risk - coming soon
- Lec-37: Topics on sample complexity and tensor products - coming soon
- Lec-38: Log-concavity and Anderson’s lemma - coming soon
- Lec-39: Lower bounds from data processing - coming soon
- Lec-40: ML estimator: efficiency and applications - coming soon
- Lec-41: Le-Cam’s two-point method - coming soon
- Lec-42: Assouad’s lemma - coming soon
- Lec-43: Fanno’s method - coming soon
- Lec-44: Yang-Barro’s construction - coming soon
- Lec-45: Pariwise comparisions - coming soon
- Lec-46: Yatraco’s class and density estimation - coming soon