📔 Information Theoretic Learning

📌 Logistics: This is a self-taught course. The topics are designed by myself mostly following the book: Information Theory: From Coding to Learning by Y. Polyanskiy and Y. Wu.

• Lec-1: Introduction and Probability review.
• Lec-2: Uncertainty Modeling and Motivating Examples
• Lec-3: Entropy and Conditional Entropy
• Lec-4: Submodularity, Han’s inequality and Shearer’s lemma
• Lec-5: Divergence
• Lec-6: Differential Entropy and Markov Kernels
• Lec-7: Conditional Divergence
• Lec-8: Divergence: local behavior and Fisher information
• Lec-9: Mutual Information: intro and examples - coming soon
• Lec-10: Conditional Mutual Information - coming soon
• Lec-11: Probability and Estimation erros - coming soon
• Lec-12: Mutual Information: geometric interpretation - coming soon
• Lec-13: Variational Charaterization of divergence - coming soon
• Lec-14: Gibb’s variational principle - coming soon
• Lec-15: Continuity of divergence - coming soon
• Lec-16: PAC-Bayes - coming soon
• Lec-17: Convexity of information measures - coming soon
• Lec-18: Saddle point and Capacity - coming soon
• Lec-19: f-divergence: definition and properties - coming soon
• Lec-20: Total Variational and Heelinger distances - coming soon
• Lec-21: Inequalities between f-divergences - coming soon
• Lec-22: Comparisions with Total Variational distance - coming soon
• Lec-23: f-divergence: examples and topics - coming soon
• Lec-24: f-divergence: local analysis - coming soon
• Lec-25: Fisher information and Renyi Divergence - coming soon
• Lec-26: f-divergence: Variational representation - coming soon
• Lec-27: Neyman-Pearson formulation and Likelihood test - coming soon
• Lec-28: Bounds of $R(P,Q)$ - coming soon
• Lec-29: Aymtotic analysis - coming soon
• Lec-30: Basics of Large deviation theory - coming soon
• Lec-31: Information Projections - coming soon
• Lec-32: Sanov’s Theorem and further topics - coming soon
• Lec-33: ($E_0,E_1$) trade-off - coming soon
• Lec-34: Sequential Hypothesis testing - coming soon
• Lec-35: Decision Thoery and Gaussian Location Model - coming soon
• Lec-36: Bayes risk and Mini-max risk - coming soon
• Lec-37: Topics on sample complexity and tensor products - coming soon
• Lec-38: Log-concavity and Anderson’s lemma - coming soon
• Lec-39: Lower bounds from data processing - coming soon
• Lec-40: ML estimator: efficiency and applications - coming soon
• Lec-41: Le-Cam’s two-point method - coming soon
• Lec-42: Assouad’s lemma - coming soon
• Lec-43: Fanno’s method - coming soon
• Lec-44: Yang-Barro’s construction - coming soon
• Lec-45: Pariwise comparisions - coming soon
• Lec-46: Yatraco’s class and density estimation - coming soon