Machine Learning Course - CS 156
Yaser Abu-Mostafa
Master theoretical machine learning with Yaser Abu-Mostafa's renowned "Learning From Data" course. Explore fundamental concepts and algorithms.
More resources on Theoretical Machine Learning
Distill.pub
Peer-reviewed web journal of machine learning explanations, publishing interactive articles on topics like feature visualisation, attention and neural network interpretability. Archive remains readable, though the journal went on indefinite hiatus in 2021 and no longer publishes.
Information Theory, Inference, and Learning Algorithms
David MacKay's textbook, free to read online, covering entropy, data compression, noisy-channel coding, error-correcting codes, Bayesian inference, Monte Carlo methods and neural networks. Readers come to understand Shannon's coding theorems and apply probabilistic reasoning to communication and learning problems.
High-Dimensional Statistics (MIT 18.S997)
Finite-sample analysis of high-dimensional methods: concentration inequalities, sparse linear regression, matrix estimation, principal component analysis and minimax lower bounds, ending with open research questions. Lecture notes and problem sets present the proof techniques behind modern optimality guarantees.
Mathematics of Machine Learning (MIT 18.657)
A mathematically rigorous introduction to statistical learning theory: generalization bounds, VC dimension, Rademacher complexity, convex surrogates and optimization, boosting, and online learning. 194 pages of lecture notes and problem sets with solutions teach learners to analyze why learning algorithms work.
Algorithmic Aspects of Machine Learning (MIT 18.409)
Designing machine learning algorithms with provable guarantees: nonnegative matrix factorization, topic models, tensor decomposition, sparse recovery, dictionary learning and learning mixture models. Moitra's detailed lecture notes, which read as a complete monograph, and problem sets teach rigorous analysis of learning algorithms.
Understanding Machine Learning: From Theory to Algorithms
A rigorous textbook treatment of statistical learning theory: PAC learning, VC dimension, uniform convergence, regularization, boosting, and SVMs. Readers finish able to state and prove generalization bounds and reason about why learning algorithms work.