Matrix Calculus for Machine Learning and Beyond (MIT 18.S096)
by Alan Edelman, Steven G. Johnson · MIT OpenCourseWare
Extends calculus to matrices and general vector spaces: derivatives as linear operators, Jacobians, derivatives of matrix factorizations, adjoint methods and automatic differentiation. 17 lecture videos, lecture notes, and problem sets with solutions teach learners to derive and compute gradients for large-scale optimization.
More resources on Mathematics for Machine Learning
StatQuest with Josh Starmer
YouTube channel by Josh Starmer that explains statistics and machine learning step by step, from p-values, linear models and ANOVA to PCA, decision trees, neural networks and transformers. Viewers build clear intuition for how common methods work before tackling the math.
Practical Statistics for Data Scientists
A concise statistics book written for programmers that covers sampling, experimental design, A/B testing, regression, classification, and resampling, with R code throughout. Readers learn which statistical ideas matter for data science and how sampling and study design shape the conclusions data can support.
The Matrix Cookbook
A reference compendium of matrix identities: derivatives of determinants, inverses, traces and norms, eigenvalue and SVD relations, and Gaussian and multivariate distribution results. Built for looking up the identity a paper skipped, not for reading front to back.
The Matrix Calculus You Need For Deep Learning
Explains the vector and matrix derivative notation used in deep learning papers: partial derivatives, Jacobians, element-wise binary operators, the vector and total-derivative chain rules, and gradients of a single neuron and of a full loss function. Assumes only single-variable calculus.
Introduction to Probability for Data Science
Undergraduate probability text written for data science: combinatorics, discrete and continuous random variables, joint distributions, expectation, transforms, sample statistics, regression and Bayesian inference, with Python and MATLAB code throughout. Free PDF, lecture videos, slides and exercises from the author.