The Matrix Calculus You Need For Deep Learning
by Terence Parr, Jeremy Howard · explained.ai / arXiv
Explains the vector and matrix derivative notation used in deep learning papers: partial derivatives, Jacobians, element-wise binary operators, the vector and total-derivative chain rules, and gradients of a single neuron and of a full loss function. Assumes only single-variable calculus.
More resources on Mathematics for Machine Learning
StatQuest with Josh Starmer
YouTube channel by Josh Starmer that explains statistics and machine learning step by step, from p-values, linear models and ANOVA to PCA, decision trees, neural networks and transformers. Viewers build clear intuition for how common methods work before tackling the math.
Practical Statistics for Data Scientists
A concise statistics book written for programmers that covers sampling, experimental design, A/B testing, regression, classification, and resampling, with R code throughout. Readers learn which statistical ideas matter for data science and how sampling and study design shape the conclusions data can support.
Matrix Calculus for Machine Learning and Beyond (MIT 18.S096)
Extends calculus to matrices and general vector spaces: derivatives as linear operators, Jacobians, derivatives of matrix factorizations, adjoint methods and automatic differentiation. 17 lecture videos, lecture notes, and problem sets with solutions teach learners to derive and compute gradients for large-scale optimization.
The Matrix Cookbook
A reference compendium of matrix identities: derivatives of determinants, inverses, traces and norms, eigenvalue and SVD relations, and Gaussian and multivariate distribution results. Built for looking up the identity a paper skipped, not for reading front to back.
Introduction to Probability for Data Science
Undergraduate probability text written for data science: combinatorics, discrete and continuous random variables, joint distributions, expectation, transforms, sample statistics, regression and Bayesian inference, with Python and MATLAB code throughout. Free PDF, lecture videos, slides and exercises from the author.