Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions
by Nathan Halko, Per-Gunnar Martinsson, Joel A. Tropp · SIAM Review / arXiv
SIAM Review survey establishing randomized low-rank approximation. Presents the randomized range-finder, power iteration, and error bounds, after which you can compute approximate SVDs of matrices too large for classical bidiagonalization and reason about accuracy tradeoffs. The primary source behind every randomized SVD implementation in scikit-learn, SLEPc and beyond. Reading it turns randomized SVD from a black-box function call into a method with known failure modes and provable error bounds.
More resources on Singular Value Decomposition
Wolfram MathWorld
MathWorld is an online mathematics encyclopedia from Wolfram Research offering detailed, browsable articles on topics across the math spectrum, including algebra, geometry, calculus, and number theory. Each entry includes definitions, theorems, formulas, diagrams, worked examples, and links to further reading.
Linear Algebra (MIT 18.06)
Matrix theory and linear algebra: systems of equations, elimination, vector spaces and subspaces, orthogonality and least squares, determinants, eigenvalues and positive definite matrices. Includes 35 lecture videos plus problem sets and exams with solutions, giving the fluency needed for applied mathematics, engineering and data science.
Matrix Methods in Data Analysis, Signal Processing, and Machine Learning (MIT 18.065)
Linear algebra for data science and deep learning: singular value decomposition, low-rank approximation, least squares, PCA, randomized linear algebra, gradient descent and the structure of neural networks. Includes 37 lecture videos and problem sets. Afterwards you can recognise the matrix computations inside modern machine learning.
Matrix Computations (4th Edition)
The standard reference for matrix computation. Its SVD chapters give the Golub-Kahan bidiagonalization algorithm, perturbation bounds, and the connection to least squares and rank determination, so you understand how libraries actually compute singular values in floating point.
Computational Linear Algebra for Coders
Free Jupyter-notebook course from the University of San Francisco analytics masters, taught by Rachel Thomas. Covers SVD, truncated and randomized SVD, NMF and robust PCA in Python, so you can implement and benchmark decompositions on real matrices.
A Singularly Valuable Decomposition: The SVD of a Matrix
Expository article from the College Mathematics Journal presenting the SVD via the variational characterization of singular values, then applying it to reduced-rank approximation and least squares. Explains why the decomposition exists, not just how to compute it.