--- title: Singular Value Decomposition description: Singular value decomposition factorizes a real or complex matrix into three constituent matrices. Learners will understand how to apply this technique for dimensionality reduction, data compression, and solving ill-posed linear systems. category: mathematics subcategory: numerical-linear-algebra difficulty: beginner, intermediate, advanced url: /subject/singular-value-decomposition --- # Singular Value Decomposition Singular value decomposition factorizes a real or complex matrix into three constituent matrices. Learners will understand how to apply this technique for dimensionality reduction, data compression, and solving ill-posed linear systems. ## Available Resources 1 Videos • 1 Books • 3 Courses • 1 Websites • 2 Papers ## Courses ### 1. Linear Algebra (MIT 18.06) **Author:** Gilbert Strang Matrix theory and linear algebra: systems of equations, elimination, vector spaces and subspaces, orthogonality and least squares, determinants, eigenvalues and positive definite matrices. Includes 35 lecture videos plus problem sets and exams with solutions, giving the fluency needed for applied mathematics, engineering and data science. **Difficulty:** Intermediate | **Price:** Free **Link:** https://ocw.mit.edu/courses/18-06-linear-algebra-spring-2010/ **Tags:** linear-algebra, vector-spaces, eigenvalues, least-squares, matrix-factorization ### 2. Matrix Methods in Data Analysis, Signal Processing, and Machine Learning (MIT 18.065) **Author:** Gilbert Strang Linear algebra for data science and deep learning: singular value decomposition, low-rank approximation, least squares, PCA, randomized linear algebra, gradient descent and the structure of neural networks. Includes 37 lecture videos and problem sets. Afterwards you can recognise the matrix computations inside modern machine learning. **Difficulty:** Intermediate | **Price:** Free **Link:** https://ocw.mit.edu/courses/18-065-matrix-methods-in-data-analysis-signal-processing-and-machine-learning-spring-2018/ **Tags:** singular-value-decomposition, matrix-factorization, principal-component-analysis, optimization, deep-learning ### 3. Computational Linear Algebra for Coders **Author:** Rachel Thomas Free Jupyter-notebook course from the University of San Francisco analytics masters, taught by Rachel Thomas. Covers SVD, truncated and randomized SVD, NMF and robust PCA in Python, so you can implement and benchmark decompositions on real matrices. **Difficulty:** Intermediate | **Language:** English | **Price:** Free **Link:** https://github.com/fastai/numerical-linear-algebra **Tags:** svd, numerical-linear-algebra, matrix-decomposition, python, pca ## Papers ### 1. Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions **Author:** Nathan Halko, Per-Gunnar Martinsson, Joel A. Tropp SIAM Review survey establishing randomized low-rank approximation. Presents the randomized range-finder, power iteration, and error bounds, after which you can compute approximate SVDs of matrices too large for classical bidiagonalization and reason about accuracy tradeoffs. The primary source behind every randomized SVD implementation in scikit-learn, SLEPc and beyond. Reading it turns randomized SVD from a black-box function call into a method with known failure modes and provable error bounds. **Difficulty:** Advanced | **Language:** English | **Price:** Free **Link:** https://arxiv.org/abs/0909.4061 **Tags:** randomized svd, low-rank approximation, randomized algorithms, numerical linear algebra, matrix decomposition ### 2. A Singularly Valuable Decomposition: The SVD of a Matrix **Author:** Dan Kalman Expository article from the College Mathematics Journal presenting the SVD via the variational characterization of singular values, then applying it to reduced-rank approximation and least squares. Explains why the decomposition exists, not just how to compute it. **Difficulty:** Intermediate | **Language:** English | **Price:** Free **Link:** https://dankalman.net/AUhome/pdffiles/svd.pdf **Tags:** svd, matrix-decomposition, low-rank-approximation, least-squares, linear-algebra ## Books ### 1. Matrix Computations (4th Edition) **Author:** Gene H. Golub, Charles F. Van Loan The standard reference for matrix computation. Its SVD chapters give the Golub-Kahan bidiagonalization algorithm, perturbation bounds, and the connection to least squares and rank determination, so you understand how libraries actually compute singular values in floating point. **Difficulty:** Advanced | **Language:** English | **Price:** Paid **Link:** https://www.amazon.com/dp/1421407949?tag=edmonddante07-20 **Tags:** numerical linear algebra, svd, matrix algorithms, least squares, floating point arithmetic ## Youtubes ### 1. Singular Value Decomposition [Data-Driven Science and Engineering] **Author:** Steve Brunton Seven-hour YouTube series from a University of Washington professor covering SVD theory, matrix approximation, pseudoinverse, PCA, eigenfaces, optimal rank truncation and randomized SVD, with MATLAB and Python code, so you can apply it to real datasets. **Difficulty:** Beginner | **Language:** English | **Price:** Free **Link:** https://www.youtube.com/playlist?list=PLMrJAkhIeNNSVjnsviglFoY2nXildDCcv **Tags:** svd, pca, matrix-approximation, randomized-svd, linear-algebra ## Videos ### 1. Lecture 29: Singular Value Decomposition (MIT 18.06) **Author:** Gilbert Strang Derives the factorization A = UΣVᵀ from the eigenvectors of AᵀA and AAᵀ, works numerical examples, and connects the singular vectors to orthonormal bases for the four fundamental subspaces. The lecture video prepares learners to construct an SVD by hand. **Difficulty:** Intermediate | **Language:** English | **Price:** Free **Link:** https://ocw.mit.edu/courses/18-06-linear-algebra-spring-2010/resources/lecture-29-singular-value-decomposition/ **Tags:** singular-value-decomposition, linear-algebra, eigenvalues, matrix-factorization, four-fundamental-subspaces ## Websites ### 1. Wolfram MathWorld **Author:** Eric W. Weisstein MathWorld is an online mathematics encyclopedia from Wolfram Research offering detailed, browsable articles on topics across the math spectrum, including algebra, geometry, calculus, and number theory. Each entry includes definitions, theorems, formulas, diagrams, worked examples, and links to further reading. **Difficulty:** Intermediate | **Language:** English | **Price:** Free **Link:** https://mathworld.wolfram.com **Tags:** mathematics-reference, encyclopedia, abstract-algebra, number-theory, geometry --- *This content is part of Dantes.io - Your Treasure Map to Knowledge* *Curated by humans at Dantes.io. Personal study use welcome; republishing this curation requires permission (team@dantes.io).* View this page online: https://dantes.io/subject/singular-value-decomposition