First-Order Methods in Optimization
by Amir Beck · SIAM (MOS-SIAM Series on Optimization)
Rigorous reference on first-order algorithms for convex problems: subgradient, proximal gradient, FISTA, mirror descent, smoothing, primal-dual methods, ADMM and conditional gradient, with full convergence proofs and a detailed treatment of proximal operators and conjugate functions.
This link may earn us a small commission at no extra cost to you. Affiliate disclosure
More resources on Convex Optimization
CVXPY Documentation and Tutorials
Official documentation for CVXPY, an open-source Python library for modeling convex problems. It includes tutorials on disciplined convex programming, supported solvers, parameters, and worked examples in machine learning, finance, control and signal processing.
Convex Optimization: Algorithms and Complexity
Monograph on the complexity theory of convex optimization: ellipsoid and cutting-plane methods, gradient descent in smooth and non-smooth settings, Nesterov acceleration with lower bounds, mirror descent, interior-point methods, and stochastic and randomized algorithms. Published in Foundations and Trends in Machine Learning.
CMU 10-725: Convex Optimization (Ryan Tibshirani)
Carnegie Mellon graduate course site with slides, scribed notes, homework and linked lecture videos. It focuses on algorithms used in statistics and machine learning: gradient and subgradient methods, proximal methods, Newton's method, duality, ADMM, coordinate descent and stochastic methods.
Geometric Algorithms and Combinatorial Optimization
Research monograph showing how the ellipsoid method and lattice basis reduction yield polynomial-time algorithms for combinatorial optimization. Establishes the equivalence of separation and optimization over polyhedra, then applies it to matroids, submodular functions, perfect graphs and other structured problems.