Brummer & Partners MathDataLab mini course:
Randomized methods in linear algebra and their applications in data science
Lecturer: Per-Gunnar Martinsson (Univ. of Texas at Austin)
The lectures will describe a set of recently developed randomized
algorithms for accelerating matrix computations and the analysis of
large data sets. A recurring theme will be the use of randomized
embeddings that reduce the effective dimensionality of data sets
while in certain respects preserving their geometric properties.
We will describe how to use the methods in practice, and how their
performance can be analyzed mathematically.
Three lectures delivered via Zoom, at 15:15 - 16:00 (CET), on November 17, 18, 19.
The lectures will be self-contained, and will in principle assume only knowledge of basic material on
linear algebra and probability theory.
Lecture slides: pdf
Surveys and articles:
Several surveys on randomized linear algebra are available,
see Section 1.7 of this paper
for a partial list.
The following papers follow the presentation in these lectures closely:
E. Liberty, F. Woolfe, P.G. Martinsson, V. Rokhlin, and M. Tygert,
"Randomized algorithms for the low-rank approximation of matrices".
Proceedings of the National Academy of Sciences, 2007 104: 20167-20172.
Local pdf. This is a short paper that succinctly
describes the main ideas behind randomized low rank approximation.
N. Halko, P.G. Martinsson, J. Tropp,
"Finding structure with randomness:
Probabilistic algorithms for constructing approximate matrix decompositions."
SIAM Review, 53(2), 2011, pp. 217-288.
This survey describes randomized low rank approximation in detail, including discussions of
how to use subspace iteration to improve accuracy, structured random maps to reduce
the asymptotic complexity, and single pass algorithms. It contains the full probabilistic
"Randomized methods for matrix computations."
The Mathematics of Data, IAS/Park City Mathematics Series,
25(4), pp. 187 - 231, 2018.
Arxiv.org report #1607.01649.
This survey is written to be accessible to users from a broad range of backgrounds.
It focuses on practical aspects rather than theoretical analysis.
"Randomized Projection Methods in Linear Algebra and Data Analysis."
SIAM News, December 2018. This is a short and easily accessible news piece written to introduce
these methods to a wide audience.
P.G. Martinsson and J. Tropp,
"Randomized Numerical Linear Algebra: Foundations & Algorithms"
Acta Numerica, 2020. Arxiv report 2002.01387.
This is a long survey that aims at summarizing major findings in the field in the past decade.
Tutorial codes implementing RSVD in Matlab.
There is GPU support via Matlab (for supported machines).
Matlab script illustrating a Gaussian embedding.
This is the code used in Lecture 1.
Matlab script generating the error curves in the slides.
(With Sergey Voronin.) CPU and GPU implementations in C of most of the techniques covered:
RSVD, randomized ID and CUR, etc.
(With Mark Tygert.)
FORTRAN and Matlab codes for RSVD, RSFT, interpolative decompositions, etc.
(With Gregorio Quintana-Orti, Nathan Heavner, and Robert van de Geijn.)
Highly optimized implementations of (Householder) column pivoted QR with randomization for pivoting.
Functions with LAPACK compatible interfaces are included.
Research support by:
P.G. Martinsson, November 2020