
Advanced Numerical Linear Algebra  AMSC/CMSC 763, Fall 2021

Basic Information



Overview


The course will survey contemporary topics in numerical linear algebra,
including
solution algorithms for sparse linear systems of equations, numerical
methods for solving eigenvalue problems,
linear algebra issues arising in machine learning,
and approximate and probabilistic methods in linear algebra.
Both theoretical and computational issues will be studied.


Outline of Topics Covered

 Iterative Methods
 Krylov subspace methods:
the conjugate gradient, MINRES, GMRES methods and generalizations
 Preconditioning methods: uses and derivation
 Multigrid methods for partial differential equations
Algebraic multigrid
 Communicationavoiding methods
 Computational Methods for Eigenvalue Problems
 Lanczos and Arnoldi methods
 The implicitly restarted Arnoldi method
 Subspace iteration
 The JacobiDavidson method
 Linear Algebra and Machine Learning
 Gradient descent and stochastic gradient descent methods
 The LevenbergMarquardt method for optimization
 BFGS for optimization
 Principal components analysis
 Approximate and Probabilistic Methods in Linear Algebra
 Lowrank computations
 Probabilistic methods
 Tensor methods
 The CUR factorization


References

There will be no assigned text.
Much of the material can be found in the references listed below.
These are largely affordable books and it will be useful to obtain
one or two of them.

J. W. Demmel,
Applied Numerical Linear Algebra,
SIAM Publications, Philadelphia, 1997.

H. C. Elman, D. J. Silvester and A. J. Wathen,
Fast Solvers and Finite Elements,
Oxford University Press, Oxford, 2005.

G. H. Golub and C. Van Loan,
Matrix Computations,
Third Edition, Johns Hopkins University Press, 1996.

A. Greenbaum,
Iterative Methods for Solving Linear Systems,
SIAM Publications, Philadelphia, 1997.

Y. Saad,
Iterative Methods for Sparse Linear Systems,
Second Edition, SIAM Publications, Philadelphia, 2003.

Y. Saad,
Numerical Methods for Large Eigenvalue Problems,
Manchester University Press, Manchester, 1992.

G. Strang,
Linear Algebra and Learning from Data,
WellesleyCambridge Press, 2019.

G. W. Stewart,
Matrix Algorithms Volume II: Eigensystems,
SIAM Publications, Philadelphia, 2001.

D. S. Watkins,
The Matrix Eigenvalue Problem: GR and Krylov Subspace Methods,
SIAM Publications, Philadelphia, 2007.
Both references by Saad are available at
http://wwwusers.cs.umn.edu/~saad/books.html.
Several of the books published by SIAM are available in electronic format, at no cost to
SIAM members, through the SIAM web page, https://siam.org/. SIAM membership is free for
UMD students who use the UMD VPN to connect.


Grading

Grades will be determined as follows:
 46 homework assignments: 40%
 Inclass midterm examination: 25%
 Course Project: 35%
The homework will consist of both analysis and computational testing.
Computations can be done using any convenient programming tools.
 
Other Policy Matters

Plagiarism:
You are welcome to discuss assignments in a general way among yourselves,
but you may not use other students' written work or programs.
Use of external references for your work should be cited.
Clear similarities between your work and others will result in a grade
reduction for all parties.
Flagrant violations will be referred to appropriate university authorities.
Masking in classrooms:
President Pines provided
clear expectations
to the University about the wearing of masks for students, faculty, and staff. Face coverings over the nose and mouth are required while you are indoors at all times. There are no exceptions when it comes to classrooms and laboratories. Students not wearing a mask will be given a warning and asked to wear one, or will be asked to leave the room immediately. Students who have additional issues with the mask expectation after a first warning will be referred to the Office of Student Conduct for failure to comply with a directive of University officials.

 
