Survival Guide for Students of Optimization
Dianne P. O'Leary, September 2017
Here is some of the folklore helpful to any serious worker in optimization.
The premier journals include SIAM Journal on Optimization and Mathematical Programming. Other journals to watch include control theory journals (e.g., IEEE Transactions on Automatic Control) and Parallel Computing.
Standard Reference Books:
Some of the classics in the field are:
George B. Dantzig, Linear Programming and Extensions, Princeton 1963.
R. Fletcher, Practical Methods of Optimization, Wiley 1987.
P.E. Gill, W. Murray, and M. H. Wright, Practical Optimization, Academic Press 1981.
David G. Luenberger, Linear and Nonlinear Programming (alias ``Luenberger's blue book"), Addison-Wesley 1984.
David G. Luenberger, Optimization by Vector Space Methods (alias ``Luenberger's red book"), Wiley 1969.
Olvi L. Mangasarian, Nonlinear Programming, McGraw-Hill 1969, reprinted by SIAM.
Helpful books providing linear algebra background are:
G.H. Golub and C. van Loan, Matrix Computations, Johns Hopkins (latest edition).
G.W. Stewart, Introduction to Matrix Computations, Academic Press 1973.
R.A. Horn and C.R. Johnson, Matrix Analysis and Topics in Matrix Analysis, Cambridge 1985 and 1991.
Good software at Netlib:
is a public repository for numerical software and related information. Information about the latest additions to the repository are contained in the newsletter SIAM News and in NA-Digest.
The Linpack project of the 1970's created a set of codes for solving linear systems and related problems. The documentation is J. Dongarra et al, LINPACK Users' Guide, SIAM Press. These codes are also available through
After 20 or 30 years of use, the Linpack and related Eispack codes for eigenvalue problems were becoming shopworn (so to speak), so a group of researchers undertook the project of unifying the two sets of codes and adapting them to achieve faster performance on very large problems and to reduce the effects of round-off error.
The Lapack codes
are available through Netlib, and SIAM publishes a user's guide.
The series of books called Numerical Recipes... provide simplified versions of many standard numerical algorithms; they work on some problems but are not robust. Linpack, Lapack, and the Matlab codes are to be preferred.
The Templates Project:
For solving large, sparse linear systems, iterative methods are useful. Software and documentation are available at
Matlab is a commerical product that contains an interface to the Lapack routines, other matrix manipulation routines, and applications packages in graphics, optimization, signal processing, control, and other areas. It is a standard tool for serious computation, and a standard idiom of communication among researchers in the area. Matlab is distributed by
The MathWorks, Inc.,
which maintains a public repository of software. There is also a Matlab Digest that is available by e-mail. To subscribe, send a message to email@example.com. There are many Matlab tutorials; use your search engine on ``matlab tutorial" to locate one.
NA-Digest and OPT-Net:
About once a week, NA-NET News Digest is sent by e-mail to all members of the ``Numerical Analysis Net" mailing list. This contains information about meetings, current contents of journals, spirited discussions of technical points, and other news. See
the NA-NET homepage.
More recently, OPT-Net has been formed. The
news digest is published somewhat irregularly.
is a repository of preprints in the field.
Most researchers in this area are members of the
Mathematical Programming Society
activity group on optimization
as well as one on
control and systems theory.
Most meetings are announced in SIAM News, NA-Digest, and Opt-Net.
Some other useful Websites:
AAAS Professional Ethics Report
Some disasters attributable to bad numerical computing
More numerical disasters and other bugs.
Global Optimization site
IEEE Control Systems Society
Institute for Operations Research and the Management Sciences
Mathematical Programming Society
NEOS optimization server
index of mathematical publications and reviews.
Survival Guide for Scientific Computing