Useful References for Mathematical Statistics
Originally based on Prof. James Gentle’s
Texts on general mathematical statistics at the level of this course,
- Lehmann, E. L., and George
Casella (1998), Theory of Point Estimation, second edition,
- Lehmann, E. L., and Joseph P.
Romano (2005), Testing Statistical Hypotheses, third edition,
There is a useful companion book called Testing Statistical Hypotheses:
Worked Solutions by some people at CWI in Amsterdam that has solutions to the
exercises in the first edition. (Most of these are also in the third
- Stuart, A. and Ord, J. K., Kendall's Advanced Theory of Statistics,
6th ed., John Wiley & Sons, Inc., New York, 1994.
- Schervish, Mark J. (1995), Theory
of Statistics, Springer.
This rigorous and quite comprehensive text has a Bayesian orientation.
- Shao, Jun (2003), Mathematical
Statistics, second edition, Springer.
Comprehensive and rigorous; better than the first edition.
- Shao, Jun (2005), Mathematical
Statistics: Exercises and Solutions, Springer.
Solutions (or partial solutions) to some exercises in Shao (2003), plus
some additional exercises and solutions.
Texts in probability and measure theory and linear spaces roughly at the
level of this course
- B. Fristedt and L. Gray
(1997), A Modern Approach to
Probability Theory, Birkhauser
- Ash, Robert B., and Catherine
A. Doleans-Dade (1999), Probability & Measure Theory, second
edition, Academic Press.
Accessible and wide-ranging text; also covers stochastic calculus.
- Athreya, Krishna
B., and Soumen N. Lahiri (2006), Measure Theory and Probability Theory,
A very solid book, but beware of typos in the first printing.
- Billingsley, Patrick (1995), Probability
and Measure, third edition, John Wiley & Sons.
This is one of the best books on probability and measure theory for
probability, in terms of coverage and rigor. No explicit coverage of
- Breiman, Leo (1968), Probability,
This is a classic book on measure-theoretic-based probability theory. No
explicit coverage of measure theory or linear spaces. The book (with
corrections) is available in the SIAM Classics in Allied Mathematics
- Dudley, R. M. (2002), Real
Analysis and Probability, second edition, Cambridge University Press.
Accessible and comprehensive.
Texts that provide good background for this course
- Berger, James O. (1985), Statistical
Decision Theory and Bayesian Analysis, second edition, Springer.
- Bickel, Peter, and Kjell A.
Doksum (2001), Mathematical Statistics: Basic Ideas and Selected
Topics, Volume I, second edition, Prentice Hall.
This book covers material from Chapters 1-6 and Chapter 10 of the first
edition, but with more emphasis on nonparametric and semiparametric models
and on function-valued parameters. It also includes more Bayesian
perspectives. The second volume will not appear for a couple of years. In
the meantime, the first edition remains a very useful text.
- Casella, George, and Roger L.
Berger (2001), Statistical Inference, second edition, Duxbury
- Robert, Christian P. (1995), The
Bayesian Choice, Springer.
This is a carefully-written book with a somewhat odd title. This book is
at a slightly higher level than the others in this grouping.
- Hogg, Robert V.; and Allen T.
Craig (1994), Introduction to Mathematical Statistics (5th
This old standard is at a slightly lower level than the others in this
grouping but has good examples and another take on the concepts..
- Barndorff-Nielson, O. E., and
D. R. Cox (1994), Inference and Asymptotics, Chapman and Hall.
- Brown, Lawrence D. (1986), Fundamentals
of Statistical Exponential Families with Applications in Statistical
Decision Theory, Institute of Mathematical Statistics.
- Lehmann, E. L. (1999), Elements
of Large-Sample Theory, Springer.
- Serfling, Robert J. (1980), Approximation
Theorems of Mathematical Statistics, John Wiley & Sons.
Interesting compendia of counterexamples
An interesting kind of book is one with the word
``counterexamples'' in its title. Counterexamples provide useful limits on
mathematical facts. As Gelbaum and Olmsted observed in the preface to their
1964 book, which was the first in this genre, ``At the risk of oversimplification,
we might say that (aside from definitions, statements, and hard work),
mathematics consists of two classes --- proof and counterexamples, and that
mathematical discovery is directed toward two major goals --- the formulation
of proofs and the construction of counterexamples.''
- Gelbaum, Bernard R., and John
M. H. Olmsted (1990), Theorems and Counterexamples in Mathematics, Springer.
- Gelbaum, Bernard R., and John
M. H. Olmsted (2003), Counterexamples in Analysis, (originally
published in 1964; corrected reprint of the second printing published by
Holden-Day, Inc., San Francisco, 1965), Dover Publications, Inc., Mineola,
- Romano, Joseph P., and Andrew
F. Siegel (1986), Counterexamples in Probability and Statistics, Chapman
In the field of mathematical statistics, this is the most useful of the
``counterexamples'' books. It has been rumored that course instructors get
problems from this book. I can neither confirm nor deny this rumor. I can
report that I have the book.
- Stoyanov, Jordan M. (1987), Counterexamples
in Probability, John Wiley & Sons.
- Wise, Gary L., and Eric B.
Hall (1993), Counterexamples in Probability and Real Aanalysis, The
Clarendon Press, Oxford
Interesting set of essays
- Various authors (2002),
Chapter 4, Theory and Methods of Statistics, in Statistics in the 21st
Century, edited by Adrian E. Raftery, Martin A. Tanner, and Martin T.
Wells, Chapman and Hall.
The "golden age" of mathematical statistics was the middle third
of the twentieth century, and the content of the books in the first
grouping above cover the developments of this period very well. The set of
essays in Chapter 4 reviews some of the more recent and ongoing work.
Good compendium on standard probability distributions
- Evans, Merran; Nicholas
Hastings; and Brian Peacock (2000), Statistical Distributions, third
edition, John Wiley & Sons.
There is also a multi-volume/multi-edition set of books by Norman Johnson
and Sam Kotz and co-authors, published by Wiley. The books have titles like
"Discrete Multivariate Distributions". (The series began with four
volumes in the 70's by Johnson and Kotz. I have those, but over the years they
have been revised, co-authors have been added, and volumes have been
subdivided. I am not sure what comprises the current set, but any or all of the
books are useful.)