Some Useful References for Mathematical Statistics

Originally based on Prof. James Gentle’s webpage
http://mason.gmu.edu/~jgentle/csi9723/

Texts on general mathematical statistics at the level of this course, more-or-less

Lehmann, E. L., and George Casella (1998), Theory of Point Estimation, second edition, Springer.
Lehmann, E. L., and Joseph P. Romano (2005), Testing Statistical Hypotheses, third edition, Springer.
There is a useful companion book called Testing Statistical Hypotheses: Worked Solutions by some people at CWI in Amsterdam that has solutions to the exercises in the first edition. (Most of these are also in the third edition.)
Stuart, A. and Ord, J. K., Kendall's Advanced Theory of Statistics, 6th ed., John Wiley & Sons, Inc., New York, 1994.
Schervish, Mark J. (1995), Theory of Statistics, Springer.
This rigorous and quite comprehensive text has a Bayesian orientation.
Shao, Jun (2003), Mathematical Statistics, second edition, Springer.
Comprehensive and rigorous; better than the first edition.
Shao, Jun (2005), Mathematical Statistics: Exercises and Solutions, Springer.
Solutions (or partial solutions) to some exercises in Shao (2003), plus some additional exercises and solutions.

Texts in probability and measure theory and linear spaces roughly at the level of this course

B. Fristedt and L. Gray (1997), A Modern Approach to Probability Theory, Birkhauser
Ash, Robert B., and Catherine A. Doleans-Dade (1999), Probability & Measure Theory, second edition, Academic Press.
Accessible and wide-ranging text; also covers stochastic calculus.
Athreya, Krishna B., and Soumen N. Lahiri (2006), Measure Theory and Probability Theory, Springer.
A very solid book, but beware of typos in the first printing.
Billingsley, Patrick (1995), Probability and Measure, third edition, John Wiley & Sons.
This is one of the best books on probability and measure theory for probability, in terms of coverage and rigor. No explicit coverage of linear spaces.
Breiman, Leo (1968), Probability, Addison-Wesley.
This is a classic book on measure-theoretic-based probability theory. No explicit coverage of measure theory or linear spaces. The book (with corrections) is available in the SIAM Classics in Allied Mathematics series (1992).
Dudley, R. M. (2002), Real Analysis and Probability, second edition, Cambridge University Press.
Accessible and comprehensive.

Texts that provide good background for this course

Berger, James O. (1985), Statistical Decision Theory and Bayesian Analysis, second edition, Springer.
Bickel, Peter, and Kjell A. Doksum (2001), Mathematical Statistics: Basic Ideas and Selected Topics, Volume I, second edition, Prentice Hall.
This book covers material from Chapters 1-6 and Chapter 10 of the first edition, but with more emphasis on nonparametric and semiparametric models and on function-valued parameters. It also includes more Bayesian perspectives. The second volume will not appear for a couple of years. In the meantime, the first edition remains a very useful text.
Casella, George, and Roger L. Berger (2001), Statistical Inference, second edition, Duxbury Press.
Robert, Christian P. (1995), The Bayesian Choice, Springer.
This is a carefully-written book with a somewhat odd title. This book is at a slightly higher level than the others in this grouping.
Hogg, Robert V.; and Allen T. Craig (1994), Introduction to Mathematical Statistics (5th Edition), Prentice-Hall.
This old standard is at a slightly lower level than the others in this grouping but has good examples and another take on the concepts..

Interesting monographs

Barndorff-Nielson, O. E., and D. R. Cox (1994), Inference and Asymptotics, Chapman and Hall.
Brown, Lawrence D. (1986), Fundamentals of Statistical Exponential Families with Applications in Statistical Decision Theory, Institute of Mathematical Statistics.
Lehmann, E. L. (1999), Elements of Large-Sample Theory, Springer.
Serfling, Robert J. (1980), Approximation Theorems of Mathematical Statistics, John Wiley & Sons.

Interesting compendia of counterexamples

An interesting kind of book is one with the word ``counterexamples'' in its title. Counterexamples provide useful limits on mathematical facts. As Gelbaum and Olmsted observed in the preface to their 1964 book, which was the first in this genre, ``At the risk of oversimplification, we might say that (aside from definitions, statements, and hard work), mathematics consists of two classes --- proof and counterexamples, and that mathematical discovery is directed toward two major goals --- the formulation of proofs and the construction of counterexamples.''

Gelbaum, Bernard R., and John M. H. Olmsted (1990), Theorems and Counterexamples in Mathematics, Springer.
Gelbaum, Bernard R., and John M. H. Olmsted (2003), Counterexamples in Analysis, (originally published in 1964; corrected reprint of the second printing published by Holden-Day, Inc., San Francisco, 1965), Dover Publications, Inc., Mineola, New York.
Romano, Joseph P., and Andrew F. Siegel (1986), Counterexamples in Probability and Statistics, Chapman and Hall.
In the field of mathematical statistics, this is the most useful of the ``counterexamples'' books. It has been rumored that course instructors get problems from this book. I can neither confirm nor deny this rumor. I can report that I have the book.
Stoyanov, Jordan M. (1987), Counterexamples in Probability, John Wiley & Sons.
Wise, Gary L., and Eric B. Hall (1993), Counterexamples in Probability and Real Aanalysis, The Clarendon Press, Oxford University Press.

Interesting set of essays

Various authors (2002), Chapter 4, Theory and Methods of Statistics, in Statistics in the 21st Century, edited by Adrian E. Raftery, Martin A. Tanner, and Martin T. Wells, Chapman and Hall.
The "golden age" of mathematical statistics was the middle third of the twentieth century, and the content of the books in the first grouping above cover the developments of this period very well. The set of essays in Chapter 4 reviews some of the more recent and ongoing work.

Good compendium on standard probability distributions

Evans, Merran; Nicholas Hastings; and Brian Peacock (2000), Statistical Distributions, third edition, John Wiley & Sons.

There is also a multi-volume/multi-edition set of books by Norman Johnson and Sam Kotz and co-authors, published by Wiley. The books have titles like "Discrete Multivariate Distributions". (The series began with four volumes in the 70's by Johnson and Kotz. I have those, but over the years they have been revised, co-authors have been added, and volumes have been subdivided. I am not sure what comprises the current set, but any or all of the books are useful.)