A Unified View of
Regression, Shrinkage, Empirical Bayes, Hierarchical Bayes, and Random Effects

Lawrence D. Brown
Wharton School, University of Pennsylvania

Abstract

    A wide range of statistical problems involve estimation of means or conditional means of multidimensional normal distributions. There are many commonly employed classes of statistical models and related approaches to such problems. The present talk will survey the interrelations among some of these approaches, and propose some issues for further investigation.

    While our survey could begin with almost any of the several related approaches, we will start by reviewing the background of shrinkage estimation.    Stein (1956) surprised the statistical world with his discovery that the ordinary least squares estimator of a multivariate normal mean is not admissible in the usual setting. James and Stein (1961) then produced their classic estimator which often provides significant improvement over the ordinary estimator. 'Shrinkage' is a core feature of the estimator.

    An empirical Bayes interpretation of shrinkage was first proposed by Stein (1962) and Lindley (1962). The interpretation has been effectively exploited by Efron and Morris (1972) and others. The empirical Bayes interpretation and its hierarchical fully Bayes first cousin, as first developed for this problem by Strawderman (1972), provide an important link to the manifestations of shrinkage in the various contemporary methodologies. Lindley and Smith (1972) presented a thoroughly Bayesian viewpoint for these interpretations but their mode of presentation can also justify a random-effects view of the situation. This perspective in turn allows for a shrinkage interpretation of the familiar prediction procedure in ordinary linear regression.

    Some analytic theory and data analyses will illustrate the main points.