Daniela Witten
University of Washington

The many flavors of penalized linear discriminant analysis

Linear discriminant analysis (LDA) is a well-known statistical method for classification in the traditional setting where the number of observations exceeds the number of features. The LDA classification rule results from three distinct arguments: (1) a geometric argument (Fisher's discriminant problem), (2) direct application of Bayes' rule to a simple normal model for the data (normal model), and (3) a direct extension of linear regression to the classification problem (optimal scoring). In the high-dimensional setting when the number of variables exceeds the number of observations, LDA cannot be directly applied, and so a number of authors have sought to extend LDA to this setting using penalization approaches taken from the regression literature. Interestingly, though Fisher's discriminant problem, the normal model, and optimal scoring yield the same classification rule in the absence of penalties, they yield different results when penalized. I will present various approaches, new and old, for regularizing these three viewpoints of LDA, and will discuss their similarities and differences.