Peter H. Westfall
Texas Tech University

Which Multiple Testing Methods are Optimal?

Decision theory provides answers. * While FDR-controlling methods are optimal for certain kinds of loss functions, the Bonferroni method is optimal for others. * For additive loss, correlation between test statistics does not affect expected loss, but quantiles of loss are affected by correlations. * The choice of additive loss is made for mathematical convenience. The practical validity of additive loss is questionable; reasonable non-additive loss functions provide different optimal tests. * Point-null multiple hypothesis testing methods have increased loss as n increases. Methods based on empirical nulls do not.