Optimality of Stepwise Tests and a General
Resampling Construction

Joseph P. Romano
Stanford University

Abstract

 
Consider the multiple testing problem of testing $k$ null
hypotheses, where the unknown family of distributions is
assumed to satisfy a certain monotonicity assumption.
Attention is restricted to procedures that control the
familywise error rate (FWE) in the strong sense and which
satisfy a monotonicity condition. Under these assumptions,
we prove certain maximin optimality results for the
well-known step-down and step-up procedures.

In the second part, we consider the general problem of
contructing methods that control the FWE. In order to
improve upon the Bonferroni method or Holm's (1979) step-down
method, Westfall and Young (1993) make effective use of
resampling to construct step-down methods that implicitly
estimate the dependence structure of the test statistics.
However, their methods depend on an assumption called subset
pivotality.

We will show how to construct methods that control the FWE,
both in finite and large samples. A key ingredient is
monotonicity of critical values which allows one to effectively
reduce the multiple testing problem of controlling the FWE to
the single testing problem of controlling the probability of
a Type 1 error.

(The first part is joint work with  Erich Lehmann and Juliet
Shaffer, and the second part is joint work with Michael Wolf.)