Penalized Least Squares and Frequentist and  Bayesian Mixed-Effects Models

Yolanda Muñoz Maldonado

Division of Biostatistics, School of Public Health
University of Texas Health Science Center at Houston



Abstract

        Penalized least squares and mixed effects models using frequentist and Bayesian approaches are commonly used tools in a wide range of settings like, for example, clustered data, smoothing problems, ridge regression, and functional data analysis, and they have a broad range of applications in areas like Biology, Environmental Sciences, Survey Sampling, etc. However, this broad range of applications has also made that some interesting relationships between these three different methodologies rest unnoticed. In this talk, I will use as platform, the well known connections between smoothing splines estimators, a particular Gaussian mixed effects model, and a Bayesian Gaussian stochastic process, to build a general framework that encompasses Penalized Least Squares Techniques and Frequentist and Bayesian Mixed Effects Models. This proposed framework has the implication that, in many cases of practical interest, an efficient, O(n), Kalman Filter algorithm can be used to obtain the desired predictors and corresponding Bayesian confidence intervals. This algorithm also permits the evaluation of the exact likelihood function with the same level of computational efficiency. To illustrate the range of applicability of our main results we use examples from three different settings: varying coefficient models, ridge regression and randomized block designs.


Keywords: Smoothing splines, ridge regression, Bayesian prediction, varying coefficients, confidence bands, Kalman filtering.