Introduction

In the analysis of high-dimensional genomic data, the absolute correlation among predictors routinely exceeds 0.9, or even 0.95. In the presence of such high collinearity, special techniques are needed to achieve reliable estimation and variable selection in the linear model because common techniques, such as the Lasso (Tibshirani, 1996), fail in this setting. Although some authors have offered improvements over the Lasso when certain features of the design matrix, such as grouping (Yuan and Lin, 2006) or ordering (Tibshirani et al., 2005), can be exploited, another challenging prediction problem occurs when no additional design structure is known or assumed. Several authors have tackled regularization amidst highly correlated predictors, with the “elastic net” (Zou and Hastie, 2005) being the most popular and most widely propagated among them. In this chapter, we examine deficiencies of the elastic net and argue in favor of a little-known competitor, the “Berhu” penalized least squares estimator (Owen, 2006), for high-dimensional regression analyses of genomic data.

Regularization describes a popular class of computational and statistical methods for estimation, prediction, or regression that can be applied to virtually any statistic. In regression, these methods may be summarized as biased regression tools that sacrifice bias to minimize prediction or classification error. These methods transcend Bayesian and frequentist paradigms and extend naturally to high-dimensional regression. Among all regularized regression estimators, ℓ1- and ℓ2-regularization are the most popular, but only the former method results in a sparse solution. Despite the popularity of ℓ1-regularization, it suffers drawbacks.