Asymptotic Behavior of $M$-Estimators of $p$ Regression Parameters when $p^2/n$ is Large. I. Consistency

Abstract

Consider the general linear model $Y = x\\beta + R$ with $Y$ and $R n$-dimensional, $\\beta p$-dimensional, and $X$ an $n \\times p$ matrix with rows $x'_i$. Let $\\psi$ be given and let $\\hat\\beta$ be an $M$-estimator of $\\beta$ satisfying $0 = \\sum x_i\\psi(Y_i - x'_i\\hat\\beta)$. Previous authors have considered consistency and asymptotic normality of $\\hat\\beta$ when $p$ is permitted to grow, but they have required at least $p^2/n \\rightarrow 0$. Here the following result is presented: in typical regression cases, under reasonable conditions if $p(\\log p)/n \\rightarrow 0$ then $\\|\\hat{\\beta} - \\beta\\|^2 = \\mathscr{O}_p(p/n)$. A subsequent paper will show that $\\hat{\\beta}$ has a normal approximation in $R^p$ if $(p \\log p)^{3/2}/n \\rightarrow 0$ and that $\\max_i|x'_i(\\hat{\\beta} - \\beta)| \\rightarrow_p 0$ (which would not follow from norm consistency if $p^2/n \\rightarrow \\infty$). In ANOVA cases, $\\hat{\\beta}$ is not norm consistent, but it is shown here that $\\max|x'_i(\\hat{\\beta} - \\beta)| \\rightarrow_p 0$ if $p \\log p/n \\rightarrow 0$. A normality result for arbitrary linear combinations $a'(\\hat{\\beta} - \\beta)$ is also presented in this case.