Multiple testing and minimax estimation in sparse linear regression
Abstract/Contents
- Abstract
- In many real-world statistical problems, we observe a response variable of interest together with a large number of potentially explanatory variables of which a majority may be irrelevant. For this type of problem, controlling the false discovery rate (FDR) guarantees that most of the selected variables, often termed discoveries in a scientific context, are truly explanatory and thus replicable. Inspired by ideas from the Benjamini-Hochberg procedure (BHq), this thesis proposes a new method named SLOPE to control the FDR in sparse high-dimensional linear regression. SLOPE is a computationally efficient procedure that works by regularizing the fitted coefficients according to their ranks: the higher the rank, the larger the penalty. This adaptive regularization is analogous to the BHq, which compares more significant p-values with more stringent thresholds. Under orthogonal designs, SLOPE with the BHq critical values is proven to control FDR at any given level. Moreover, we demonstrate empirically that this method also appears to have appreciable inferential properties under more general design matrices while offering substantial power. The thesis proceeds to explore the estimation properties of SLOPE. Although SLOPE was developed from a multiple testing viewpoint, we show the surprising result that it achieves optimal squared errors under Gaussian random designs. This optimality holds under a weak assumption on the l0-sparsity level of the underlying signals, and is sharp in the sense that this is the best possible error any estimator can achieve. An appealing feature is that SLOPE does not require any knowledge of the degree of sparsity, and yet automatically adapts to yield optimal total squared errors over a wide range of l0-sparsity classes. Finally, we conclude this thesis by focusing on Nesterov's accelerated scheme, which is integral to a fast algorithmic implementation of SLOPE. Specifically, we prove that, as the step size vanishes, this scheme converges in a rigorous sense to a second-order ordinary differential equation (ODE). This continuous time ODE allows for a better understanding of Nesterov's scheme, and thus it can serve as a tool for analyzing and generalizing this scheme. A fruitful application of this tool yields a family of schemes with similar convergence rates. The ODE interpretation also suggests restarting Nesterov's scheme leading to a new algorithm, which is proven to converge at a linear rate whenever the objective is strongly convex.
Description
| Type of resource | text |
|---|---|
| Form | electronic; electronic resource; remote |
| Extent | 1 online resource. |
| Publication date | 2016 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Associated with | Su, Weijie |
|---|---|
| Associated with | Stanford University, Department of Statistics. |
| Primary advisor | Candès, Emmanuel J. (Emmanuel Jean) |
| Thesis advisor | Candès, Emmanuel J. (Emmanuel Jean) |
| Thesis advisor | Johnstone, Iain |
| Thesis advisor | Siegmund, David, 1941- |
| Advisor | Johnstone, Iain |
| Advisor | Siegmund, David, 1941- |
Subjects
| Genre | Theses |
|---|
Bibliographic information
| Statement of responsibility | Weijie Su. |
|---|---|
| Note | Submitted to the Department of Statistics. |
| Thesis | Thesis (Ph.D.)--Stanford University, 2016. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2016 by Weijie Su
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...