Boosting like path algorithms for L1 regularized infinite dimensional convex neural networks
Abstract/Contents
- Abstract
- This work falls at the intersection of three fields or research: Neural Networks, Boosting and Sparse Inverse Problems. Neural Networks, despite being very successful, are poorly understood. Convexification has been an important tool in analyzing a single hidden layered network in recent years. Boosting is one of the most popular tools available for non-linear regression and classification. While boosting is a very general idea, only one variety (greedy gradient boosting with trees) seems to completely dominate the field, leaving scope for expanding the applicability of boosting; and also the algorithms used for fitting it. We employ a model for convex neural network which has been previously studied, but our focus will be on actually fitting it. In doing so, we also see that we can better understand boosting. This leads us to greatly expand both the applicability and algorithms for boosting. Given that our formulation of the convex neural network is a sparse inverse problem, the Julia based framework we develop can be used to solve any sparse inverse problem in the various methods we describe with minimal code development. Finally, as a bonus, we see that the methods we present are not just pedagogical but they do have competitive performance on real datasets. They match the performance of not only Neural Networks of comparable complexity that are hard to train, but also that of state-of-the-art boosting technologies like XGBoost with trees.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2019; ©2019 |
| Publication date | 2019; 2019 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Achanta, Rakesh Kumar |
|---|---|
| Degree supervisor | Hastie, Trevor |
| Thesis advisor | Hastie, Trevor |
| Thesis advisor | Taylor, Jonathan E |
| Thesis advisor | Tibshirani, Robert |
| Degree committee member | Taylor, Jonathan E |
| Degree committee member | Tibshirani, Robert |
| Associated with | Stanford University, Department of Statistics. |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Rakesh Achanta. |
|---|---|
| Note | Submitted to the Department of Statistics. |
| Thesis | Thesis Ph.D. Stanford University 2019. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2019 by Rakesh Kumar Achanta
- License
- This work is licensed under a Creative Commons Attribution 3.0 Unported license (CC BY).
Also listed in
Loading usage metrics...