A neural network with feature sparsity
Abstract/Contents
- Abstract
- First, we propose a neural network model with a separate linear (residual) term, that explicitly bounds the input layer weights for a feature by the linear weight for that feature. The model can be seen as a modification of so-called residual neural networks to produce a path of models that are feature- sparse, that is, use only a subset of the features. This is analogous to the solution path from the usual Lasso (L1-regularized) linear regression. We call the proposed procedure LassoNet and develop a projected proximal gradient algorithm for its optimization. This approach can sometimes give as low or lower test error than a standard neural network, and its feature selection provides more interpretable solutions. This thesis illustrates the method using both simulated and real data examples, and shows that it is often able to achieve competitive performance with a much smaller number of input features. We also discuss extensions of this work beyond supervised learning, which includes unsupervised learning, matrix completion, and sparsity in learned features. Second, we consider the problem of local feature attribution and selection for arbitrary black-box models. We introduce a geometric method which we call RbX, for Region-Based Explanations. This method relies on approximating the prediction model's level sets by convex polytopes, thus helping to simplify and interpret the model. We demonstrate the effectiveness of the method on a variety of synthetic and real data sets.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2021; ©2021 |
| Publication date | 2021; 2021 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Lemhadri, Ismael |
|---|---|
| Degree supervisor | Tibshirani, Robert |
| Thesis advisor | Tibshirani, Robert |
| Thesis advisor | Duchi, John |
| Thesis advisor | Hastie, Trevor |
| Degree committee member | Duchi, John |
| Degree committee member | Hastie, Trevor |
| Associated with | Stanford University, Department of Statistics |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Ismael Lemhadri. |
|---|---|
| Note | Submitted to the Department of Statistics. |
| Thesis | Thesis Ph.D. Stanford University 2021. |
| Location | https://purl.stanford.edu/yk085xh9902 |
Access conditions
- Copyright
- © 2021 by Ismael Lemhadri
- License
- This work is licensed under a Creative Commons Attribution 3.0 Unported license (CC BY).
Also listed in
Loading usage metrics...