Laplacian regularized stratified model fitting
Abstract/Contents
- Abstract
- A large number of practical problems in optimization involve the fitting of models with categorical features. These problems arise in a variety of fields, such as machine learning, statistics, finance, medicine, and data analysis. A common and simple paradigm parameterizes the model by a parameter vector, and uses convex optimization to minimize an empirical loss on a training data set plus a regularization term that (hopefully) skews the model parameter toward one for which the model generalizes to out of sample data. In this dissertation, we discuss a framework of model fitting methods that depend on a number of categorical features. We build a model of some data that depends in an arbitrary way on these categorical features, by building a separate model for the data that take on each of the possible values of these categorical features. We call these models stratified models. Furthermore, we propose augmenting a basic stratified model fitting problem with an additional regularization term, Laplacian regularization, which encourages the parameters found for each value to be close to their neighbors on some specified weighted graph on the categorical values. We begin this dissertation with an early work on solving Laplacian regularized problems. We then define these models as Laplacian regularized stratified models, and propose an efficient distributed solution method based on the alternating direction method of multipliers (ADMM), which allows us to solve the Laplacian regularized stratified model fitting problem in a distributed fashion and at large scale, in spite of the model parameters being coupled due to the Laplacian regularization term. We then provide an extension of Laplacian regularized stratified models, where we show that if the model parameters vary smoothly over the graph Laplacian, then it can be well approximated by a linear combination of a small number of the Laplacian eigenvectors, which can drastically reduce the number of variables in the problem. Finally, we provide two examples: one example of fitting Laplacian regularized Gaussian models, and another, much larger example which brings the ideas of this paper together by fitting Laplacian regularized stratified risk and return models to used in a realistic quantitative finance setting.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2021; ©2021 |
| Publication date | 2021; 2021 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Tuck, Jonathan Eric |
|---|---|
| Degree supervisor | Boyd, Stephen P |
| Thesis advisor | Boyd, Stephen P |
| Thesis advisor | Osgood, Brad |
| Thesis advisor | Srivastava, Ashok |
| Degree committee member | Osgood, Brad |
| Degree committee member | Srivastava, Ashok |
| Associated with | Stanford University, Department of Electrical Engineering |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Jonathan Eric Tuck. |
|---|---|
| Note | Submitted to the Department of Electrical Engineering. |
| Thesis | Thesis Ph.D. Stanford University 2021. |
| Location | https://purl.stanford.edu/zm017yr2171 |
Access conditions
- Copyright
- © 2021 by Jonathan Eric Tuck
Also listed in
Loading usage metrics...