Nonlinear analysis of biological models and stochastic analysis of machine learning methods
Abstract/Contents
- Abstract
- This dissertation consists of two independent parts: nonlinear analysis of biological models (Part I), and stochastic analysis of machine learning methods (Part II). In the first part, we investigate two nonlinear partial differential equations arising in biology. The first model we consider is the Euler alignment system, which is a hydrodynamics limit of the Cucker-Smale model describing the collective dynamics of a flock of $N$ individuals (birds, fish, etc.) that tend to align their velocities locally. We prove that, as long as the weakly singular interaction kernel is not integrable, the solutions of the Euler alignment system stay globally regular. This result extends the previous regularity results to the critical case. The second model we consider is the Burgers-FKPP equation, which is a reaction-diffusion equation equipped with an advection term of the Burgers type. The Burgers-FKPP equation appears in many applications from chemical physics to population genetics, but the large time behavior of its solutions is rarely studied. An interesting feature of Burgers-FKPP is that when the coefficient of the Burgers nonlinearity increases, the propagating solutions have a phase transition from pulled fronts to pushed fronts. In this work, we show the convergence of a solution to a single traveling wave in the Burgers-FKPP equation, as well as some side discoveries including front asymptotics in higher orders. In the second part, we study the several machine learning models from the stochastic analysis viewpoint. Taking the continuous-time limit and using approximate stochastic differential equations (SDE) to analyze stochastic gradients algorithms has become popular in recent years, since it provides many new insights and compact proofs using developed toolkits. We exhibit the power of stochastic analysis in machine learning from two independent projects. In the first project, we consider the asynchronous stochastic gradient descent (ASGD) algorithm that updates iterates with a delay read, which plays an important role in large scale parallel computing. We derive corresponding SDEs to characterize the dynamics of the ASGD algorithm. Based on that, we can further explore algorithmic properties by considering the temperature factors in Langevin type equations, as well as identifying optimal hyper-parameters by using optimal control theory. In the second project, we consider data, model, and stochastic optimization algorithms as an integrated system. On the one side, we focus on comparing resampling and reweighting for correcting sampling biases. On the other side, we propose a combined resampling and reweighting strategy to handle the data feature disparities. Both problems arise as the models are non-convex, and by SDE approaches we explain how stochastic gradient algorithms select the minimum in different regions.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2021; ©2021 |
| Publication date | 2021; 2021 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | An, Jing |
|---|---|
| Degree supervisor | Ryzhik, Leonid |
| Degree supervisor | Ying, Lexing |
| Thesis advisor | Ryzhik, Leonid |
| Thesis advisor | Ying, Lexing |
| Thesis advisor | Papanicolaou, George |
| Degree committee member | Papanicolaou, George |
| Associated with | Stanford University, Institute for Computational and Mathematical Engineering |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Jing An. |
|---|---|
| Note | Submitted to the Institute for Computational and Mathematical Engineering. |
| Thesis | Thesis Ph.D. Stanford University 2021. |
| Location | https://purl.stanford.edu/jk773cp3305 |
Access conditions
- Copyright
- © 2021 by Jing An
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...