Deep learning for anomaly detection and representation learning
Abstract/Contents
- Abstract
- Anomaly Detection (AD) refers to the process of identifying abnormal observations that deviate from what is defined as normal. With applications in many real-world domains, anomaly detection has become an important research field in machine leaning and artificial intelligence. However, detecting anomalies in high-dimensional space is challenging. In some high-dimensional cases, previous anomaly detection algorithms fail to correctly model the normal data distribution. Also the understanding on the detection mechanism of AD models remained limited. How are anomalies detected by a AD algorithm? Why is a specific instance detected as anomalous? To address these challenges and questions, first we propose the Regularized Cycle-consistent GAN (RCGAN) that introduces a penalty distribution in the modeling of normal data distribution. We theoretically show that the penalty distribution regularizes the discriminator and generator towards the normal data manifold. This results in higher detection accuracy and more precise normal data modeling. Second, we introduce the memory augmented GAN (MEMGAN) for anomaly detection. By leveraging adversarial learning and enforcing cycle-consistency, the trained memory units enclose the encoded normal data. We also theoretically and numerically show that the encoded normal data reside in the convex hull of the memory units, while the abnormal data are expected to be located outside. Third, we explore anomaly detection with domain adaptation where the normal data distribution is non-static. We propose to extract the common features of source and target domain data and train an anomaly detector using the extracted features. The second part of the thesis is on language representation learning. Language representation learning refers to the task of mapping language token to a numerical vector. Previous sentence representation models largely rely on labeled data and supervised training. However, these data are expensive to obtain, making it hard to to cover more domains and languages. To solve this challenge, we propose Conditional Masked Language Modeling (CMLM) that integrates sentence representation learning into unsupervised Masked Language Modeling (MLM) training. We also propose GEM, a non-parameterized method that exploits the geometric structure of a sentence subspace to build the sentence representation. Finally, we explore the problem of out-of-vocabulary (OOV) embeddings imputation by using graph convolutional networks and grounded language information.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2021; ©2021 |
| Publication date | 2021; 2021 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Yang, Ziyi |
|---|---|
| Degree supervisor | Darve, Eric |
| Thesis advisor | Darve, Eric |
| Thesis advisor | Kochenderfer, Mykel J, 1980- |
| Thesis advisor | Ratner, Daniel |
| Degree committee member | Kochenderfer, Mykel J, 1980- |
| Degree committee member | Ratner, Daniel |
| Associated with | Stanford University, Department of Mechanical Engineering |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Ziyi Yang. |
|---|---|
| Note | Submitted to the Department of Mechanical Engineering. |
| Thesis | Thesis Ph.D. Stanford University 2021. |
| Location | https://purl.stanford.edu/fr740yc5473 |
Access conditions
- Copyright
- © 2021 by Ziyi Yang
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...