Compiling deep learning kernels to locality-aware dataflow
Abstract/Contents
- Abstract
- Emerging deep learning applications require unprecedented computation and memory capacity. To accelerate these applications, novel processing systems such as dataflow accelerators strive to exploit multiple dimensions of parallelism within deep learning models, e.g., tensor and pipeline parallelism. Although these systems provide ultra-high performance when fully utilized, compiling deep learning applications to harness their computation capability remains a challenging problem. With recent advances in domain-specific programming language, accelerator design, and machine learning, we now have the potential to better serve the needs of training and evaluating large deep learning applications on dataflow accelerators through algorithm, software, and hardware co-design. In this dissertation, I present the design and development of efficient deep learning optimizations and programming frameworks. I present two frameworks: SpatialRNN for accelerating recurrent neural network language models on spatial accelerators and Sigma for expressing and accelerating high-data-reuse deep learning kernels using reconfigurable dataflow accelerators. Our end-to-end evaluation using Sigma demonstrates a 5.4x speedup on kernels encompassing financial applications, traditional machine learning, language modeling and computer vision tasks over an Nvidia V100 GPU accelerator.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2023; ©2023 |
| Publication date | 2023; 2023 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Zhao, Tian |
|---|---|
| Degree supervisor | Olukotun, Oyekunle Ayinde |
| Thesis advisor | Olukotun, Oyekunle Ayinde |
| Thesis advisor | Raina, Priyanka, (Assistant Professor of Electrical Engineering) |
| Thesis advisor | Ré, Christopher |
| Degree committee member | Raina, Priyanka, (Assistant Professor of Electrical Engineering) |
| Degree committee member | Ré, Christopher |
| Associated with | Stanford University, School of Engineering |
| Associated with | Stanford University, Department of Electrical Engineering |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Tian Zhao. |
|---|---|
| Note | Submitted to the Department of Electrical Engineering. |
| Thesis | Thesis Ph.D. Stanford University 2023. |
| Location | https://purl.stanford.edu/hd752ps4385 |
Access conditions
- Copyright
- © 2023 by Tian Zhao
- License
- This work is licensed under a Creative Commons Attribution Non Commercial Share Alike 3.0 Unported license (CC BY-NC-SA).
Also listed in
Loading usage metrics...