High performance cloud computing
Abstract/Contents
- Abstract
- Running fast computations at a large scale increasingly stresses task scheduling performance. As CPU performance becomes the performance bottleneck, it is common to optimize CPU performance, further parallelize computations by using more nodes, and split computations into more finely-grained tasks. As a result, a cloud framework running an application on thousands of cores must schedule hundreds of thousands of tasks per second. A framework must be also able to dynamically reschedule tasks in order to achieve high CPU utilization in the presence of an unbalanced load, stragglers, and node failures. Today, systems must choose either high task throughput or dynamic, centralized scheduling: no existing system can provide both. This dissertation proposes a data-centric asynchronous control plane, which schedules hundreds of thousands of tasks per second while retaining a central interface to dynamically reschedule tasks. In this design, a central controller decides how to redistribute work, but workers locally decide between themselves when to redistribute. The controller avoids issuing per-task scheduling decisions by using data placement as the control signal for load distribution. So this design provides a declarative, central, and asynchronous load control mechanism without limiting task throughput. This control plane depends on a new program abstraction, called task recipes, that enables workers to locally schedule tasks and asynchronously migrate data and computation. This dissertation implements the design in a from-scratch in-memory cloud framework called Canary and evaluates its performance on data analytics and machine learning benchmarks. Furthermore, this dissertation examines how applications benefit from the design by using fluid simulation as a case study, and it proposes a new load-balancing algorithm in this context.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2018; ©2018 |
| Publication date | 2018; 2018 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Qu, Hang |
|---|---|
| Degree supervisor | Levis, Philip |
| Thesis advisor | Levis, Philip |
| Thesis advisor | Aiken, Alexander |
| Thesis advisor | Rosenblum, Mendel |
| Degree committee member | Aiken, Alexander |
| Degree committee member | Rosenblum, Mendel |
| Associated with | Stanford University, Department of Electrical Engineering. |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Hang Qu. |
|---|---|
| Note | Submitted to the Department of Electrical Engineering. |
| Thesis | Thesis Ph.D. Stanford University 2018. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2018 by Hang Qu
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...