Reproducible network research with high-fidelity emulation
Abstract/Contents
- Abstract
- In an ideal world, all research papers would be runnable: simply click to replicate the results, using the same setup as the authors. In many computational fields, like Machine Learning or Programming Languages, creating a runnable paper means packaging up the code and data in a virtual machine. However, for Network Systems, the path to a realistic, runnable paper is not so clear. This class of experiments requires many servers, network elements, and packets to run in parallel, and their results depend on accurate timing. Current platform options either provide realism but lack flexibility (e.g., shared testbeds like Emulab cannot support arbitrary topologies) or provide flexibility but lack realism (e.g., discrete-event simulators like ns-2 model end-host code). This dissertation presents a new approach to enable realistic yet reproducible network experiments: high-fidelity emulation. High-fidelity emulation couples a resource- isolating emulator with a monitor to verify properties of the emulation run. Every (wired) network comprises the same basic components, like links, switches, and virtual hosts, and these components behave in highly predictable ways, since they are implemented in hardware. A correct emulation run will maintain the behavior of these components. For example, a wired link should have a constant delay, while a queue should have a fixed capacity. We call these properties "network invariants", and they are universal: they apply regardless of the experiment being run, the system upon which that experiment is run, and even the emulation code. By logging and processing network events, the monitor can quantify the error in an emulation run. Unlike a simulator, the code is fully real: it is the same code that would run on multiple systems, and it captures implementation quirks such as OS interactions, lock conflicts, and resource limits that simulation models inherently abstract away. The second contribution of this dissertation is Mininet-HiFi, an open-source tool for creating reproducible network system experiments. Mininet-HiFi runs an environment of virtual hosts, switches, and links on a modern multi-core server, using real application and kernel code with software-emulated network elements. The approach builds upon recent advancements in lightweight OS-level virtualization to combine the convenience, flexibility, and low cost of discrete-event simulation with the realism of testbeds. To produce evidence that an experiment ran accurately, it logs system events and extracts indicators of fidelity from these logs. In addition to running experiments, the tool has proven useful for interactively developing, testing, sharing, and demonstrating network systems. The third contribution of this dissertation is the collected outcomes of putting Mininet-HiFi to the test, by using it to reproduce key results from published network experiments such as DCTCP, Hedera, and router buffer sizing. Many lessons were learned during Stanford CS244 in Spring 2012, where 37 students attempted to replicate 18 different published results of their own choosing, atop EC2 virtual machines in the cloud. Their experiences suggest that Mininet HiFi makes research results easier to understand, easier to reproduce, and most importantly, easier to build upon. As a community we seek high-quality results, but our results are rarely reproduced. It is our hope that Mininet-HiFi will spur such a change, by providing networking researchers a way to transform their research papers from "read-only" to "read, write, and execute".
Description
| Type of resource | text |
|---|---|
| Form | electronic; electronic resource; remote |
| Extent | 1 online resource. |
| Publication date | 2013 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Associated with | Heller, Brandon |
|---|---|
| Associated with | Stanford University, Department of Computer Science. |
| Primary advisor | McKeown, Nick |
| Thesis advisor | McKeown, Nick |
| Thesis advisor | Kozyrakis, Christoforos, 1974- |
| Thesis advisor | Parulkar, Gurudatta M |
| Advisor | Kozyrakis, Christoforos, 1974- |
| Advisor | Parulkar, Gurudatta M |
Subjects
| Genre | Theses |
|---|
Bibliographic information
| Statement of responsibility | Brandon Heller. |
|---|---|
| Note | Submitted to the Department of Computer Science. |
| Thesis | Ph.D. Stanford University 2013 |
| Location | electronic resource |
Access conditions
- Copyright
- © 2013 by Brandon David Heller
Also listed in
Loading usage metrics...