Fine-grain in-memory deduplication for large-scale workloads

Placeholder Show Content


Memory is a large component of computer system cost and current trends indicate this cost is increasing as a fraction of the total. Emerging applications such as in-memory databases, virtual machines, and big-data key-value stores demand more memory relative to compute. Some of these high-memory applications incidentally store many duplicate values in memory: in some cases, duplicates account for over 75% of the total. Recent work on the HICAMP architecture provided a sophisticated hardware mechanism for memory deduplication to implement memory versioning, but without support for current software stacks. This thesis extends work on HICAMP by evaluating a deduplicated memory that is compatible with existing hardware and software. Memory content from actual workloads indicates that deduplicated memory effectively doubles capacity. After understanding the baseline cost-benefit tradeoff in terms of capacity, performance, and energy, this work proposes novel optimizations for machines with deduplicated memory. These optimizations reduce memory traffic and improve performance relative to both the baseline deduplicated memory and, in many cases, relative to the original machine. Energy consumption is reduced because memory devices are reduced with no penalty to performance. Further, deduplication reduces data transfer and improves performance for certain scientific applications. This thesis argues that in-memory deduplication is warranted by its own benefits, which are likely to grow in the future, and that it enables low-cost memory snapshots, as in the HICAMP architecture.


Type of resource text
Form electronic; electronic resource; remote
Extent 1 online resource.
Publication date 2013
Issuance monographic
Language English


Associated with Stevenson, John Peter
Associated with Stanford University, Department of Electrical Engineering.
Primary advisor Horowitz, Mark (Mark Alan)
Thesis advisor Horowitz, Mark (Mark Alan)
Thesis advisor Cheriton, David R
Thesis advisor Hanrahan, P. M. (Patrick Matthew)
Advisor Cheriton, David R
Advisor Hanrahan, P. M. (Patrick Matthew)


Genre Theses

Bibliographic information

Statement of responsibility John Peter Stevenson.
Note Submitted to the Department of Electrical Engineering.
Thesis Thesis (Ph.D.)--Stanford University, 2013.
Location electronic resource

Access conditions

© 2013 by John Peter Stevenson
This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).

Also listed in

Loading usage metrics...