How to design and analyze online A/B tests within decentralized organizations
Abstract/Contents
- Abstract
- From e-commerce to digital media to social networks, essentially any company that does business online is using A/B tests -- randomized experiments on its customers -- in order to optimize its service. A standard approach to designing and analyzing these experiments, based on classical statistical theory, is ubiquitous in industrial practice. Namely, a sample size for each test should be set in advance, and the data collected should be analyzed in isolation through p-values and confidence intervals that are computed based on a 2-sample t-test of means. This thesis investigates four ways that this default approach proves insufficient, due to the decentralized manner in which A/B tests at these companies are run. For each of the four, we empower experimenters to continue their current behavior, while we offer novel methodology that is simple to implement and ensures inferential reliability. Some of our methods are now in use in industry. We supplement our theoretical results with empirical data from these industrial deployments.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2019; ©2019 |
| Publication date | 2019; 2019 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Walsh, David Jonathan Max |
|---|---|
| Degree supervisor | Johari, Ramesh, 1976- |
| Thesis advisor | Johari, Ramesh, 1976- |
| Thesis advisor | Efron, Bradley |
| Thesis advisor | Owen, Art B |
| Degree committee member | Efron, Bradley |
| Degree committee member | Owen, Art B |
| Associated with | Stanford University, Department of Statistics. |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | David Walsh. |
|---|---|
| Note | Submitted to the Department of Statistics. |
| Thesis | Thesis Ph.D. Stanford University 2019. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2019 by David Jonathan Max Walsh
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...