Experimental design and decision-making in marketplace platforms
Abstract/Contents
- Abstract
- Online platforms often rely on experiments to aid decision-making. When considering a new change, they test the intervention on a subset of the users before deciding whether to launch platform-wide. However, in the setting of marketplace platforms, prior work shows that treatment effect estimates can be biased. Users in a market interact with each other, which violates the Stable Unit Treatment Value Assumption (SUTVA), creates biased estimates, and may impact the resulting decisions made from these experiments. We develop models to capture market dynamics and investigate the effect of interference on different designs and estimators. In particular, we are able to highlight and formalize the relationship between the magnitude of the treatment effect bias in commonly run experiments and the level of supply and demand imbalance in the market. Building on these insights, we propose a novel class of experimental designs and estimators using two-sided randomization (TSR), as a method to reduce bias. In addition, we show that the commonly used standard error estimates are also biased in these marketplace settings. We analyze the impact of the statistical biases on the resulting decisions based on the experiment, show that both forms of biases interact to negatively impact decision-making, and propose practical methods to mitigate such biases.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2022; ©2022 |
| Publication date | 2022; 2022 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Li, Hannah Qiuhan |
|---|---|
| Degree supervisor | Johari, Ramesh, 1976- |
| Degree supervisor | Weintraub, Gabriel |
| Thesis advisor | Johari, Ramesh, 1976- |
| Thesis advisor | Weintraub, Gabriel |
| Thesis advisor | Ugander, Johan |
| Degree committee member | Ugander, Johan |
| Associated with | Stanford University, Department of Management Science and Engineering |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Hannah Li. |
|---|---|
| Note | Submitted to the Department of Management Science and Engineering. |
| Thesis | Thesis Ph.D. Stanford University 2022. |
| Location | https://purl.stanford.edu/wt771yc2020 |
Access conditions
- Copyright
- © 2022 by Hannah Qiuhan Li
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...