Essays on ride-sharing and sequential hypothesis testing
Abstract/Contents
- Abstract
- Chapter 1 of this dissertation, co-authored with Dominic Coey and Erjie Ang from Facebook, presents simple and broadly applicable sequential testing procedures for shortening the duration of online experiments. Our procedures are derived by formulating a sequential testing problem as a multiple hypothesis testing problem, and adapting family-wise Type I error controlling methods from the multiple hypothesis testing literature. Using a dataset of 420 advertising effectiveness experiments run at Facebook that lasted an average of 33 days, we show that a sequential test based on the Bonferroni correction would have shortened 45% of experiments and reduced average experiment length by 32%. Back of the envelope calculations show that the average advertiser spending on an experiment would be reduced by 38%. In our data, we find the additional experiment length reduction from adapting theoretically more powerful multiple hypothesis testing correction tools such as Hochberg (1988) and Benjamini-Hochberg (1995) to sequential tests to be very small. We also provide practical advice on how to alleviate inferential biases due to early experiment termination, and how to reduce Type II errors. Chapter 2 investigates experimental conditions where asymptotic exactness indeed brings substantial reductions in experiment length and gains in power over an asymptotically non-exact procedure such as the "Bonferroni Sequential Test" in Ang et al. (2017). I do this by first deriving a family of asymptotically exact sequential tests using the idea of "Type I error spending rate" in Lan et al. (1983). I then prove that under the same experimental conditions and set of sequential rejection or non-rejection decision making schedule, there is no realization of an experiment where i) the Bonferroni Sequential Test rejects, and the asymptotically exact sequential test with an even Type I error spending rate does not reject, ii) the Bonferroni Sequential Test's rejection is earlier than the latter test's rejection. I run a Monte Carlo study to show that this theoretical result has sizable practical implications on power and experiment length reduction especially under relatively small treatment effects, relatively small samples and cases where the experimenter wants to be able to make rejection/continuation decisions with relatively higher frequency. Overall, reconsidering empirical value of the features of a testing procedure, such as asymptotic exactness, within the context of an online experiments' larger data sets would help statisticians re-prioritize their testing procedure development efforts. Chapter 3, co-authored with Larry Levin and Frank Wolak, measures how the growth rate of geo-market level expenses on other cabs change after the launch of ride-sharing across the U.S.. Making these measurements is a pre-requisite for informed policy debates on ride-sharing vis a vis its other cabs. We provide the first set of such measurements by using Visa Card swipe data from January 2011 to June 2016 from 108 ride-sharing geo-markets in the U.S.. Our change in growth rate measurements are not in line with the hypothesis that the launch of ride-sharing is associated with the slow down of other cabs across the entire U.S.: We find evidence for a decrease in the growth of other cab expenses following the inception of ride-sharing in only 44 out of 108 U.S. geo-markets.
Description
| Type of resource | text |
|---|---|
| Form | electronic; electronic resource; remote |
| Extent | 1 online resource. |
| Publication date | 2017 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Associated with | Polat, Hasan Onder |
|---|---|
| Associated with | Stanford University, Department of Economics. |
| Primary advisor | Wolak, Frank A |
| Thesis advisor | Wolak, Frank A |
| Thesis advisor | Hong, Han |
| Thesis advisor | Larsen, Bradley J |
| Advisor | Hong, Han |
| Advisor | Larsen, Bradley J |
Subjects
| Genre | Theses |
|---|
Bibliographic information
| Statement of responsibility | Hasan Onder Polat. |
|---|---|
| Note | Submitted to the Department of Economics. |
| Thesis | Thesis (Ph.D.)--Stanford University, 2017. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2017 by Hasan Onder Polat
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...