DECISION-MAKING IN UNCERTAIN ENVIRONMENTS: METRICS, SCREENING, AND MAINTENANCE
Abstract/Contents
- Abstract
- The dissertation comprises three chapters, each functioning as a standalone paper. They are organized in the sequence of their completion. A shared theme across all chapters is the focus on improving decision-making to be more effective and reliable, especially in environments filled with data. In "Optimal Design of North-Star Metrics", we focus on developing more robust and better noth-start metrics. Many data-driven short-term decisions require fast-reacting metrics about their success, for example when companies run experiments to improve the targeting of their services. In these cases, the long-term outcomes of interest may not be measured in time to inform the decision, may be too sparse, or may be measured with excess noise, while available fast-reacting proxy variables may still be able to predict long-term consequences. Yet when we learn from past data which short-term proxy variables best predict some long-term outcome of interest, then turning these proxies into new north-star metrics may break their predictive power and have unintended consequences. In this chapter, we formalize the design of proxy metrics from past data as part of a statistical learning problem. Motivated by the central role of north-star metrics for the targeting of services and the optimization of platforms, we ask how to optimally leverage data about outcomes and proxy measurements over time to optimize for long-term outcomes, and how optimal choices compare to ad-hoc proxies fitted on past data. We demonstrate that leveraging proxies involves additional learning, leading to an explore--explore--exploit trade-off in finding optimal policies over time. Having established how proxy variables can be used efficiently in a dynamic learning problem, we extend our model to a dynamic principal-agent problem with misaligned preferences and private information. In this model, a principal uses a proxy metric to incentivize innovation by an agent and align short-term individual choices with the long-term goal of the organization. Similar to the case of dynamic learning, we show that learning from the short-run relationship between proxy variables and outcomes leads to biased and inefficient choices, while the optimal proxy metric can be learned by experimenting with different proxy metrics over time. The second chapter is titled "Screening with Uncertain Preferences". In many screening decisions and assortment optimization problems, organizations select a subset of available options for further evaluation. This includes screening job candidates in hiring and assembling consideration sets on dating and e-commerce platforms. In this chapter, we formulate, analyze, and solve a two-stage screening problem. In the first stage, a platform is presented with a large number of options, from which it must select a small subset based on limited available information about each option and the preferences over them. In the second stage, the performance of the selected options is evaluated against a decision-maker's preference, leading to the selection of the option that best fits the decision-maker's needs. We argue that the problem of selecting an initial consideration set is conceptually and computationally challenging. First, when preferences over options are uncertain, then optimizing for subsets to further consider goes beyond ranking options individually and instead requires considering their complementarity. Second, even simple instances of the initial subset-selection problem are known to be computationally hard. To solve the challenge of selecting effective consideration sets, we propose a two-step approximation algorithm that first eliminates dominated options and then selects a subset among the remaining candidates using sample average approximation. Theoretically, we show that the approximation algorithm finds the optimal subset as we increase the scenario size of an algorithm. In simulations, we further demonstrate that our approach performs significantly better than alternatives based on naive heuristics, while remaining computationally feasible for realistic instances. Practically, we argue that our results have implications for the role of diversity in selection processes. It is sometimes assumed that there is a trade-off between the individual accuracy and diversity of consideration sets in selection processes such as hiring. In contrast, our results highlight a crucial insight for decision-makers: prioritizing diversity at the initial stage of the selection process is more advantageous than merely focusing on the most individually qualified candidates. The third chapter focuses on efficiency degradation and the improvement of maintenance strategies through data analysis at coal power plants in the United States, and is titled "Efficiency Degradation and Maintenance". In this chapter, using detailed data on hourly production, efficiency, and maintenance records from U.S. coal power plants, we examine how their efficiency decreases over time and how maintenance activities help regain this lost efficiency. Our study aims to understand these plants' wear and tear patterns and assess the effectiveness of their maintenance routines in restoring performance. By leveraging high-frequency data on output, fuel consumption, and maintenance decisions, we address the literature regarding using real-world efficiency metrics to evaluate maintenance policies. Our study employs a difference-in-differences approach to assess the efficiency of power plants pre- and post-maintenance, uncovering two key findings: a notable decline in coal power generators' efficiency leading up to maintenance and a post-maintenance efficiency improvement of approximately 1\%. Further, we develop a theoretical maintenance model to determine the optimal maintenance strategy for plant owners, highlighting the effectiveness of a constant-control policy for stationary parameters and adapting strategies for non-stationary settings through clustering production cycles. Looking at past maintenance decisions, we have noticed a trend towards more precise scheduling. This is linked to mergers and acquisitions, which have made maintenance decisions about 30\% more accurate. Our findings contribute to the broader discourse on power plant operational optimization, offering a nuanced understanding of maintenance's role in enhancing efficiency in the U.S. electricity sector.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2024; ©2024 |
| Publication date | 2024; 2024 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Coskun, Abdullah |
|---|---|
| Degree supervisor | Spiess, Jann |
| Thesis advisor | Spiess, Jann |
| Thesis advisor | Bayati, Mohsen |
| Thesis advisor | Karaduman, Omer |
| Degree committee member | Bayati, Mohsen |
| Degree committee member | Karaduman, Omer |
| Associated with | Stanford University, Graduate School of Business |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Abdullah Coskun. |
|---|---|
| Note | Submitted to the School of Business Administration. |
| Thesis | Thesis Ph.D. Stanford University 2024. |
| Location | https://purl.stanford.edu/jr567mw6045 |
Access conditions
- Copyright
- © 2024 by Abdullah Coskun
- License
- This work is licensed under a Creative Commons Attribution 3.0 Unported license (CC BY).
Also listed in
Loading usage metrics...