Using conflicting information and holistic judgments in diagnostic computer models
Abstract/Contents
- Abstract
- Diagnostic computer models use observable variables, referred to as features, to generate predictions. Experts often identify feature observations as the input for diagnostic computer models. In addition to these feature observations, I propose obtaining those experts' holistic judgments about the diagnosis to improve the overall system diagnostic performance. I develop what I call the Holistic Feature Integration (HFI) framework, which allows a decision maker to integrate the holistic judgments of multiple experts along with their (possibly conflicting) feature observations into an overall diagnosis. To resolve expert disagreement I need to model the accuracy of each expert; because the HFI framework only requires one such assessment for each expert, it can be easily implemented in practice. In those situations where the experts can be calibrated on test cases, I develop a statistical procedure for estimating accuracy parameters. Through a case study of medical diagnosis with thyroid sonograms, I demonstrate that the HFI framework substantially improves prediction quality by incorporating experts' holistic judgments. Some of this benefit can be obtained by a simple mixture model of the diagnostic computer model with the expert judgments, especially when there is only one expert. In my case study, the holistic judgments alone in the HFI framework are more beneficial than the diagnostic computer model alone. However, using both holistic judgments and the diagnostic computer model yields even better performance in the HFI framework. I see increasing marginal benefit as additional experts were included. I also develop a new technique for eliciting experts' holistic judgments, which are valuable in the HFI framework. I introduce a new class of strictly proper scoring rules parameterized by a simple tradeoff significant to the decision maker and the risk attitude of the expert. I prove that my new scoring rule incentivizes experts to report their risk attitudes and assessments honestly.
Description
| Type of resource | text |
|---|---|
| Form | electronic; electronic resource; remote |
| Extent | 1 online resource. |
| Publication date | 2012 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Associated with | Sachchamarga, Wititchai |
|---|---|
| Associated with | Stanford University, Department of Management Science and Engineering |
| Primary advisor | Howard, Ronald A. (Ronald Arthur), 1934- |
| Primary advisor | Shachter, Ross D |
| Thesis advisor | Howard, Ronald A. (Ronald Arthur), 1934- |
| Thesis advisor | Shachter, Ross D |
| Thesis advisor | Rubin, Daniel (Daniel L.) |
| Advisor | Rubin, Daniel (Daniel L.) |
Subjects
| Genre | Theses |
|---|
Bibliographic information
| Statement of responsibility | Wititchai Sachchamarga. |
|---|---|
| Note | Submitted to the Department of Management Science and Engineering. |
| Thesis | Thesis (Ph.D.)--Stanford University, 2012. |
| Location | electronic resource |
Access conditions
- Copyright
- © 2012 by Wititchai Sachchamarga
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...