The yellow brick road to artificial intelligence : an empirical study of developers developing artificial intelligent conversational socialbots
Abstract/Contents
- Abstract
- Artificial Intelligence (AI) technologies are increasingly becoming ubiquitous and invisible - nudging, making recommendations, influencing and making decisions, providing information, forming long-term relationships with users, or merely providing company. Despite this prevalence, few studies have examined how these technologies are designed and developed. This study examines how developers of AI technologies, faced with high levels of ambiguity, approach their work. Development of artificial intelligent (AI) technologies presents a unique phenomenon in which two of the three, input-process-output variables are ambiguous. There is opacity in cause-effect, that is, it is difficult if not impossible to know how inputs to AI technologies are related to outputs, thereby making multiple interpretations of causality plausible. The evaluation of outputs of AI technologies is often based on datasets called ground truth that are inherently ambiguous and dependent upon subjective decisions such as what categories to include in the classification system, and on the interpretation of people assigning these categories to different entities in the dataset. Through this ethnographic study of the development process of conversational AI technologies, I find that developers engage in three ambiguity attitudes - avoiding ambiguity by using manually coded, rule-based response generation techniques to exert control over the output of the technology. They also exhibit ambiguity seeking by employing opaque, deep learning, large language models to auto-generate responses to build resilience in the technology to produce an output in unexpected situations in which the technology would otherwise 'fall off the cliff' or fail. At the same time, developers attempt to resolve ambiguity by engaging in a process of building an empirical understanding from first principles of the phenomenon being automated, by ad hoc experimentation with proxy metrics and intuitions. I call this process 'reverse-building of phenomena.' Developers who embraced ambiguity and built resilient technologies fared better in the competition than those who did not. I contribute to an understanding of how modern-day work is changing for developers with the advent of opaque and ambiguous artificial intelligent technologies.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2023; ©2023 |
| Publication date | 2023; 2023 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Jain, Prachee |
|---|---|
| Degree supervisor | Hinds, Pamela |
| Thesis advisor | Hinds, Pamela |
| Thesis advisor | Christin, Angèle |
| Thesis advisor | Valentine, Melissa (Melissa A.) |
| Degree committee member | Christin, Angèle |
| Degree committee member | Valentine, Melissa (Melissa A.) |
| Associated with | Stanford University, School of Engineering |
| Associated with | Stanford University, Department of Management Science and Engineering |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Prachee Jain. |
|---|---|
| Note | Submitted to the Department of Management Science and Engineering. |
| Thesis | Thesis Ph.D. Stanford University 2023. |
| Location | https://purl.stanford.edu/kf812bt6103 |
Access conditions
- Copyright
- © 2023 by Prachee Jain
Also listed in
Loading usage metrics...