Auditing bias and representation in sociotechnical systems
Abstract/Contents
- Abstract
- Algorithms are ubiquitous and critical sources of information, increasingly acting as gatekeepers and intermediaries on virtually any topic, including our friends and family, news and politics, entertainment, and even health and well-being. However, studying such content poses considerable challenges, as it is both dynamic and ephemeral: these algorithms are constantly changing, and frequently changing silently, with no record of the content to which users have been exposed over time. To address these challenges, in this dissertation I argue for the need for an interdisciplinary approach that combines strategies, methods, and insights from computational and behavioral sciences. In particular, one strategy that has proven effective is the algorithm audit: a method of repeatedly querying an algorithm and observing its output to draw conclusions about the algorithm's opaque inner workings and possible external impact. Algorithm audits can, without access to underlying algorithms, identify patterns in algorithmic content with important social implications in domains including politics, discrimination and bias, and news media. In my dissertation, I present an overview of the algorithm audit methodology, including the history of audit studies in the social sciences from which this method is derived; a summary of key algorithm audits over the last two decades in a variety of domains; and a set of best practices for conducting algorithm audits today. I concretize these contributions by detailing two case studies, scraping algorithm audits I have conducted. Subsequently I describe a new class of algorithm audits I term intervention auditing, and a system developed for researchers to conduct such audits. Finally, I conclude by discussing the social, ethical, and political dimensions of auditing algorithms, and proposing normative standards for the use of this method, in particular advocating for algorithm auditors to consider this method as a possible vehicle for activism—a method with the potential to bring about social change for the greater good.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2021; ©2021 |
| Publication date | 2021; 2021 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Metaxa-Kakavouli, Danaë |
|---|---|
| Degree supervisor | Hancock, Jeff |
| Degree supervisor | Landay, James A, 1967- |
| Thesis advisor | Hancock, Jeff |
| Thesis advisor | Landay, James A, 1967- |
| Thesis advisor | Bernstein, Michael S, 1984- |
| Thesis advisor | Karahalios, Karrie |
| Degree committee member | Bernstein, Michael S, 1984- |
| Degree committee member | Karahalios, Karrie |
| Associated with | Stanford University, Computer Science Department |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Danaë Metaxa-Kakavouli. |
|---|---|
| Note | Submitted to the Computer Science Department. |
| Thesis | Thesis Ph.D. Stanford University 2021. |
| Location | https://purl.stanford.edu/hf228sh9665 |
Access conditions
- Copyright
- © 2021 by Danae Metaxa-Kakavouli
- License
- This work is licensed under a Creative Commons Attribution Non Commercial 3.0 Unported license (CC BY-NC).
Also listed in
Loading usage metrics...