Unpacking the Black Box: Regulating Algorithmic Decisions
Abstract/Contents
- Abstract
- We characterize optimal oversight of algorithms in a world where an agent designs a complex prediction function but a principal is limited in the amount of information she can learn about the prediction function. We show that limiting agents to prediction functions that are simple enough to be fully transparent is inefficient as long as the bias induced by misalignment between principal's and agent's preferences is small relative to the uncertainty about the true state of the world. Ex-post algorithmic audits can improve welfare, but the gains depend on the design of the audit tools. Tools that focus on minimizing overall information loss, the focus of many post-hoc explainer tools, will generally be inefficient since they focus on explaining the average behavior of the prediction function rather than sources of mis-prediction, which matter for welfare-relevant outcomes. Targeted tools that focus on the source of incentive misalignment, e.g., excess false positives or racial disparities, can provide first-best solutions. We provide empirical support for our theoretical findings using an application in consumer lending.
Description
| Type of resource | text |
|---|---|
| Date created | August 25, 2021 |
Creators/Contributors
| Author | Blattner, Laura |
|---|---|
| Author | Nelson, Scott |
| Author | Speiss, Jan |
| Organizer of meeting | Matvos, Gregor |
| Organizer of meeting | Seru, Amit |
Subjects
| Subject | economics |
|---|---|
| Genre | Text |
| Genre | Working paper |
| Genre | Grey literature |
Bibliographic information
Access conditions
- Use and reproduction
- User agrees that, where applicable, content will not be used to identify or to otherwise infringe the privacy or confidentiality rights of individuals. Content distributed via the Stanford Digital Repository may be subject to additional license and use restrictions applied by the depositor.
- License
- This work is licensed under a Creative Commons Attribution 4.0 International license (CC BY).
Preferred citation
- Preferred citation
- Blattner, L., Nelson, S., and Speiss, J. (2022). Unpacking the Black Box: Regulating Algorithmic Decisions. Stanford Digital Repository. Available at https://purl.stanford.edu/ry909yd9970
Collection
SITE Conference 2021
View other items in this collection in SearchWorksContact information
- Contact
- siteworkshop@stanford.edu
Also listed in
Loading usage metrics...