A Systematic Analysis of Model Sensitivity: Investigating the Effect of Wildfire Smoke PM2.5 on Mortality
Abstract/Contents
- Abstract
- Wildfire smoke-related fine particulate matter pollution (PM2.5) is a health hazard known to affect human health and has made up an increasingly large burden of overall PM2.5 pollution in the US over the past two decades. Previous literature studying PM2.5 and mortality has primarily relied on cohort studies and daily time series, with a smaller subset utilizing annually aggregated time series data. We identified several papers that used coarser, ecological data and two-way fixed effects (TWFE) models to study the association between PM2.5 and mortality. Within that literature, there are a variety of different preprocessing and modeling choices. These “researcher degrees of freedom” can lead to a “garden of forking paths”, in which reasonable a priori decisions can produce qualitatively different research findings. To investigate the potential sensitivity of these researcher decisions, we utilized county-month level all-cause mortality data from the CDC’s Multiple Cause of Death files along with new smoke PM2.5 estimates to systematically vary parameters within the TWFE modeling approach used by previous studies. We tested 75 models, 56 of which had negative point estimates. 59 of the models reported statistically significant results when using IID-based standard errors (SEs) while 8 were statistically significant when using robust SEs. The results of our systematic analysis suggest that applying TWFE models to non-daily time series data produces estimates that are minimally robust to different specifications and too wide to detect an effect when robust SEs are used. Based on our replication of Ma et al. 2023 and comparison of the `gnm` and `fixest` packages in R, we believe it is likely that studies finding significant associations between PM2.5 and mortality using non-daily data rely on the IID assumption, and that their confidence intervals may be insufficiently conservative. We recommend that future research studies in this area clearly define the assumptions behind their modeling choices, carefully choose appropriately conservative standard errors, and publish their code to maximize replicability and transparency.
Description
| Type of resource | text |
|---|---|
| Date created | June 1, 2023 |
| Publication date | June 2, 2023 |
Creators/Contributors
| Author | Kaplan, Jordan |
|---|---|
| Advisor | Kiang, Mathew |
| Advisor | Rehkopf, David |
Subjects
| Subject | Particulate matter |
|---|---|
| Subject | Wildfires |
| Subject | Mortality |
| Genre | Text |
| Genre | Thesis |
Bibliographic information
Access conditions
- Use and reproduction
- User agrees that, where applicable, content will not be used to identify or to otherwise infringe the privacy or confidentiality rights of individuals. Content distributed via the Stanford Digital Repository may be subject to additional license and use restrictions applied by the depositor.
- License
- This work is licensed under a Creative Commons Attribution 4.0 International license (CC BY).
Preferred citation
- Preferred citation
- Kaplan, J. (2023). A Systematic Analysis of Model Sensitivity: Investigating the Effect of Wildfire Smoke PM2.5 on Mortality. Stanford Digital Repository. Available at https://purl.stanford.edu/sx623kv4007. https://doi.org/10.25740/sx623kv4007.
Collection
Epidemiology & Clinical Research Masters Theses
View other items in this collection in SearchWorksContact information
Also listed in
Loading usage metrics...