publications

Pre-printing and disclosure practices for papers under revision vary across disciplines and journals. Here I follow field-specific conventions. If you would like a copy of a paper not provided here, please email me. First and last author positions indicate primary contributions.

My work spans a few different areas, with the overall objective of bridging decision analytic models and observational causal inference. The tags below roughly filter papers (with ! for papers about which I'm excited).

! Infectious Disease Women's Health Simulation Modeling Causal Inference DiD clear

working papers

ID

CI

DiD

excited

Parallel Trends in an Unparalleled Pandemic: Difference-in-differences for infectious disease outcomes

Shuo Feng and Alyssa Bilinski

Revise and resubmit at Journal of the American Statistical Association

Abs DOI PDF

Researchers frequently employ difference-in-differences (DiD) to study infectious disease policy. This paper formalizes often unaddressed epidemiological assumptions required for common DiD specifications, assuming an underlying Susceptible-Infectious-Recovered (SIR) data-generating process, and proposes more robust specifications for infectious disease outcomes. We first demonstrate that popular specifications can encode strict assumptions: DiD modeling incident infections or rates will produce biased treatment effect estimates unless expected untreated potential outcomes are equal across groups. Modeling log incidence or growth allows for different initial infections under an “infinite susceptible population" assumption, but invokes strong conditions on transmission parameters. We propose alternative specifications based on epidemiological parameters – the effective reproduction number and the effective contact rate – that are both more robust to differences between groups and can be extended to more complex transmission dynamics. We show how treatment effects from our specifications can be transformed to average marginal effects on incidence. In simulations, we highlight minimal difference in power between incidence and log incidence models; our alternative specifications have power lower than incidence or log incidence models, but higher than log growth models. We also show robustness of our effective contact rate specifications to model misspecification. We illustrate practical implications re-analyzing studies of COVID-19 mask policies.
ID

SM

CI

excited

Synthesizing Interrupted Time Series and Transmission Dynamic Models for Evaluation of Mass Insecticide-Treated Net Distribution on Malaria Transmission in Lao People’s Democratic Republic

Shuo Feng, Edric Luo, Julia Dunn, and 7 more authors

Abs

Mass distribution of insecticide-treated nets (ITNs) is a cornerstone of global malaria control, but their real-world impact can vary across settings and be challenging to rigorously quantify as countries approach elimination. While randomized controlled trials consistently demonstrate substantial reductions in malaria incidence and mortality attributable to ITNs, observational evaluations yield more heterogeneous effects due to differences in epidemiological contexts, program implementation, as well as variations in analytic approaches. In particular, interrupted time series (ITS) designs, widely used to model cases or log cases when untreated comparison units are unavailable, may not appropriately account for underlying transmission dynamics. To address this, we propose an ITS framework embedded within a mechanistic infectious disease model that estimates intervention effects on a transmission parameter, which can be transformed to incident infections or cases. Given the risk of model misspecification, we further propose a model validation method using placebo tests in pre-intervention data, which empirically estimates out-of-sample model error in predicting untreated potential outcomes. Applying our method to malaria surveillance data from Laos, where a 2022 mass ITN distribution targeted all health facility catchment areas with recent malaria cases, we estimate a treatment effect of 844 (95% CI: 159, 1640) cases averted in the three quarters post-distribution in the highest-risk stratum. We also show that our proposed estimator outperforms traditional nonmechanistic case and log case ITS specifications in placebo tests. Our findings provide a framework for robust retrospective evaluation of infectious disease control interventions, even absent appropriate untreated comparison groups.
GPTs for Those Who Know and Love OLS: The statistics of large language models

Alyssa Bilinski, Jeremy Goldwasser, and S. Ozair Ali

Abs PDF

Large language models (LLMs) have become increasingly ubiquitous, but most researchers interact with them through chat interfaces without understanding their statistical structure. This paper provides an overview of how generative pre-trained transformers (GPTs) work for researchers with backgrounds in biostatistics, epidemiology, or health economics. We frame GPTs as an extension of familiar statistical methods: ordinary least squares, generalized linear models, and neural networks. We then describe the specific features that enable GPTs to generate text at scale across applications, including tokenization, embeddings, and the attention mechanism that allows models to weigh the relevance of different parts of an input sequence. Throughout, we emphasize that the mathematical operations underlying these models (e.g., matrix multiplication, gradient descent, softmax transformations) are conceptually accessible to researchers with quantitative training, even as optimal architectures and training procedures remain areas of active research. We conclude by discussing factors that have driven recent improvements in model performance, including increased scale, preference learning, expanded context windows, chain-of-thought reasoning, and supporting infrastructure.
SM

What Goes In Must Come Out: Functional testing for complex simulation models

Alyssa Bilinski, Luke Massa, Andrea Ciaranello, and 2 more authors

Abs

Accurate computer code is critical in complex simulation models for health decision science. While previous work has described the value of unit testing each component of model code, there is a dearth of best practices for performing and reporting simulation model tests. Adapting a functional testing paradigm from software engineering, we propose a process for comprehensive testing to verify that the written description of a model, including input parameters and structural assumptions, is accurately reflected in code. To conduct software tests, programmers typically check that code, when provided with a set of inputs, produces expected outputs. However, researchers often create simulation models precisely to avoid closed-form estimation of complex nonlinear functions when it is not straightforward to calculate the expected output (e.g., health outcomes after an intervention). This renders it difficult to assess whether the code performs as expected. To address this, this tutorial describes how to define intermediate outcomes that can be used to assess whether each input performs as expected within the system dynamics defined by the model. We provide a worked example of a simple agent-based infectious disease transmission model as well as a full-scale model of school transmission. We provide a public R script to replicate tests. We propose and demonstrate a flexible, transparent framework for rigorously testing complex simulation models. Functional testing can reduce coding errors and facilitate uniform standards for model code validation.
WH

How Big is the Grey Area? Understanding health-threatening complications in the context of state abortion bans

Alyssa Bilinski, Aileen Gariepy, Rachel Slimovitch, and 4 more authors

Abs

Following the Dobbs decision, abortion restrictions in 18 US states have created clinical uncertainty regarding exceptions for maternal health emergencies. We examine the frequency and consequences of health-threatening pre-viability complications, particularly previable preterm prelabor rupture of membranes (pPROM), where delayed intervention poses serious risks. Drawing on recent surveillance data, we estimate approximately 1 health-threatening complication before viability per 350 live births. Delayed care increases risks of chorioamnionitis, sepsis, hysterectomy, and maternal death, with patient deterioration occurring rapidly. These findings underscore urgent need for clearer statutory language, medical board guidance, and hospital policies to ensure timely, appropriate care during obstetric emergencies.
WH

The Pregnancy Paradox: Fewer conditions treated during pregnancy but continued exposure to medications without safety data

Natalia Emanuel, Ben Lahey, Arianna Pereira, and 2 more authors

Abs

Although 94 percent of individuals report using a medication during pregnancy and medications may have long-term impacts on embryos and fetuses, most prescription medications have not been well-studied during pregnancy. The objective of this study was to compare rates of medication use and conditions treated during pregnancy to those among non-pregnant individuals and identify how these rates differed based on patient demographics and medications’ safety information. We examined medication utilization within the last 30 days and unique conditions treated by those medications among 1,290 pregnant individuals in a representative sample of Americans in the National Health and Nutrition Examination Survey, 2001-2020 compared 10,794 non-pregnant individuals, with statistical adjustment to match demographics of the pregnant population. Pregnant individuals were 32 percent less likely to use a prescription medication in the last 30 days than demographically similar non-pregnant women (p<0.0001). Among pregnant women who took medications, 85 percent took medications that did not have adequate or excellent data to show that they are safe during pregnancy. Pregnant individuals used medications for 43 percent fewer conditions per person than demographically-similar non-pregnant women (p<0.0001). There was, accordingly, a smaller range of unique medications used to treat pregnant patients. Uncertainty about medications’ impacts may contribute to fewer conditions being treated during pregnancy and using a smaller range of medications among pregnant individuals. Additional safety data would allow clinicians and patients to treat conditions with fewer concerns about teratogenic impacts. Limitations include only prescription medication and an inability to determine pregnancy trimester.
WH

Trends in Sexual Activity, Condom Use, and Use of Pregnancy Prevention Methods Among Female Adolescents, 2007-2023

Rachel Slimovitch, Joshua A Salomon, and Alyssa Bilinski

Abs

Adolescent birth rates in the United States are declining, yet regional and racial disparities in teen birth rates and sexually transmitted infections persist. This study examines trends in adolescent sexual behaviors and contraceptive use, using national and state-level Youth Risk Behavior Survey (YRBS) data from 2007 to 2023. We analyzed changes in sexual activity, condom use, pregnancy prevention methods, and forced sexual intercourse among sexually active female high school students, stratifying by region, race/ethnicity, and age. We found a national decrease in sexual activity and an increase in use of highly/moderately effective pregnancy prevention methods, particularly long-acting reversible contraceptives. However, the use of condoms has declined among sexually active female students, and racial disparities in contraceptive use remain, especially among Black and Hispanic adolescents. These trends highlight the need for continued efforts in sexual health education and contraceptive access to address ongoing risks of unintended pregnancy and sexually transmitted infections.
ID

Association Between School Modality and Parental Mental Health during the COVID-19 Pandemic

Rachel Slimovitch, Jennifer R. Head, Mark N. Lurie, and 1 more author

Abs

Importance: School closures during the COVID-19 pandemic were a contentious policy with unclear effects on parent mental health. Objective: To evaluate the impact of virtual schooling on parental mental health during the first year of the COVID-19 pandemic. Design: Retrospective observational study using data from the U.S. Census Bureau’s Household Pulse Survey (HPS). Setting: United States, July 2020 - April 2021. Participants: Primary caregivers of school-aged children (K-12) in the HPS, excluding parents in areas where schools were ordered closed by state and local policies during HPS collection. Main Outcomes and Measures: Mental health outcomes included depression and anxiety, measured by the PHQ-2 and GAD-7. We used multivariable linear regression to examine the association between parental mental health and school modality (virtual vs. in-person) with state-month fixed effects. To investigate whether mental health effects were associated with mode of schooling, we compared across (a) parents with 1 or more children in virtual school vs. parents with all children in in-person school; and (b) families with children in different types of virtual schooling. Results: We found no significant association between having a child in virtual school and anxiety (-0.02; 95 percent CI: -0.05, 0.02; p = 0.31) or depression (0.02; 95 percent CI: -0.00, 0.04; p = 0.053) symptoms when compared to parents with all children in in-person school. However, within parents with a child in virtual school, we found increased depression (-0.09; 95 percent CI: -0.13, -0.05; p < 0.001) and anxiety (-0.06; 95 percent CI: -0.13, 0.00; p = 0.035) symptoms when virtual schooling included live, online interaction as compared to virtual schooling from only pre-recorded lessons. Findings were robust to multiple sensitivity analyses. Discussion: Our results indicate that mental health effects of virtual schooling depend on the type of remote schooling. The increased parental burden of live, interactive virtual schooling associated with worse mental health is an unintended consequence that should be considered in future emergency school policy planning.
ID

CI

Dynamic Case-Control Sampling for Rapid Estimation of Vaccine Effectiveness Against an Emerging Infectious Disease Variant

Taylor Fortnam, Laura Chambers, Alyssa Bilinski, and 7 more authors

Revise and resubmit at Biostatistics

Abs

New SARS-CoV-2 variants arise frequently with different viral properties that can impact the effectiveness of the vaccines. Updating estimates of vaccine effectiveness (VE) in public health surveillance can be limited by the necessity of conducting a distinct study that entails analysis of prospective cohort data or using a test-negative design. We introduce a method for dynamically-updating estimates of VE using data that accumulate in real time. Our method uses dynamic case-control sampling to estimate VE against a newly-emerging variant relative to a previous variant. Dynamic case-control sampling is a technique that continuously updates VE estimates by comparing individuals infected with a newly emerging variant (defined as cases) to those infected with a previously-circulating variant (defined as controls). We use this estimate in combination with information about VE from the previous variant (these estimates are typically available from larger, traditional studies) to infer VE against the emerging variant. We demonstrate the utility of this method on the BA.1 and BA.2 sub-lineages of the Omicron variant. The method produces estimates of VE comparable to those produced using traditional methods, although with increased standard error. The increase in error, however, is reasonable given a much smaller sample size than other studies, and error ranges of the estimates could be significantly improved by sequencing a larger proportion of identified cases. Our method, which assumes only a fraction of the new cases are being sequenced, can be applied by health departments using routinely-collected data to produce timely, rigorous VE estimates to rapidly identify potential changes in VE.
ID

SM

Estimating Time-Varying Epidemic Severity Rates with Adaptive Deconvolution

J Goldwasser, A Hu, Alyssa Bilinski, and 2 more authors

Abs DOI PDF

Several key metrics in public health convey the probability that a primary event will lead to a more serious secondary event in the future. These severity rates can change over the course of an epidemic in response to shifting conditions like new therapeutics, variants, or public health interventions. In practice, time-varying parameters such as the case-fatality rate are typically estimated from aggregate count data. Prior work has demonstrated that commonly-used ratio-based estimators can be highly biased, motivating the development of new methods. In this paper, we develop an adaptive deconvolution approach based on approximating a Poisson-binomial model for secondary events, and we regularize the maximum likelihood solution in this model with a trend filtering penalty to produce smooth but locally adaptive estimates of severity rates over time. This enables us to compute severity rates both retrospectively and in real time. Experiments based on COVID-19 death and hospitalization data, both real and simulated, demonstrate that our deconvolution estimator is generally more accurate than the standard ratio-based methods, and displays reasonable robustness to model misspecification.
ID

SM

Challenges in Estimating Time-Varying Epidemic Severity Rates from Aggregate Data

J Goldwasser, A Hu, Alyssa Bilinski, and 2 more authors

Abs DOI PDF

Severity rates like the case-fatality rate and infection-fatality rate are key metrics in public health. To guide decision-making in response to changes like new variants or vaccines, it is imperative to understand how these rates shift in real time. In practice, time-varying severity rates are typically estimated using a ratio of aggregate counts. We demonstrate that these estimators are capable of exhibiting large statistical biases, with concerning implications for public health practice, as they may fail to detect heightened risks or falsely signal nonexistent surges. We supplement our mathematical analyses with experimental results on real and simulated COVID-19 data. Finally, we briefly discuss strategies to mitigate this bias, drawing connections with effective reproduction number (Rt) estimation.

in press

Reproducibility in Biomedical Research: A Systems Prescription

Gray Babbs, Ishani Ganguli, and Alyssa Bilinski

Journal of General Internal Medicine

Conditional acceptance

Abs

Despite advances in data accessibility and computing power, reproducibility in biomedical research remains poor: only 4% of recent JAMA publications shared code and 5% shared data. Key barriers include limited training in code testing and review, lack of clear standards, time pressures, fears of misuse and scrutiny, and publication disincentives favoring novelty over replication. Researchers can improve practices through systematic code testing, code review by independent programmers, and repeat coding. However, systemic change requires action from journals and funders to establish clear expectations, require code sharing, and support replication studies. Addressing these barriers is essential for maintaining scientific trustworthiness amid increasingly complex research.
WH

excited

Why It’s Unethical Not to Conduct Randomized Trials in Pregnant Women

Alyssa Bilinski

JAMA

In press

Abs

Despite modern drug safety standards, less than 1% of clinical trials include pregnant participants, leaving most medications without rigorous pregnancy evidence. This exclusion paradoxically harms pregnant women: they still take untested medications while researchers cannot systematically learn from their experiences. Recent FDA draft guidance encouraging pregnancy inclusion represents progress but lacks mandates. Regulatory reform should require pregnancy-specific trials for new medications, mirroring successful pediatric drug testing legislation, combined with public funding to address evidence gaps in existing drugs.
“Re: Confounders, Mediators, or Colliders What Types of Shared Covariates Does a Sibling Comparison Design Control For?”

Alyssa Bilinski

Epidemiology

In press

ID

SM

Forecasting Local Surges in COVID-19 Hospitalizations through Adaptive Decision Tree Classifiers

R Murray-Watson, Alyssa Bilinski, and Reza Yaesoubi

Medical Decision Making, 2026

Abs DOI PDF

Introduction: During the COVID-19 pandemic, many communities across the United States experienced surges in hospitalizations, which strained the local hospital capacity. Some risk metrics, such as the Center for Disease Control and Prevention’s (CDC’s) Community Levels, were developed to predict the impact of COVID-19 on the community-level health care system based on routine surveillance data. However, they had limited utility as they were not routinely updated based on accumulating data and were not directly linked to specific outcomes, such as surges in COVID-19 hospitalizations beyond local capacities. Methods: In this article, we evaluated decision tree classifiers developed in real time to predict surges in local hospitalizations due to COVID-19 between July 2020 and November 2022. These classifiers would have provided visually intuitive and interpretable decision rules and, by being updated weekly, would have responded to changes in the epidemic. We compared the performance of these classifiers with that of logistic regression and neural network models using various metrics, including the area under the receiver-operating characteristic curve (auROC) and the area under the precision-recall curve (auPRC). Results: Decision tree classifiers achieved an auROC of >80% for most pandemic weeks and outperformed the CDC’s Community Levels in predicting high hospital occupancy. The auPRC, sensitivity, and specificity of the classifiers varied more substantially over time (between 20% and 100%) and in sync with pandemic waves. Decision tree classifiers demonstrated similar performance compared with logistic regression and neural network models while presenting more interpretable classification rules. Conclusions: Using routinely collected hospital surveillance data, decision tree classifiers can be adaptively updated to predict surges in local hospitalizations. However, the sensitivity and specificity of these classifiers could change markedly during different pandemic waves.
ID

CI

Defining and Estimating Outcomes Directly Averted by a Vaccination Program when Rollout Occurs Over Time

Katherine Min Jia, Christopher B Boyer, Alyssa Bilinski, and 1 more author

Epidemiology, 2026

Abs DOI PDF

During the COVID-19 pandemic, estimating the total deaths averted by vaccination has been of great public health interest. Instead of estimating total deaths averted by vaccination among both vaccinated and unvaccinated individuals, some studies empirically estimated only "directly averted" deaths among vaccinated individuals, typically suggesting that vaccines prevented more deaths overall than directly due to the indirect effect. Here, we define the causal estimand to quantify outcomes "directly averted" by vaccination–i.e., the impact of vaccination for vaccinated individuals, holding vaccination coverage fixed–for vaccination at multiple time points, and show that this estimand is a lower bound on the total outcomes averted when the indirect effect is non-negative. We develop an unbiased estimator for the causal estimand in a one-stage randomized controlled trial (RCT) and explore the bias of a popular "hazard difference" estimator frequently used in empirical studies. We show that even in an RCT, the hazard difference estimator is biased if vaccination has a non-null effect, as it fails to incorporate the greater depletion of susceptibles among the unvaccinated individuals. In simulations, the overestimation is small for averted deaths when infection-fatality rate is low, as for many important pathogens. However, the overestimation can be large for averted infections given a high basic reproduction number. Additionally, we define and compare estimand and estimators for avertible outcomes (i.e., outcomes that could have been averted by vaccination, but were not due to failure to vaccinate). Future studies can explore the identifiability of the causal estimand in observational settings.
WH

excited

Oversights in global gynaecological disability measurement

Alyssa Bilinski and Ezekiel Emanuel

Lancet, 2026

Abs DOI PDF Slides

The Global Burden of Disease (GBD) Study has elevated awareness of non-fatal disabilities, yet its estimates for non-cancerous gynaecological and urogynaecological conditions appear substantially underestimated. Common debilitating conditions, including urinary incontinence, dysmenorrhoea, and menopause symptoms, are absent or underrepresented, while premenstrual syndrome accounts for one-third of reported gynaecological DALYs. Three factors contribute to undercounting: reliance on claims data that miss unreported conditions, omission of conditions lacking disability weights, and potential gender biases in disability weight estimation. We show that US urinary incontinence alone would exceed 350,000 DALYs annually, surpassing any reported gynaecological condition. Accurate measurement requires applying the same rigorous accounting used for other disabling conditions to gynaecological health.
CI

DiD

Difference-in-differences analysis with repeated cross-sectional survey data

K Yee, Alyssa Bilinski, and Youjin Lee

Health Services and Outcomes Research Methodology, 2025

Abs DOI PDF

Difference-in-differences (DiD) approach is one of the most widely used approaches for evaluating policy effects. However, traditional DiD methods may not recover the population-level average treatment effect on the treated (ATT) in the absence of population-level panel data, particularly when the composition of units in the treatment group changes over time. In this work, we address the following two challenges when applying DiD methods with repeated cross-sectional (RCS) survey data: (1) heterogeneous compositions of study samples across different time points, and (2) availability of data for only a sample of the population. We introduce a policy-relevant target estimand and establish its identification conditions. We then propose a new weighting approach that incorporates both estimated propensity scores and given survey weights. We establish the theoretical properties of the proposed method and examine its finite-sample performance through simulations. Finally, we apply our proposed method to a real-world data application, estimating the effect of a beverage tax on adolescent soda consumption in Philadelphia.
CI

DiD

Difference-in-Differences for Health Policy and Practice: A review of modern methods

Shuo Feng, Ishani Ganguli, Youjin Lee, and 3 more authors

Statistics in Medicine, 2025

Abs DOI PDF

Difference-in-differences (DiD) is a popular observational causal inference method in health policy, employed to evaluate the real-world impact of policies and programs. To estimate treatment effects, DiD relies on a "parallel trends assumption" that treatment and comparison groups would have had parallel trajectories on average in the absence of an intervention. Recent years have seen both growing use of DiD in health policy and medicine and rapid advancements in DiD methods. To support DiD implementation in these fields, this paper reviews and synthesizes best practices and recent innovations. We provide recommendations to practitioners in four areas: (1) assessing causal assumptions; (2) adjusting for covariates and other approaches to relax causal assumptions; (3) accounting for staggered treatment timing; and (4) conducting robust inference, especially when normal-based clustered standard errors are inappropriate. For each, we explain challenges and common pitfalls in traditional DiD and recommend methods to address these. We explore current treatment of these topics through a focused literature review of medical DiD studies.
WH

excited

Sins of Omission: Model-based estimates of the health effects of excluding pregnant participants from randomized controlled trials

Alyssa Bilinski, Natalia Emanuel, and Andrea Ciaranello

Annals of Internal Medicine, 2025

Abs DOI PDF

Background: More than 90 million women in the United States have given birth. Randomized controlled trials (RCTs) of medications almost always exclude pregnant participants. Objective: To quantify the health effects of excluding pregnant participants from RCTs. Design: Decision analytic framework applied to case studies of thalidomide, COVID-19 vaccines, and dolutegravir. Setting: Varied. Participants: Pregnant people and their children. Measurements: The authors modeled the ex post facto health effects of RCTs, comparing projected health effects of medication uptake had an RCT been conducted versus historically observed outcomes. They also modeled the a priori health effects that could have been anticipated in trial planning. They converted health effect estimates to monetary value using standard benchmarks. Results: Across case studies, health benefits from conducting RCTs during pregnancy were projected to far exceed expected adverse effects (AEs) from RCTs. For example, had thalidomide been tested in a completed RCT with 200 treated participants, about 33 children would have experienced severe AEs, whereas knowledge from the RCT would have prevented 8000 thalidomide-related birth defects, 99.6% of all thalidomide-related birth defects from 1956 to 1962. Likewise, if RCTs for COVID-19 vaccines had included pregnant participants and if posttrial pregnant uptake were conservatively assumed to mirror that of age- and state-matched nonpregnant women, a projected 20% of COVID-19-related maternal deaths and stillbirths (8% of all maternal deaths and 1% of all stillbirths) in the United States would have been prevented from March to November 2021. Across case studies, the a priori value of RCT data would have exceeded the approximately $100 million cost of phase 1 to 3 RCTs. Limitation: Parameter uncertainty. Conclusion: Systematic inclusion in RCTs could benefit both pregnant people and their children by both speeding AE detection and increasing uptake of beneficial medications.
WH

DiD

Trends in Maternal, Fetal and Infant Mortality in the United States, 2000-2023

R Park, Alyssa Bilinski, Robbie Parks, and 1 more author

JAMA Pediatrics, 2025

Abs DOI PDF

Importance: Accurately measuring maternal mortality trends has been challenging due to changes in data collection. This work disambiguates trends from the effects of introducing the pregnancy checkbox on death certificates and also analyzes closely related fetal and infant mortality. Objective: To describe trends in maternal, fetal, and infant deaths since 2000, including the impact of the COVID-19 pandemic. Design, setting, and participants: A national, population-level, epidemiological, cross-sectional analysis during 2000 to 2023 was conducted as well as a staggered difference-in-differences analysis on the pregnancy checkbox, using the US Centers for Disease Control and Prevention Wide-Ranging Online Data for Epidemiologic Research (WONDER) database on underlying causes of death in the US to identify maternal, infant, and fetal deaths. Study population was restricted to mothers aged 15 to 44 years for all definitions of maternal mortality. Exposures: Staggered introduction of the pregnancy checkbox on death certificates across different states. Main outcomes and measures: Longitudinal study (2000-2023) reporting crude rates per 100000 population for adjusted maternal mortality and per 1000 population for fetal and infant mortality at the national level and by US Census Bureau-designated main census regions, age groups, and race and ethnicity. Staggered difference-in-differences counterfactuals (1999-2023) on impact of pregnancy checkbox. Results: The introduction of the pregnancy checkbox was associated with 6.78 (95% CI, 1.47-12.09) deaths per 100000 live births increase in reported maternal mortality, 66% (95% CI, 14%-117%) of the total increase from 2000 to 2019, with a smaller impact on maternal mortality excluding cause unspecified (adjusted maternal death rates).
WH

Abortion May Be Controversial. Supporting Children Need Not Be

Alyssa Bilinski

JAMA, 2025

Abs DOI PDF

This editorial responds to two JAMA studies demonstrating that state abortion restrictions increased birth rates by 1.7% ( 22,000 excess births) and infant mortality by 6% ( 500 excess deaths) from 2021-2023, with disproportionate effects among minoritized, low-income, and unmarried individuals. While acknowledging ongoing abortion debates, the author argues that supporting children and families merit bipartisan support. Policy recommendations include maintaining Medicaid access, expanding paid parental leave and affordable childcare, and reinstating the enhanced child tax credit. The editorial also calls for improved clinician support in navigating abortion restrictions and continued research on long-term maternal and child health outcomes.
WH

Fewer than 1% of Clinical Trials Include Pregnant Participants

Alyssa Bilinski and Natalia Emanuel

American Journal of Obstetrics and Gynecology, 2025

Abs DOI PDF

Despite advocacy to increase pregnant representation in research, systematic data on enrollment in drug trials remains limited. We analyzed ClinicalTrials.gov data to quantify pregnant enrollment in drug randomized clinical trials (RCTs) from 2008-2023. Of 44,160 RCTs meeting inclusion criteria (female participants aged 18-45), we found only 362 (0.8%) enrolled pregnant participants, while 75% explicitly excluded them. This proportion has remained stable over 15 years. Among trials including pregnant participants, most focused on labor/delivery (38%), pregnancy conditions (22%), or preterm labor (15%); only 19 addressed chronic conditions. Trials enrolling pregnant participants were significantly less likely to be industry-funded (23% vs. 77%). Our findings highlight critical gaps in pregnancy-specific evidence needed for high-quality clinical care.
ID

CI

Association between COVID-19 Incidence and Elementary School Attendance in California: A regression discontinuity analysis

E Lin, Alyssa Bilinski, Philip Collender, and 6 more authors

JAMA Network Open, 2024

Abs DOI PDF

Importance: Understanding the role of school attendance on transmission of SARS-CoV-2 among children is of importance for responding to future epidemics. Estimating discontinuities in outcomes by age of eligibility for school attendance has been used to examine associations between school attendance and a variety of outcomes, but has yet to be applied to describe associations between school attendance and communicable disease transmission. Objective: To estimate the association between eligibility for elementary school and COVID-19 incidence. Design, setting, and participants: This case series used data on all pediatric COVID-19 cases reported to California’s disease surveillance system between May 16, 2020, and December 15, 2022, among children within 24 months of the age threshold for school eligibility. Exposure: Birthdate before or after the age threshold for elementary school eligibility during periods when school was remote vs in person. Main outcomes and measures: COVID-19 cases and hospitalizations. Results: Between May 16, 2020, and December 15, 2022, there were 688278 cases of COVID-19 (348957 cases [50.7%] among boys) and 1423 hospitalizations among children who turned 5 years within 24 months of September 1 of the school year when their infection occurred. The mean (SD) age of the study sample was 5.0 (1.3) years. After adjusting for higher rates of testing in schooled populations, the estimated pooled incidence rate ratio among kindergarten-eligible individuals compared with those born just after the eligibility threshold for in-person fall 2021 semester was 1.52 (95% CI, 1.36-1.68), for in-person spring 2022 semester was 1.26 (95% CI, 1.15-1.39), and for in-person fall 2022 semester was 1.19 (95% CI, 1.03-1.38).
CI

DiD

Methodological Considerations for Difference-in-Differences

Alyssa Bilinski and Ishani Ganguli

JAMA Internal Medicine, 2024

Abs DOI PDF

Difference in differences (DiD) is a popular observational study design that compares pre and post differences between treated and comparison groups. We reviewed a DiD study examining electronic health record use following adoption of team based documentation support, highlighting methodological strengths and limitations. Commendable practices included event study plots to assess parallel trends assumptions and estimators accounting for staggered treatment rollout. However, defining treatment by uptake rather than availability may overestimate effects and limit generalizability, as physicians who adopt scribes likely differ systematically from nonadopters. DiD studies require transparent reporting about assumptions and careful interpretation of treatment effect estimates.
ID

Evaluation of Rhode Island’s Early Geographic COVID-19 Vaccine Prioritization Policy

Taylor M Fortnam, Laura C Chambers, Alyssa Bilinski, and 4 more authors

American Journal of Public Health, 2024

Abs DOI PDF

Objectives. To determine whether geographic prioritization of limited COVID-19 vaccine supply was effective for reducing geographic disparities in case rates. Methods. Rhode Island allocated a portion of the initial COVID-19 vaccine supply to residents of Central Falls, a community already affected by structural policies and inadequate systems that perpetuate health inequities and experiencing disproportionately high COVID-19 morbidity and mortality. The policy was implemented with a culturally and linguistically appropriate community engagement plan and was intended to reduce observed disparities. Using a Bayesian causal analysis with population surveillance data, we evaluated the impact of this prioritization policy on recorded cases over the subsequent 16 weeks. Results. Early geographic prioritization of Central Falls accelerated vaccine uptake, averting an estimated 520 cases (95% confidence interval = 22, 1418) over 16 weeks and reducing cases by approximately 34% during this period (520 averted vs 1519 expected without early prioritization). Conclusions. Early geographic prioritization increased vaccine uptake and reduced cases in Central Falls, thereby reducing geographic disparities. Public Health Implications. Public health institutions should consider geographic prioritization of limited vaccine supply to reduce geographic disparities in case rates.
ID

SM

Screening Strategies to Reduce COVID-19 Mortality in Nursing Homes: A model-based cost-effectiveness analysis

S Dong, Eric Jutkowitz, John Giardina, and 1 more author

JAMA Health Forum, 2024

Abs DOI PDF

Importance: Nursing home residents continue to bear a disproportionate share of COVID-19 morbidity and mortality, accounting for 9% of all US COVID-19 deaths in 2023, despite comprising only 0.4% of the population. Objective: To evaluate the cost-effectiveness of screening strategies in reducing COVID-19 mortality in nursing homes. Design and Setting: An agent-based model was developed to simulate SARS-CoV-2 transmission in the nursing home setting. Parameters were determined using SARS-CoV-2 virus data and COVID-19 data from the Centers for Medicare and Medicaid Services and US Centers for Disease Control and Prevention that were published between 2020 and 2023, as well as data on nursing homes published between 2010 and 2023. The model used in this study simulated interactions and SARS-CoV-2 transmission between residents, staff, and visitors in a nursing home setting. The population used in the simulation model was based on the size of the average US nursing home and recommended staffing levels, with 90 residents, 90 visitors (1 per resident), and 83 nursing staff members. Exposure: Screening frequency (none, weekly, and twice weekly) was varied over 30 days against varying levels of COVID-19 community incidence, booster uptake, and antiviral use. Main Outcomes and Measures: The main outcomes were SARS-CoV-2 infections, detected cases per 1000 tests, and incremental cost of screening per life-year gained. Results: Nursing home interactions were modeled between 90 residents, 90 visitors, and 83 nursing staff over 30 days, completing 4000 to 8000 simulations per parameter combination. The incremental cost-effectiveness ratios of weekly and twice-weekly screening were less than $150,000 per resident life-year with moderate (50 cases per 100000) and high (100 cases per 100000) COVID-19 community incidence across low-booster uptake and high-booster uptake levels.
ID

excited

Adaptive Metrics for an Evolving Pandemic: A dynamic approach to area-level COVID-19 risk designations

Alyssa Bilinski, Joshua A Salomon, and Laura Hatfield

Proceedings of the National Academy of Sciences, 2023

Abs DOI PDF

Throughout the COVID-19 pandemic, policymakers have proposed risk metrics, such as the CDC Community Levels, to guide local and state decision-making. However, risk metrics have not reliably predicted key outcomes and have often lacked transparency in terms of prioritization of false-positive versus false-negative signals. They have also struggled to maintain relevance over time due to slow and infrequent updates addressing new variants and shifts in vaccine- and infection-induced immunity. We make two contributions to address these weaknesses. We first present a framework to evaluate predictive accuracy based on policy targets related to severe disease and mortality, allowing for explicit preferences toward false-negative versus false-positive signals. This approach allows policymakers to optimize metrics for specific preferences and interventions. Second, we propose a method to update risk thresholds in real time. We show that this adaptive approach to designating areas as high risk improves performance over static metrics in predicting 3-wk-ahead mortality and intensive care usage at both state and county levels. We also demonstrate that with our approach, using only new hospital admissions to predict 3-wk-ahead mortality and intensive care usage has performed consistently as well as metrics that also include cases and inpatient bed usage. Our results highlight that a key challenge for COVID-19 risk prediction is the changing relationship between indicators and outcomes of policy interest. Adaptive metrics therefore have a unique advantage in a rapidly evolving pandemic context.
ID

SM

First Do No Harm? Modeling risks and benefits of challenge trials for hepatitis C vaccine development

Alyssa Bilinski, R Slimovitch, Andrew Mendlowitz, and 2 more authors

Clinical Infectious Diseases, 2023

Abs DOI PDF

Background: In 2019, about 58 million individuals were chronically infected with hepatitis C virus. Some experts have proposed challenge trials for hepatitis C virus vaccine development. Methods: We modeled incremental infections averted through a challenge approach, under varying assumptions regarding trial duration, number of candidates, and vaccine uptake. We computed the benefit-risk ratio of incremental benefits to risks for challenge versus traditional approaches. We also benchmarked against monetary costs of achieving incremental benefits through treatment. Results: Our base case assumes 3 vaccine candidates, each with an 11% chance of success, corresponding to a 30% probability of successfully developing a vaccine. Given this probability, and assuming a 5-year difference in duration between challenge and traditional trials, a challenge approach would avert an expected 185000 incremental infections with 20% steady-state uptake compared to a traditional approach and 832000 with 90% uptake (quality-adjusted life-year benefit-risk ratio, 72000 and 323000). It would cost at least 92 million and 416 million, respectively, to obtain equivalent benefits through treatment. BRRs vary considerably across scenarios, depending on input assumptions. Conclusions: Benefits of a challenge approach increase with more vaccine candidates, faster challenge trials, and greater uptake.
ID

Prevalence and Risk Factors for School-Associated Transmission of SARS-CoV-2

Sandra B Nelson, Caitlin M Dugdale, Isaac Ravi Brenner, and 5 more authors

JAMA Health Forum, 2023

Abs DOI PDF

Importance: School-associated SARS-CoV-2 transmission is described as uncommon, although the true transmission rate is unknown. Objective: To identify the SARS-CoV-2 secondary attack rate (SAR) in schools and factors associated with transmission. Design, setting, and participants: This cohort study examined the risk of school-based transmission of SARS-CoV-2 among kindergarten through grade 12 students and staff in 10 Massachusetts school districts during 2 periods: fall 2020/spring 2021 (F20/S21) and fall 2021 (F21). School staff collected data on SARS-CoV-2 index cases and school-based contacts, and SAR was defined as the proportion of contacts acquiring SARS-CoV-2 infection. Exposure: SARS-CoV-2. Main outcomes and measures: Potential factors associated with transmission, including grade level, masking, exposure location, vaccination history, and Social Vulnerability Index (SVI), were analyzed using univariable and multivariable logistic regression models. Results: For F20/S21, 8 school districts (70 schools, more than 33000 students) were included and reported 435 index cases with 1771 school-based contacts. For F21, 5 districts (34 schools, more than 18000 students) participated and reported 309 index cases with 1673 school-based contacts. The F20/S21 SAR was 2.2%, and the F21 SAR was 2.8%. In multivariable analysis, during F20/S21, masking was associated with a lower odds of transmission compared with not masking (odds ratio, 0.12; 95% CI, 0.02-0.75). During F21, student contacts in elementary grades had a lower odds of transmission compared with high school contacts (OR, 0.42; 95% CI, 0.20-0.89). Higher district-level SVI was associated with higher odds of transmission during both periods. Conclusions and relevance: In this cohort study, school-based transmission of SARS-CoV-2 was relatively uncommon, and masking and younger age appeared protective. Communities with higher social vulnerability experienced higher transmission rates.
ID

SM

Projected COVID-19 Mortality Reduction from Test-to-Treat: A model-based analysis

S Kumar, M Khunte, Joshua A Salomon, and 1 more author

JAMA Health Forum, 2023

Abs DOI PDF

COVID-19 was the third leading cause of death in the US in 2022, yet the population level impact of Paxlovid rollout has not been estimated. We modeled COVID-19 hospitalization and mortality reductions associated with Paxlovid during a surge comparable to the 2022 winter Omicron wave. At current 5% uptake among eligible individuals, we estimated relative reductions of 2.7% in hospitalizations and 3.2% in mortality. If uptake increased to 80%, we projected 42% and 51% reductions in hospitalizations and mortality, respectively, requiring 75.3 million symptomatic tests and 39.8 million Paxlovid courses. Our framework quantifies resource requirements and benefits for expanding test to treat initiatives.
CI

DiD

What’s Trending in Difference-in-Differences? A synthesis of the recent econometrics literature

Jonathan Roth, Pedro H C Sant’Anna, Alyssa Bilinski, and 1 more author

Journal of Econometrics, 2023

Abs DOI PDF

This paper synthesizes recent advances in the econometrics of difference-in-differences (DiD) and provides concrete recommendations for practitioners. We begin by articulating a simple set of canonical assumptions under which the econometrics of DiD are well-understood. We then argue that recent advances in DiD methods can be broadly classified as relaxing some components of the canonical DiD setup, with a focus on (i) multiple periods and variation in treatment timing, (ii) potential violations of parallel trends, or (iii) alternative frameworks for inference. Our discussion highlights the different ways that the DiD literature has advanced beyond the canonical model, and helps to clarify when each of the papers will be relevant for empirical work. We conclude by discussing some promising areas for future research.
ID

SM

Estimated Testing, Tracing, and Vaccination Targets for Containment of the US Mpox Outbreak

Melanie H Chitwood, Jiye Kwon, Alexandra Savinkina, and 3 more authors

JAMA Network Open, 2023

Abs DOI PDF

Reducing transmission of mpox virus among men who have sex with men (MSM) is crucial for outbreak containment, yet target levels for public health measures have not been established. We developed a deterministic branching model to estimate testing, contact tracing, and vaccination levels required to reduce the effective reproduction number below 1 among high risk MSM. We found that testing and contact tracing alone can achieve containment if the basic reproduction number is below 1.4 with sufficient case detection. With moderate public health response, vaccination thresholds ranged from 5% to 43% of the high risk population, corresponding to 170,000 to 1.4 million vaccine doses depending on transmissibility assumptions.
ID

COVID-19 and Excess All-Cause Mortality in the US and 18 Comparison Countries by Vaccination Rate, March 2020-March 2022

Alyssa Bilinski, K Thompson, and Ezekiel Emanuel

JAMA, 2023

Abs DOI PDF

The US experienced high COVID-19 mortality during 2020, but cross national differences following widespread vaccination and new variants remained unclear. We compared COVID-19 and excess all cause mortality in the US, the 10 most and least vaccinated states, and 20 peer countries during the Delta and Omicron waves (June 2021 to March 2022). The US reported 112 COVID-19 deaths per 100,000, significantly exceeding all peer countries. The 10 most vaccinated states (73% coverage) experienced 75 deaths per 100,000 compared with 146 in the least vaccinated states (52% coverage). These findings highlight that the US continued to lag peer countries, with notably lower mortality in highly vaccinated states.
Modeling Cost and Outcomes of SARS-CoV-2 School Testing Programs–Reply

Alyssa Bilinski and Andrea Ciaranello

JAMA Pediatrics, 2023

DOI PDF
ID

SM

Estimating Deaths Averted and Cost per Life Saved by Scaling up mRNA COVID-19 Vaccination in Low-income and Lower-middle-income countries in the COVID-19 Omicron Variant Era: A modelling study

Alexandra Savinkina, Alyssa Bilinski, Meagan Fitzpatrick, and 5 more authors

BMJ Open, 2022

Abs DOI PDF

Objectives: While almost 60% of the world has received at least one dose of COVID-19 vaccine, the global distribution of vaccination has not been equitable. Only 4% of the population of low-income countries (LICs) has received a full primary vaccine series, compared with over 70% of the population of high-income nations. Design: We used economic and epidemiological models, parameterised with public data on global vaccination and COVID-19 deaths, to estimate the potential benefits of scaling up vaccination programmes in LICs and lower-middle-income countries (LMICs) in 2022 in the context of global spread of the Omicron variant of SARS-CoV2. Setting: Low-income and lower-middle-income nations. Main outcome measures: Outcomes were expressed as number of avertable deaths through vaccination, costs of scale-up and cost per death averted. We conducted sensitivity analyses over a wide range of parameter estimates to account for uncertainty around key inputs. Findings: Globally, universal vaccination in LIC/LMIC with three doses of an mRNA vaccine would result in an estimated 1.5 million COVID-19 deaths averted with a total estimated cost of US61 billion and an estimated cost-per-COVID-19 death averted of US40800 (sensitivity analysis range: US7400-US81500). Lower estimated infection fatality ratios, higher cost-per-dose and lower vaccine effectiveness or uptake lead to higher cost-per-death averted estimates in the analysis. Conclusions: Scaling up COVID-19 global vaccination would avert millions of COVID-19 deaths and represents a reasonable investment in the context of the value of a statistical life. Given the magnitude of expected mortality facing LIC/LMIC without vaccination, this effort should be an urgent priority.
ID

Evaluating the Performance of Centers for Disease Control and Prevention COVID-19 Community Levels as Leading Indicators of COVID-19 Mortality

Joshua A Salomon and Alyssa Bilinski

Annals of Internal Medicine, 2022

Abs DOI PDF

Background: Centers for Disease Control and Prevention (CDC) defines low, medium, and high COVID-19 community levels to guide interventions, but associated mortality rates have not been reported. Objective: To evaluate the diagnostic performance of CDC COVID-19 community level metrics as predictors of elevated community mortality risk. Design: Time series analysis over the period of 30 May 2021 through 4 June 2022. Setting: U.S. states and counties. Participants: U.S. population. Measurements: CDC COVID-19 community level metrics based on hospital admissions, bed occupancy, and reported cases; reported COVID-19 deaths; and sensitivity, specificity, and predictive values for CDC and alternative metrics. Results: Mean and median weekly mortality rates per 100000 population after onset of high COVID-19 community level 3 weeks prior were, respectively, 2.6 and 2.4 across 90 high episodes in states and 4.3 and 2.1 across 7987 high episodes in counties. Alternative metrics based on lower hospital admissions or case thresholds were associated with lower mortality and had higher sensitivity and negative predictive value for elevated mortality, but the CDC metrics had higher specificity and positive predictive value. Ratios between cases, hospitalizations, and deaths have varied substantially over time. Limitations: Aggregate mortality does not account for nonfatal outcomes or disparities. Continuing evolution of viral variants, immunity, clinical interventions, and public health mitigation strategies complicate prediction for future waves.
ID

Determining the Optimal Length of Quarantine–Transmission, Social, and Economic Considerations

Alyssa Bilinski

JAMA Network Open, 2022

Abs DOI PDF

Quarantine of exposed contacts is a mainstay of infectious disease control, yet the burden during the COVID-19 pandemic has been substantial. We argue that transmission risk should not be the sole consideration in determining optimal quarantine length. Overly restrictive policies create psychological, social, logistical, and financial harms while potentially decreasing adherence. Test to stay approaches in schools demonstrate that less restrictive alternatives can achieve similar transmission outcomes with markedly reduced disruption. We call for rigorous evaluation of alternative quarantine policies through randomized trials in frequently tested cohorts, investigating both transmission impacts and adherence. Infectious disease control strategies must balance benefits against costs and be willing to innovate on traditional approaches.
ID

SM

SARS-CoV-2 Testing Strategies to Contain School-Associated Transmission: Model-based analysis of impact and cost of diagnostic testing, screening, and surveillance

Alyssa Bilinski, Andrea Ciaranello, Meagan C Fitzpatrick, and 4 more authors

JAMA Pediatrics, 2022

Abs HTML PDF

Importance: In addition to illness, the COVID-19 pandemic has led to historic educational disruptions. In March 2021, the federal government allocated $10 billion for COVID-19 testing in US schools. Objective: Costs and benefits of COVID-19 testing strategies were evaluated in the context of full-time, in-person kindergarten through eighth grade (K-8) education at different community incidence levels. Design, setting, and participants: An updated version of a previously published agent-based network model was used to simulate transmission in elementary and middle school communities in the United States. Assuming dominance of the delta SARS-CoV-2 variant, the model simulated an elementary school (638 students in grades K-5, 60 staff) and middle school (460 students grades 6-8, 51 staff). Exposures: Multiple strategies for testing students and faculty/staff, including expanded diagnostic testing (test to stay) designed to avoid symptom-based isolation and contact quarantine, screening (routinely testing asymptomatic individuals to identify infections and contain transmission), and surveillance (testing a random sample of students to identify undetected transmission and trigger additional investigation or interventions). Main outcomes and measures: Projections included 30-day cumulative incidence of SARS-CoV-2 infection, proportion of cases detected, proportion of planned and unplanned days out of school, cost of testing programs, and childcare costs associated with different strategies.
Affordability and Value in Decision Rules for Cost-Effectiveness: A survey of health economists

Alyssa Bilinski, Evan MacKay, Joshua A Salomon, and 1 more author

Value in Health, 2022

Abs DOI PDF

Objectives: New health technologies are often expensive, but may nevertheless meet standard thresholds for cost effectiveness, a situation exemplified by recent hepatitis C cures. Currently, cost-effectiveness analysis (CEA) does not supply practical means of weighing trade-offs between cost-effectiveness and affordability, particularly when costs and benefits are temporally separated and in health systems with multiple payers, such as the United States. We formally characterized disagreements in CEA theory and identified how these trade-offs are presently addressed in practice. Methods: We surveyed 170 health economics researchers. Results: When presented with a hypothetical cost-effective drug therapy in the United States that would require 20% of a state’s Medicaid budget over 5 years, 34% of survey respondents recommended that policy makers fund the drug for all patients and 26% for a subset. By contrast, 26% recommended against funding the drug. We found additional disagreement regarding whether the willingness-to-pay threshold should be based on the budget (42%) or societal preferences (41%) and identified 4 approaches to weighing cost-effectiveness and affordability. A total of 61% of respondents did not believe that the threshold used in their last article (most often 1x-3x per capita gross domestic product) represented either the budget or societal willingness-to-pay threshold. Conclusions: We use these findings to recommend metrics that can inform translation of CEA theory into practice. By contextualizing cost and value, researchers can provide more actionable policy recommendations.
ID

SM

Model-Estimated Association Between Simulated US Elementary School-Related SARS-CoV-2 Transmission, Mitigation Interventions, and Vaccine Coverage Across Local Incidence Levels

John Giardina, Alyssa Bilinski, Meagan C Fitzpatrick, and 4 more authors

JAMA Network Open, 2022

Abs DOI PDF

Importance: With recent surges in COVID-19 incidence and vaccine authorization for children aged 5 to 11 years, elementary schools face decisions about requirements for masking and other mitigation measures. These decisions require explicit determination of community objectives and quantitative estimates of the consequences of changing mitigation measures. Objective: To estimate the association between adding or removing in-school mitigation measures and COVID-19 outcomes within an elementary school community at varying student vaccination and local incidence rates. Design, setting, and participants: This decision analytic model used an agent-based model to simulate SARS-CoV-2 transmission within a school community, with a simulated population of students, teachers and staff, and their household members. Transmission was evaluated for a range of observed local COVID-19 incidence (0-50 cases per 100000 residents per day, assuming 33% of all infections detected). The population used in the model reflected the mean size of a US elementary school, including 638 students and 60 educators and staff members in 6 grades with 5 classes per grade. Exposures: Variant infectiousness, mitigation effectiveness, and student vaccination levels were varied. Main outcomes and measures: The main outcomes were probability of at least 1 in-school transmission per month and mean increase in total infections per month among the immediate school community associated with a reduction in mitigation. Results: With student vaccination coverage of 70% or less and moderate assumptions about mitigation effectiveness, mitigation could only be reduced when local case incidence was 14 or fewer cases per 100000 residents per day to keep the mean additional cases associated with reducing mitigation to 5 or fewer.
ID

CI

Problems with Evidence Assessment in COVID-19 Health Policy Impact Evaluation (PEACHPIE): A systematic review of evidence strength

Noah A Haber, Emma Clarke-Deelder, Avi Feller, and 21 more authors

BMJ Open, 2022

Abs DOI PDF

Introduction: Assessing the impact of COVID-19 policy is critical for informing future policies. However, there are concerns about the overall strength of COVID-19 impact evaluation studies given the circumstances for evaluation and concerns about the publication environment. Methods: We included studies that were primarily designed to estimate the quantitative impact of one or more implemented COVID-19 policies on direct SARS-CoV-2 and COVID-19 outcomes. After searching PubMed for peer-reviewed articles published on 26 November 2020 or earlier and screening, all studies were reviewed by three reviewers first independently and then to consensus. The review tool was based on previously developed and released review guidance for COVID-19 policy impact evaluation. Results: After 102 articles were identified as potentially meeting inclusion criteria, we identified 36 published articles that evaluated the quantitative impact of COVID-19 policies on direct COVID-19 outcomes. Nine studies were set aside because the study design was considered inappropriate for COVID-19 policy impact evaluation (n=8 pre/post; n=1 cross-sectional), and 27 articles were given a full consensus assessment. 20/27 met criteria for graphical display of data, 5/27 for functional form, 19/27 for timing between policy implementation and impact, and only 3/27 for concurrent changes to the outcomes. Only 4/27 were rated as overall appropriate. Including the 9 studies set aside, reviewers found that only four of the 36 identified published and peer-reviewed health policy impact evaluation studies passed a set of key design checks for identifying the causal impact of policies on COVID-19 outcomes. Discussion: The reviewed literature directly evaluating the impact of COVID-19 policies largely failed to meet key design criteria for inference of sufficient rigour to be actionable by policy-makers.
ID

COVID Trends and Impact Survey in the United States, 2020-2021: Continuous real-time measurement of COVID-19 symptoms, risks, protective behaviors, testing and vaccination

Joshua A Salomon, Alex Reinhart, Alyssa Bilinski, and 10 more authors

Proceedings of the National Academy of Sciences, 2021

Abs DOI PDF

The US COVID-19 Trends and Impact Survey (CTIS) is a large, cross-sectional, internet-based survey that has operated continuously since April 6, 2020. By inviting a random sample of Facebook active users each day, CTIS collects information about COVID-19 symptoms, risks, mitigating behaviors, mental health issues, testing, vaccination, and other key priorities. The large scale of the survey-over 20 million responses in its first year of operation-allows tracking of trends over short timescales and allows comparisons at fine demographic and geographic detail. The survey has been repeatedly revised to respond to emerging public health priorities. In this paper, we describe the survey methods and content and give examples of CTIS results that illuminate key patterns and trends and help answer high-priority policy questions relevant to the COVID-19 epidemic and response. These results demonstrate how large online surveys can provide continuous, real-time indicators of important outcomes that are not subject to public health reporting delays and backlogs. The CTIS offers high value as a supplement to official reporting data by supplying essential information about behaviors, attitudes toward policy and preventive measures, economic impacts, and other topics not reported in public health surveillance systems.
ID

SM

Passing the Test: A model-based analysis of safe school-reopening strategies

Alyssa Bilinski, Joshua A Salomon, John Giardina, and 2 more authors

Annals of Internal Medicine, 2021

Abs DOI PDF

Background: The COVID-19 pandemic has induced historic educational disruptions. In April 2021, about 40% of U.S. public school students were not offered full-time in-person education. Objective: To assess the risk for SARS-CoV-2 transmission in schools. Design: An agent-based network model was developed to simulate transmission in elementary and high school communities, including home, school, and interhousehold interactions. Setting: School structure was parametrized to reflect average U.S. classrooms, with elementary schools of 638 students and high schools of 1451 students. Daily local incidence was varied from 1 to 100 cases per 100000 persons. Participants: Students, faculty, staff, and adult household members. Intervention: Isolation of symptomatic individuals, quarantine of an infected individual’s contacts, reduced class sizes, alternative schedules, staff vaccination, and weekly asymptomatic screening. Measurements: Transmission was projected among students, staff, and families after a single infection in school and over an 8-week quarter, contingent on local incidence. Results: School transmission varies according to student age and local incidence and is substantially reduced with mitigation measures. Nevertheless, when transmission occurs, it may be difficult to detect without regular testing because of the subclinical nature of most children’s infections. Teacher vaccination can reduce transmission to staff, and asymptomatic screening improves understanding of local circumstances and reduces transmission. Limitation: Uncertainty exists about the susceptibility and infectiousness of children, and precision is low regarding the effectiveness of specific countermeasures, particularly with new variants. Conclusion: With controlled community transmission and moderate mitigation, elementary schools can open safely, but high schools require more intensive mitigation. Asymptomatic screening can facilitate reopening at higher local incidence while minimizing transmission risk.
ID

Better Late than Never: Trends in COVID-19 infection rates, risk perceptions, and behavioral responses in the USA

Alyssa Bilinski, Ezekiel Emanuel, Joshua A Salomon, and 1 more author

Journal of General Internal Medicine, 2021

Abs DOI PDF

Misinformation and inconsistent messaging surrounding COVID-19 may lead individuals to incorrectly assess transmission risk and undermine precautionary behaviors. We analyzed data from a large ongoing survey of nearly 10 million Facebook users from May through November 2020 to assess state level patterns in risk perceptions and behaviors relative to case rates. We found strong correspondence between rising cases and increases in risk perceptions and mitigating behaviors, with changes most pronounced in states experiencing the largest outbreaks. However, the temporal coincidence between surging cases and behavior changes suggests this reactive dynamic is likely too late to prevent substantial outbreaks. Clear and consistent messaging may help accelerate public responses.
ID

COVID-19 and Excess All-Cause Mortality in the US and 18 Comparison Countries

Alyssa Bilinski and Ezekiel J Emanuel

JAMA, 2020

Abs DOI PDF

The US experienced high COVID-19 mortality, but excess all cause mortality provides a more complete picture by capturing indirect pandemic effects and avoiding differences in death coding. We compared US excess all cause mortality through July 2020 with 14 comparison countries. US excess mortality was 71.6 per 100,000, exceeding moderate mortality countries where rates remained negligible. While some high mortality countries initially exceeded the US, after May 10 and June 7, US excess all cause mortality surpassed all comparison countries. If the US matched other high mortality countries from May onward, 24% to 44% of excess deaths could have been averted, suggesting a comparatively poor longer term pandemic response.
ID

SM

Modeling Contact Tracing Strategies for COVID-19 in the Context of Relaxed Physical Distancing Measures

Alyssa Bilinski, Farzad Mostashari, and Joshua A Salomon

JAMA Network Open, 2020

Abs DOI PDF

Contact tracing is recommended as a key component of COVID-19 control, but its potential to reduce transmission under varying implementation conditions remained unclear. We developed a deterministic branching model to examine contact tracing effectiveness under different assumptions for case detection, tracing success, and quarantine efficacy. When community detection and contact tracing were both below 50%, programs reduced the effective reproductive number by less than 10%. Testing asymptomatic contacts increased program benefits by a median factor of 1.28. Contact tracing can support partial relaxation of physical distancing but not a full return to pre lockdown contact levels.
ID

County-Level Association of Social Vulnerability with COVID-19 Cases and Deaths in the USA

Rohan Khazanchi, Evan R Beiter, Suhas Gondi, and 3 more authors

Journal of General Internal Medicine, 2020

Abs DOI PDF

Vulnerable populations faced greater disease burden in past pandemics, but the relationship between social vulnerability and COVID-19 outcomes remains unclear. We performed a county level cross sectional analysis using the CDC Social Vulnerability Index and COVID-19 data through April 2020 across 2754 US counties. People in the most vulnerable counties had 1.63 fold greater risk of COVID-19 diagnosis and 1.73 fold greater risk of death compared with least vulnerable counties. The minority status and language domain showed the strongest associations, with 4.94 fold and 4.74 fold greater risks respectively. Targeted interventions addressing geographically variable social vulnerabilities may be necessary to improve inequitable pandemic outcomes.
ID

Partnering with Facebook on a University-Based Rapid Turn-Around Global Survey

Frauke Kreuter, Neta Barkay, Alyssa Bilinski, and 16 more authors

Survey Research Methods, 2020

Abs DOI PDF

This paper describes a partnership between Facebook and academic institutions to create a global COVID-19 symptom survey. The survey is available in 56 languages. A representative sample of Facebook users is invited on a daily basis to report on symptoms, social distancing behavior, mental health issues, and financial constraints. Facebook provides weights to reduce nonresponse and coverage bias. Privacy protection and disclosure-avoidance mechanisms are implemented by both partners to meet global policy and industry requirements. Country and region-level statistics are published daily via dashboards, and microdata are available for researchers via data use agreements. Over 1 million responses are collected weekly.
Potential Unintended Effects of Medicare’s Bundled Payments for Care Improvement Program – Comment

Alyssa Bilinski and Laura Hatfield

JAMA, 2019

DOI PDF
ID

Spatially Targeted Screening to Reduce Tuberculosis Transmission in High-Incidence Settings

Patrick Cudahy, Jason Andrews, Alyssa Bilinski, and 6 more authors

Lancet Infectious Diseases, 2018

Abs DOI PDF

As the leading infectious cause of death worldwide and the primary proximal cause of death in individuals living with HIV, tuberculosis remains a global concern. Existing tuberculosis control strategies that rely on passive case-finding appear insufficient to achieve targets for reductions in tuberculosis incidence and mortality. Active case-finding strategies aim to detect infectious individuals earlier in their infectious period to reduce onward transmission and improve treatment outcomes. Empirical studies of active case-finding have produced mixed results and determining how to direct active screening to those most at risk remains a topic of intense research. Our systematic review of literature evaluating the effects of geographically targeted tuberculosis screening interventions found three studies in low tuberculosis incidence settings, but none conducted in high tuberculosis incidence countries. We discuss open questions related to the use of spatially targeted approaches for active screening in countries where tuberculosis incidence is highest.
ID

SM

Tuberculosis Control Interventions Targeted to Previously Treated People in a High-Incidence Setting: A modelling study

Florian Marx, Reza Yaesoubi, Nicolas Menzies, and 4 more authors

Lancet Global Health, 2018

Abs DOI PDF

Background: In high-incidence settings, recurrent disease among previously treated individuals contributes substantially to the burden of incident and prevalent tuberculosis. The extent to which interventions targeted to this high-risk group can improve tuberculosis control has not been established. We aimed to project the population-level effect of control interventions targeted to individuals with a history of previous tuberculosis treatment in a high-incidence setting. Methods: We developed a transmission-dynamic model of tuberculosis and HIV in a high-incidence setting with a population of roughly 40000 people in suburban Cape Town, South Africa. The model was calibrated to data describing local demography, TB and HIV prevalence, TB case notifications and treatment outcomes using a Bayesian calibration approach. We projected the effect of annual targeted active case finding in all individuals who had previously completed tuberculosis treatment and targeted active case finding combined with lifelong secondary isoniazid preventive therapy. We estimated the effect of these targeted interventions on local tuberculosis incidence, prevalence, and mortality over a 10 year period (2016-25). Findings: We projected that, under current control efforts in this setting, the tuberculosis epidemic will remain in slow decline for at least the next decade. Additional interventions targeted to previously treated people could greatly accelerate these declines. We projected that annual targeted active case finding combined with secondary isoniazid preventive therapy in those who previously completed tuberculosis treatment would avert 40% (95% uncertainty interval 21-56) of incident tuberculosis cases and 41% (16-55) of tuberculosis deaths occurring between 2016 and 2025. Interpretation: In this high-incidence setting, the use of targeted active case finding in combination with secondary isoniazid preventive therapy in previously treated individuals could accelerate decreases in tuberculosis morbidity and mortality.
Bacterial Contamination of Reusable Bottled Drinking Water in Ecuador

Jillian Golden, Katherine Mills, Alyssa Bilinski, and 6 more authors

Journal of Water, Sanitation and Hygiene for Development, 2017

Abs DOI PDF

In northern coastal Ecuador, water is routinely sold in 20 L reusable bottles for household consumption. These bottles are filled at central treatment facilities and distributed by private water companies. Similar bottled water markets are found in countries around the world. Commercially available bottled water offers an alternative source of drinking water in locations where piped infrastructure may be unsafe or non-existent. In this study we found that 73% (n = 94/128) of water sold in reusable containers in the Esmeraldas province of Ecuador was contaminated with coliform bacteria. In comparison, 25% (n = 9/36) of non-reusable bottles and 9% (n = 2/22) of water samples taken directly from the water treatment system contained coliform, suggesting that most observed bacterial contamination occurred due to inadequate cleaning of reusable bottles between use. The coliform contamination may pose a health risk to the Esmeraldas population. The present study may be indicative of similar situations in low- and middle-income countries around the world, given the widespread use of reusable bottles for water.
ID

Distance to Care, Enrollment and Retention of HIV Patients during Decentralization of Antiretroviral Therapy in Neno District, Malawi

Alyssa Bilinski, Elizabeth Birru, Megan Peckarsky, and 4 more authors

PLoS One, 2017

Abs DOI PDF

HIV/AIDS remains the second most common cause of death in low and middle-income countries (LMICs), and only 34% of eligible patients in Africa received antiretroviral therapy (ART) in 2013. This study investigated the impact of ART decentralization on patient enrollment and retention in rural Malawi. We reviewed electronic medical records of patients registered in the Neno District ART program from August 1, 2006, when ART first became available, through December 31, 2013. We used GPS data to calculate patient-level distance to care, and examined number of annual ART visits and one-year lost to follow-up (LTFU) in HIV care. The number of ART patients in Neno increased from 48 to 3,949 over the decentralization period. Mean travel distance decreased from 7.3 km when ART was only available at the district hospital to 4.7 km when ART was decentralized to 12 primary health facilities. For patients who transferred from centralized care to nearer health facilities, mean travel distance decreased from 9.5 km to 4.7 km. Following a transfer, the proportion of patients achieving the clinic’s recommended 4 or more annual visits increased from 89% to 99%. In Cox proportional hazards regression, patients living 8 km or more from a health facility had a greater hazard of being LTFU compared to patients less than 8 km from a facility (adjusted HR: 1.7; 95% CI: 1.5-1.9). ART decentralization in Neno District was associated with increased ART enrollment, decreased travel distance, and increased retention in care. Increasing access to ART by reducing travel distance is one strategy to achieve the ART coverage and viral suppression objectives of the 90-90-90 UNAIDS targets in rural impoverished areas.
When Cost-Effective Interventions are Unaffordable: Integrating cost-effectiveness and budget impact in priority setting for global health programs

Alyssa Bilinski, Peter Neumann, Joshua Cohen, and 3 more authors

PLoS Medicine, 2017

Abs DOI PDF

Cost effectiveness analysis (CEA) informs health priority setting, yet cost effective interventions are not always affordable. We assessed the use of budget impact analysis (BIA), which estimates short term payer costs, in global health economic evaluations. Reviewing the Tufts Global Health CEA Registry, we found only 3% of articles conducted formal BIA and 10% included informal budget impact measures. When both analyses were presented, conclusions often diverged: more than half of articles with BIA findings concluded that cost effective interventions might be unaffordable. We identify several reasons for this divergence, including willingness to pay thresholds that exceed actual budgets, differences in analytical perspective, distribution of costs and benefits over time, and discounting practices. We recommend researchers report BIA alongside CEA and explain divergent conclusions. Policymakers should recognize that not all cost effective interventions are affordable and interpret cost effectiveness in the context of available budgets when designing essential health service packages.
ID

SM

One Health Approach to Cost-Effective Rabies Control in India

Meagan C Fitzpatrick, Hiral Shah, Abhishek Pandey, and 6 more authors

Proceedings of the National Academy of Sciences, 2016

Abs DOI PDF

Over 20,000 rabies deaths occur annually in India, representing one-third of global human rabies. The Indian state of Tamil Nadu has pioneered a One Health committee to address the challenge of rabies in dogs and humans. Currently, rabies control in Tamil Nadu involves postexposure vaccination of humans after dog bites, whereas potential supplemental approaches include canine vaccination and sterilization. We developed a data-driven rabies transmission model fit to human rabies autopsy data and human rabies surveillance data from Tamil Nadu. Integrating local estimates for canine demography and costs, we predicted the impact of canine vaccination and sterilization on human health outcomes and evaluated cost-effectiveness according to the WHO criteria for India, which correspond to thresholds of 1,582 and 4,746 per disability-adjusted life-years (DALYs) for very cost-effective and cost-effective strategies, respectively. We found that highly feasible strategies focused on stray dogs, vaccinating as few as 7% of dogs annually, could very cost-effectively reduce human rabies deaths by 70% within 5 y, and a modest expansion to vaccinating 13% of stray dogs could cost-effectively reduce human rabies by almost 90%. Through integration over parameter uncertainty, we find that, for a cost-effectiveness threshold above $1,400 per DALY, canine interventions are at least 95% likely to be optimal. If owners are willing to bring dogs to central point campaigns at double the rate that campaign teams can capture strays, expanded annual targets become cost-effective. This case study of cost-effective canine interventions in Tamil Nadu may have applicability to other settings in India and beyond.
ID

SM

Optimal Frequency of Rabies Vaccination Campaigns in Sub-Saharan Africa

Alyssa Bilinski, Meagan C Fitzpatrick, Charles Rupprecht, and 2 more authors

Proceedings of the Royal Society B, 2016

Abs DOI PDF

Rabies causes more than 24000 human deaths annually in Sub-Saharan Africa. The World Health Organization recommends annual canine vaccination campaigns with at least 70% coverage to control the disease. While previous studies have considered optimal coverage of animal rabies vaccination, variation in the frequency of vaccination campaigns has not been explored. To evaluate the cost-effectiveness of rabies canine vaccination campaigns at varying coverage and frequency, we parametrized a rabies virus transmission model to two districts of northwest Tanzania, Ngorongoro (pastoral) and Serengeti (agro-pastoral). We found that optimal vaccination strategies were every 2 years, at 80% coverage in Ngorongoro and annually at 70% coverage in Serengeti. We further found that the optimality of these strategies was sensitive to the rate of rabies reintroduction from outside the district. Specifically, if a geographically coordinated campaign could reduce reintroduction, vaccination campaigns every 2 years could effectively manage rabies in both districts. Thus, coordinated campaigns may provide monetary savings in addition to public health benefits. Our results indicate that frequency and coverage of canine vaccination campaigns should be evaluated simultaneously and tailored to local canine ecology as well as to the risk of disease reintroduction from surrounding regions.
ID

New Hepatitis C Drugs are Very Costly and Unavailable to Many State Prisoners

Adam Beckman, Alyssa Bilinski, Ryan Boyko, and 6 more authors

Health Affairs, 2016

Abs DOI PDF

Prisoners bear much of the burden of the hepatitis C epidemic in the United States. Yet little is known about the scope and cost of treating hepatitis C in state prisons-particularly since the release of direct-acting antiviral medications. In the forty-one states whose departments of corrections reported data, 106,266 inmates (10 percent of their prisoners) were known to have hepatitis C on or about January 1, 2015. Only 949 (0.89 percent) of those inmates were being treated. Prices for a twelve-week course of direct-acting antivirals such as sofosbuvir and the combination drug ledipasvir/sofosbuvir varied widely as of September 30, 2015 (43,418-84,000 and 44,421-94,500, respectively). Numerous corrections departments received smaller discounts than other government agencies did. To reduce the hepatitis C epidemic, state governments should increase funding for treating infected inmates. State departments of corrections should consider collaborating with other government agencies to negotiate discounts with pharmaceutical companies and with qualified health care facilities to provide medications through the federal 340B Drug Discount Program. Helping inmates transition to providers in the community upon release can enhance the gains achieved by treating hepatitis C in prison.
WH

Geographic Poverty and Racial/Ethnic Disparities in Cervical Cancer Precursor Rates in Connecticut, 2008-2009

Linda Niccolai, Pamela Julian, Alyssa Bilinski, and 5 more authors

American Journal of Public Health, 2012

Abs DOI PDF

Objectives. We examined associations of geographic measures of poverty, race, ethnicity, and city status with rates of cervical intraepithelial neoplasia grade 2 or higher and adenocarcinoma in situ (CIN2+/AIS), known precursors to cervical cancer. Methods. We identified 3937 cases of CIN2+/AIS among women aged 20 to 39 years in statewide surveillance data from Connecticut for 2008 to 2009. We geocoded cases to census tracts and used census data to calculate overall and age-specific rates. Poisson regression determined whether rates differed by geographic measures. Results. The average annual rate of CIN2+/AIS was 417.6 per 100,000 women. Overall, higher rates of CIN2+/AIS were associated with higher levels of poverty and higher proportions of Black residents. Poverty was the strongest and most consistently associated measure. However, among women aged 20 to 24 years, we observed inverse associations between poverty and CIN2+/AIS rates. Conclusions. Disparities in cervical cancer precursors exist for poverty and race, but these effects are age dependent. This information is necessary to monitor human papillomavirus vaccine impact and target vaccination strategies.
WH

Human Papillomavirus Vaccination History Among Women With Precancerous Cervical Lesions: Disparities and barriers

Nidhi Mehta, Pamela Julian, James Meek, and 6 more authors

Obstetrics and Gynecology, 2012

Abs DOI PDF

Objective: To estimate racial, ethnic, and socioeconomic differences in HPV vaccination history among women aged 18-27 years with precancerous cervical lesions diagnosed, barriers to vaccination, and timing of vaccination in relation to the abnormal cytology result that preceded the diagnosis of the cervical lesion. Methods: High-grade cervical lesions are reportable conditions in Connecticut for public health surveillance. Telephone interviews and medical record reviews were conducted during 2008-2010 for women (n=269) identified through the surveillance registry. Results: Overall, 43% of women reported history of one or more doses of HPV vaccine. The mean age at vaccination was 22 years. Publicly insured (77%) and uninsured (85%) women were more likely than privately insured women (48%) to report no history of vaccination (P<.05). Among unvaccinated women, being unaware of HPV vaccine was reported significantly more often among Hispanics than non-Hispanics (31% compared with 13%, P=.02) and among those with public or no insurance compared with those with private insurance (26% and 36% compared with 6%, P<.05 for both). The most commonly reported barrier was lack of provider recommendation (25%). Not having talked to a provider about vaccine was reported significantly more often among those with public compared with private insurance (41% compared with 18%, P<.001). Approximately 35% of women received vaccine after an abnormal cytology result; this occurred more frequently among African American women compared with white women (80% compared with 30%, P<.01). Conclusion: Catch-up vaccination strategies should focus on provider efforts to increase timely coverage among low-income and minority women.
CI

DiD

excited

Nothing to See Here: A non-inferiority approach to parallel trends tests

Alyssa Bilinski and Laura Hatfield

Statistics in Medicine

In press

Abs DOI PDF

Difference-in-differences is a popular method for observational health policy evaluation. It relies on a causal assumption that in the absence of intervention, treatment groups’ outcomes would have evolved in parallel to those of comparison groups. Researchers frequently look for parallel trends in the pre-intervention period to bolster confidence in this assumption. The popular "parallel trends test" evaluates a null hypothesis of parallel trends and, failing to find evidence against the null, concludes that the assumption holds. This tightly controls the probability of falsely concluding that trends are not parallel but may have low power to detect non-parallel trends. When used as a screening step, it can also introduce bias in treatment effect estimates. We propose a non-inferiority/equivalence approach that tightly controls the probability of missing large violations of parallel trends measured on the scale of the treatment effect. Our framework nests several common use cases, including linear trend tests and event studies. We show that our approach may induce no or minimal bias when used as a screening step under commonly-assumed error structures, and absent violations, can offer a higher-power alternative to testing treatment effects in more flexible models. We illustrate our ideas by re-considering a study of the impact of the Affordable Care Act’s dependent coverage provision.