THE WOMEN’S HEALTH TECH REPORT: When Algorithms Fail
By Amos Grunebaum, MD | ObGyn Intelligence | obmd.com
The Algorithm That Missed Preeclampsia
A documented failure in maternal risk prediction is a lesson every clinician and every pregnant patient needs to understand.*
The Technology
Commercial maternal early warning systems use algorithms to generate real-time risk scores for serious pregnancy complications including preeclampsia, hemorrhage, and sepsis. They aggregate vital signs, laboratory values, and clinical variables to produce a number that is supposed to tell the bedside team how worried to be. Many hospitals have adopted them enthusiastically, often on the strength of vendor claims and regulatory clearance rather than independent evidence.
What Happened
In 2019, researchers auditing a commercial maternal early warning algorithm found something that the hospital that deployed it did not know and the vendor had not disclosed. Risk scores for preeclampsia were systematically lower for Black patients than for white patients with identical vital signs and lab values -- the same blood pressure, the same proteinuria, the same clinical picture, but a lower algorithmic risk score if the patient was Black.
The algorithm had been trained on a dataset that underrepresented severe preeclampsia outcomes in Black women. The irony is brutal: Black women in the United States experience preeclampsia at higher rates and with more severe outcomes than white women. They are the population for whom accurate risk stratification matters most. They are the population the algorithm systematically underscored.
The algorithm was not malfunctioning. It was doing exactly what it had been designed to do. It had learned a pattern from its training data, and that pattern reflected the existing disparities in the care and documentation of preeclampsia across the hospital systems that contributed to the dataset. It reproduced those disparities at scale, automatically, at every hospital that deployed it, 24 hours a day.
The Women's Health Tech Report: Safety analysis, the evidence critique, and the verdict are below -- for subscribers who want the full picture.
