Bias in Clinical Risk Prediction Models: Challenges in Application to Observational Health Data

Abstract

With the increasing use of machine learning and AI in healthcare, ensuring the fairness of algorithms is paramount to prevent health disparities and inequities from being reproduced in the algorithm-guided medical and policy decisions. In this work we investigate algorithmic bias in clinical prediction models and discuss challenges in analyzing bias in observational health data. We show that potential disparities in treatment opportunity exist between races in the data for patients with opioid use disorder, and that the direction of bias favoring one race over the other depends on the choice of outcome label or fairness metric. We further demonstrate how debiasing algorithms can effectively mitigate the apparent bias in most experimental settings. This study exemplifies the importance of thorough bias assessment in prediction tasks based on healthcare data.