Pediatric asthma is the most prevalent chronic childhood illness, afflicting about 6.2 million children in the United States. However, asthma could be better managed by identifying and avoiding triggers, educating about medications and proper disease management strategies.
Parkland Center for Clinical Innovation (PCCI) has been working with the Parkland Community Health Plan (PCHP) for the past four years to help them with timely identification of high risk asthma members in the pediatric population. We have deployed a logistic regression model on claims data that is the foundation for an innovative care redesign and text based patient engagement program that has show consistent cost savings and clinical outcomes improvement for PCHP members. You can learn more about this program here.
Thus, this issue for us is not merely an academic exercise. We realize that a predictive model with improved statistical performance can yield even better health outcomes for these families and improved cost savings for PCHP. Keeping that in mind, we have regularly looked for ways to continue to improve our model that is not just limited to retraining the model but also finding ways to improve data quality and bringing in additional data sets for workflow improvement. More recently, we decided to retrain the model using deep learning techniques — more specifically using an ANN.
Our deep learning model produced an AUC of 0.845 which was only slightly better than the AUC of the current logistic regression model at 0.842.
You can read about details of our work at arXiv here.
Healthcare has high expectations for the level of transparency from machine learning algorithms deployed in a clinical setting. Deep learning models with their relative lack of transparency are not always the best contender for those situations. However, if the improvement in performance of the model for relevant statistical measures is significant enough, then there can be a strong case for deployment of a deep learning model over say, a logistic regression model.
However, in this case, with the exact same data set and initial feature list, ANN model only produced slightly higher statistical classification power than the Lasso logistic regression. This is consistent with the results from some other published research that has compared logistic regression and ANN models in multiple medical data classification tasks. This study further confirmed that the Lasso logistic regression model developed by PCCI in 2015 could produce desirable statistical performance that is non-inferior to deep learning models which are more difficult to interpret. And in order for our predictive models to be deployed and effectively improve patient care, we need to work closely with clinicians to explain predictions in comprehensive and interpretable formats to build trust and transparency with stakeholders.
For future studies, blender algorithms would be tested against other singular models to achieve better statistical performances. We would explore the temporal relationships in claims data using other deep learning models, like Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM).
