Leveraging Knowledge Distillation for Efficient on-device Deployment of Deep Learning Models in Medical Imaging

Abstract

Deep learning (DL) is revolutionizing medical imaging with applications in disease classification. Chest radiography usingmobile X-ray systems is deemed as a key approach for screening COVID-19 patients. However, training accurate DLmodels usually involves optimizing millions of model parameters and deploying these models on portable devices can poseoperational challenges because of their size. Knowledge distillation (KD) is a model compression method in whichknowledge is transferred from a large model (or ensemble of models) to a smaller one. We demonstrate the utilization ofthe KD framework for creating a compact, yet very accurate, disease classification model on Chest X-Ray (CXR) images.