Project Team
Students
Alan Baxley
Computer Science
Penn State Harrisburg
Faculty Mentors
Dr. Truong Tran
Penn State Harrisburg
Computer Science
Project
Project Video
Project Abstract
Interpretability of machine learning algorithms in medical image analysis has been a persistent challenge, hindering widespread clinician adoption. We propose a hybrid CNN vector library model to map the training set to the vector space most relevant to the model output, providing post-hoc interpretability by retrieving the k-nearest training samples to an inference image. Using the Figshare, SARTAJ, and BR35H brain tumor datasets, we implemented this model with two CNNs: YOLOv8m for state-of-the-art object detection and ResNet-50 for multi-class classification. The FAISS (Facebook AI Similarity Search) vector library was used to store vectorized training features and perform similarity comparisons with inference images. We compared our method to an intrinsically interpretable case-based model (ProtoPNet) using a ResNet-50 backbone by examining the features. Analysis over 26 test samples revealed Pearson and Spearman correlation means of 0.3402 and 0.3621, respectively. The first canonical correlation from Canonical Correlation Analysis (CCA) after dimension reduction with Principal Component Analysis (PCA) to 50 components was 0.9467. These results indicate that while Pearson and Spearman correlations are modest, the high canonical correlation suggests our approach captures significant underlying relationships. This implies that our CNN vector library hybrid approach may provide some degree of interpretability while preserving the underlying CNN’s performance. However, further research is required to conclusively determine the extent of interpretability offered.
Evaluate this Project
Use this form link to provide feedback to the presenters, and add your project evaluation for award(s) consideration.