Explainable AI for Enhanced Brain Tumor Segmentation in Multi-modal MRI Scans
Loading...
Date
Journal Title
Journal ISSN
Volume Title
Publisher
Department of Computer Science and Engineering(CSE), Islamic University of Technology(IUT), Board Bazar, Gazipur-1704, Bangladesh
Abstract
In the recent years, Explainable AI (XAI) and convolutionl neural networks (CNNs)
have achieved significant breakthrough in the medical image analysis. This thesis
investigates the application of deep learning models for the segmentation of brain
tumors in multi-modal MRI scans, incorporating Explainable AI (XAI) methodologies to enhance interpretability. Using the BraTS Challenge 2020 dataset, this study
implements and compares various deep learning architectures, including U-Net, UnetR, swin U-Net to segment brain tumors effectively. The research addresses critical
challenges such as the complexity of MRI image preprocessing, limited availability of
annotated data, and high computational requirements. To ensure robust and reliable
model performance, advanced loss functions and data augmentation techniques including flipping, rotation, scaling, brightness adjustment, and elastic deformation are
used. The core contribution of this work lies in the integration of XAI techniques like
GradCAM, Feature Ablation, and Kernel SHAP to elucidate the decision-making processes of the neural networks. This approach aims to bridge the gap between model
predictions and clinical interpretability, providing heatmaps and visual explanations
that highlight the most influential regions in the MRI scans. The results demonstrate the efficacy of the proposed models in accurately segmenting tumor regions,
with the XAI methods offering valuable insights into model behavior and reliability.
This research contributes to the development of more transparent and trustworthy AI
systems in brain tumor segmentation. The findings hold significant implications for
enhancing clinical decision-making and personalized treatment planning for brain
tumor patients.
Description
Supervised by
Dr Md Azam Hossain,
Associate Professor,
Department of Computer Science and Engineering (CSE)
Islamic University of Technology (IUT)
Board Bazar, Gazipur, Bangladesh
This thesis is submitted in partial fulfillment of the requirement for the degree of Bachelor of Science in Computer Science and Engineering, 2024
Citation
[1] A. Abdollahi, B. Pradhan, and A. Alamri, “Vnet: An end-to-end fully convolu tional neural network for road extraction from high-resolution remote sensing data,” Ieee Access, vol. 8, pp. 179 424–179 436, 2020. [2] M. Adewole, J. D. Rudie, A. Gbdamosi, et al., “The brain tumor segmentation (brats) challenge 2023: Glioma segmentation in sub-saharan africa patient pop ulation (brats-africa),” ArXiv, 2023. [3] S. Bakas, H. Akbari, A. Sotiras, et al., “Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features,” Scien tific data, vol. 4, no. 1, pp. 1–13, 2017. [4] S. Bakas, M. Reyes, A. Jakab, et al., “Identifying the best machine learning algo rithms for brain tumor segmentation, progression assessment, and overall sur vival prediction in the brats challenge,” arXiv preprint arXiv:1811.02629, 2018. [5] S. Cai, Y. Tian, H. Lui, H. Zeng, Y. Wu, and G. Chen, “Dense-unet: A novel mul tiphoton in vivo cellular image segmentation model based on a convolutional neural network,” Quantitative imaging in medicine and surgery, vol. 10, no. 6, p. 1275, 2020. [6] H. Cao, Y. Wang, J. Chen, et al., Swin-unet: Unet-like pure transformer for med ical image segmentation, 2021. arXiv: 2105.05537 [eess.IV]. [7] M. J. Cardoso, W. Li, R. Brown, et al., Monai: An open-source framework for deep learning in healthcare, 2022. arXiv: 2211.02701 [cs.LG]. [Online]. Available: https://arxiv.org/abs/2211.02701. [8] C. S. Center,Cancer statistics. [Online]. Available: https://cancerstatisticscenter. cancer.org/. [9] S. L. Chau, R. Hu, J. Gonzalez, and D. Sejdinovic, Rkhs-shap: Shapley values for kernel methods, 2022. arXiv: 2110.09167 [stat.ML]. [Online]. Available: https://arxiv.org/abs/2110.09167. 42 [10] Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, 3d u-net: Learning dense volumetric segmentation from sparse annotation, 2016. arXiv: 1606.06650 [cs.CV]. [Online]. Available: https://arxiv.org/abs/1606. 06650. [11] M. D. Cirillo, D. Abramian, and A. Eklund, What is the best data augmentation for 3d brain tumor segmentation? 2021. arXiv: 2010.13372 [eess.IV]. [12] I. Csiszar, “𝐼-Divergence Geometry of Probability Distributions and Minimiza tion Problems,” The Annals of Probability, vol. 3, no. 1, pp. 146–158, 1975. doi: 10.1214/aop/1176996454. [Online]. Available: https://doi.org/10.1214/ aop/1176996454. [13] F. I. Diakogiannis, F. Waldner, P. Caccetta, and C. Wu, “Resunet-a: A deep learn ing framework for semantic segmentation of remotely sensed data,” ISPRS Jour nal of Photogrammetry and Remote Sensing, vol. 162, pp. 94–114, Apr. 2020, issn: 0924-2716. doi: 10.1016/j.isprsjprs.2020.01.013. [Online]. Avail able: http://dx.doi.org/10.1016/j.isprsjprs.2020.01.013. [14] L. R. Dice, “Measures of the amount of ecologic association between species,” Ecology, vol. 26, no. 3, pp. 297–302, 1945. [15] F. K. Došilović, M. Brčić, and N. Hlupić, “Explainable artificial intelligence: A survey,” in 2018 41st International convention on information and communica tion technology, electronics and microelectronics (MIPRO), IEEE, 2018, pp. 0210– 0215. [16] A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., An image is worth 16x16 words: Transformers for image recognition at scale, 2021. arXiv: 2010.11929 [cs.CV]. [Online]. Available: https://arxiv.org/abs/2010.11929. [17] M. Drozdzal, E. Vorontsov, G. Chartrand, S. Kadoury, and C. Pal, The impor tance of skip connections in biomedical image segmentation, 2016. arXiv: 1608. 04117 [cs.CV]. [18] D. Duque-Arias, S. Velasco-Forero, J.-E. Deschaud, et al., “On power Jaccard losses for semantic segmentation,” in VISAPP 2021 : 16th International Con ference on Computer Vision Theory and Applications, Vienne (on line), Austria, Feb. 2021. [Online]. Available: https://hal.science/hal-03139997. [19] D. Fourure, R. Emonet, E. Fromont, D. Muselet, A. Tremeau, and C. Wolf, Resid ual conv-deconv grid network for semantic segmentation, 2017. arXiv: 1707 . 07958 [cs.CV]. [20] G. Gerig, M. Jomier, and M. Chakos, “Valmet: A new validation tool for assess ing and improving 3d object segmentation,” in Medical Image Computing and 43 Computer-Assisted Intervention–MICCAI 2001: 4th International Conference Utrecht, The Netherlands, October 14–17, 2001 Proceedings 4, Springer, 2001, pp. 516– 523. [21] A. Hatamizadeh, Y. Tang, V. Nath, et al., Unetr: Transformers for 3d medical image segmentation, 2021. arXiv: 2103.10504 [eess.IV]. [Online]. Available: https://arxiv.org/abs/2103.10504. [22] G. Huang, Z. Liu, and K. Weinberger, “Densely connected convolutional net works,” p. 12, Aug. 2016. [23] H. Huang, L. Lin, R. Tong, et al., “Unet 3+: A full-scale connected unet for med ical image segmentation,” in ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, 2020, pp. 1055–1059. [24] H. Kervadec, J. Bouchtiba, C. Desrosiers, E. Granger, J. Dolz, and I. B. Ayed, “Boundary loss for highly unbalanced segmentation,” in International confer ence on medical imaging with deep learning, PMLR, 2019, pp. 285–296. [25] B. Kim, M. Wattenberg, J. Gilmer, et al., “Interpretability beyond feature attri bution: Quantitative testing with concept activation vectors (TCAV),” in Pro ceedings of the 35th International Conference on Machine Learning, J. Dy and A. Krause, Eds., ser. Proceedings of Machine Learning Research, vol. 80, PMLR, Oct. 2018, pp. 2668–2677. [Online]. Available: https://proceedings.mlr. press/v80/kim18d.html. [26] X. Li, H. Chen, X. Qi, Q. Dou, C.-W. Fu, and P. A. Heng, H-denseunet: Hybrid densely connected unet for liver and tumor segmentation from ct volumes, 2018. arXiv: 1709.07330 [cs.CV]. [27] Z. Liu, Y. Lin, Y. Cao, et al., Swin transformer: Hierarchical vision transformer using shifted windows, 2021. arXiv: 2103.14030 [cs.CV]. [Online]. Available: https://arxiv.org/abs/2103.14030. [28] Z. Liu, L. Tong, L. Chen, et al., “Deep learning based brain tumor segmentation: A survey,” Complex & intelligent systems, vol. 9, no. 1, pp. 1001–1026, 2023. [29] J. Long, E. Shelhamer, and T. Darrell, Fully convolutional networks for semantic segmentation, 2015. arXiv: 1411.4038 [cs.CV]. [30] S. M. Lundberg and S. Lee, “A unified approach to interpreting model predic tions,” CoRR, vol. abs/1705.07874, 2017. arXiv: 1705 . 07874. [Online]. Avail able: http://arxiv.org/abs/1705.07874. 44 [31] B. H. Menze, A. Jakab, S. Bauer, et al., “The multimodal brain tumor image seg mentation benchmark (brats),” IEEE transactions on medical imaging, vol. 34, no. 10, pp. 1993–2024, 2014. [32] Y.-H. Nai, B. W. Teo, N. L. Tan, et al., “Comparison of metrics for the evaluation of medical segmentations using prostate mri dataset,” Computers in biology and medicine, vol. 134, p. 104 497, 2021. [33] M. T. Ribeiro, S. Singh, and C. Guestrin, “"why should I trust you?": Explaining the predictions of any classifier,” CoRR, vol. abs/1602.04938, 2016. arXiv: 1602. 04938. [Online]. Available: http://arxiv.org/abs/1602.04938. [34] O. Ronneberger et al., “Unet: Convolutional networks for biomedical image seg mentation,” Journal of Biomedical Imaging, 2015. [35] O. Ronneberger, P. Fischer, and T. Brox, U-net: Convolutional networks for biomed ical image segmentation, 2015. arXiv: 1505.04597 [cs.CV]. [Online]. Available: https://arxiv.org/abs/1505.04597. [36] S. S. M. Salehi, D. Erdogmus, and A. Gholipour, “Tversky loss function for image segmentation using 3d fully convolutional deep networks,” in Machine Learning in Medical Imaging, Q. Wang, Y. Shi, H.-I. Suk, and K. Suzuki, Eds., Cham: Springer International Publishing, 2017, pp. 379–387. [37] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based local ization,” International Journal of Computer Vision, vol. 128, no. 2, pp. 336–359, Oct. 2019, issn: 1573-1405. doi: 10.1007/s11263- 019- 01228- 7. [Online]. Available: http://dx.doi.org/10.1007/s11263-019-01228-7. [38] R. R. Selvaraju, A. Das, R. Vedantam, M. Cogswell, D. Parikh, and D. Batra, “Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization,” CoRR, vol. abs/1610.02391, 2016. arXiv: 1610. 02391. [Online]. Available: http://arxiv.org/abs/1610.02391. [39] M. Sharma and N. Miglani, “Automated brain tumor segmentation in mri im ages using deep learning: Overview, challenges and future,” Deep learning tech niques for biomedical and health informatics, pp. 347–383, 2020. [40] A. A. Taha and A. Hanbury, “Metrics for evaluating 3d medical image segmen tation: Analysis, selection, and tool,” BMC medical imaging, vol. 15, pp. 1–28, 2015. [41] H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou, Train ing data-efficient image transformers & distillation through attention, 2021. arXiv: 45 2012.12877 [cs.CV]. [Online]. Available: https://arxiv.org/abs/2012. 12877. [42] B. H. van der Velden, H. J. Kuijf, K. G. Gilhuijs, and M. A. Viergever, “Explain able artificial intelligence (xai) in deep learning-based medical image analysis,” Medical Image Analysis, vol. 79, p. 102 470, 2022, issn: 1361-8415. doi: https: //doi.org/10.1016/j.media.2022.102470. [Online]. Available: https: //www.sciencedirect.com/science/article/pii/S1361841522001177. [43] B. Wang, F. Deng, P. Jiang, S. Wang, X. Han, and H. Zheng, “Witunet: A u shaped architecture integrating cnn and transformer for improved feature align ment and local information fusion,” arXiv preprint arXiv:2404.09533, 2024. [44] Y. Xie, J. Yan, L. Kang, Y. Guo, J. Zhang, and X. Luan, “Fct: Fusing cnn and transformer for scene classification,” International Journal of Multimedia In formation Retrieval, vol. 11, no. 4, pp. 611–618, 2022. [45] A. Zaki and K. Ali, “Segmentation of human brain gliomas tumour images us ing u-net architecture with transfer learning,” Diyala Journal of Engineering Sci ences, vol. 15, pp. 17–29, Mar. 2022. doi: 10.24237/djes.2022.15102. [46] Y. J. Zhang, “A review of recent evaluation methods for image segmentation,” in Proceedings of the sixth international symposium on signal processing and its applications (Cat. No. 01EX467), IEEE, vol. 1, 2001, pp. 148–151. [47] Z. Zhang and M. Sabuncu, “Generalized cross entropy loss for training deep neural networks with noisy labels,” Advances in neural information processing systems, vol. 31, 2018. [48] Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: A nested u-net architecture for medical image segmentation,” in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4, Springer, 2018, pp. 3–11. [49] K. J. Zülch, Brain tumors: their biology and pathology. Springer-Verlag, 2013.