Lightweight Deep Learning Model for Skin Cancer Detection using Knowledge Distillation
Loading...
Date
Journal Title
Journal ISSN
Volume Title
Publisher
Department of Electrical and Electronic Engineering (EEE), Islamic University of Technology(IUT), Board Bazar, Gazipur-1704, Bangladesh
Abstract
Skin cancer is one of the most common forms of cancer worldwide, and its early detection
plays a crucial role in improving survival rates. Traditional diagnostic practices rely on visual
inspection and biopsy, which are time-consuming and often inaccessible in resource-limited
regions. This thesis presents a lightweight deep learning model for skin cancer detection that
combines knowledge distillation and GAN-based data augmentation to achieve both high
accuracy and computational efficiency.
The proposed system utilizes the HAM10000 dataset, balancing its class distribution with
synthetic image generation. Two high-capacity teacher models, ResNet50 and DenseNet161,
were trained and their knowledge distilled into a compact student CNN. The student model
contains only 2.8 million parameters, almost 90% fewer than the teacher networks, while
retaining competitive classification performance.
Experimental results demonstrate an accuracy of 85.5% and an AUC of 0.9834, showing
strong potential for deployment on mobile and edge devices. This research contributes to
making advanced diagnostic tools more accessible, particularly in regions with limited
medical resources, bridging the gap between artificial intelligence research and real-world
healthcare applications.
Description
Supervised by
Mr. Quazi Nafees Ul Islam,
Assistant Professor,
Department of Electrical and Electronic Engineering (EEE)
Islamic University of Technology (IUT)
Board Bazar, Gazipur, Bangladesh
This thesis is submitted in partial fulfillment of the requirement for the degree of Bachelor of Science in Electrical and Electronic Engineering, 2025
Keywords
Citation
[1] R. Yadav and A. Bhat, "Skin Cancer Detection using Knowledge Distillation," in Proceedings of the 3rd International Conference for Innovation in Technology (INOCON), 2024, pp. 1504–1508. [2] M. R. Kabir, R. H. Borshon, M. K. Wasi, R. M. Sultan, and A. Hossain, "Skin cancer detection using lightweight model souping and ensembling knowledge distillation for memory constrained devices," Intelligence-Based Medicine, vol. 10, p. 100043, 2024. [3] Y. Wang, Y. Wang, J. Cai, T. K. Lee, C. Miao, and Z. J. Wang, "SSD-KD: A self-supervised diverse knowledge distillation method for lightweight skin lesion classification using dermoscopic images," Medical Image Analysis, vol. 84, p. 102693, 2023. [4] K. Deng, H. Zhang, Y. Liu, and L. Wang, "An Improved Lightweight Segmentation Neural Network for Dermoscopic Lesion Images Based on Knowledge Distillation," in Proceedings of the 2024 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), 2024, pp. 1–6. [5] M. Bhargavi, S. Settu, and S. K. Sahu, "Ensemble Learning for Skin Lesion Classification: A Robust Approach for Improved Diagnostic Accuracy (ELSLC)," in Proceedings of the 2023 International Conference on Intelligent Sustainable Systems (ICISS), 2023, pp. 1–6. [6] U. Saha, I. U. Ahamed, M. A. Imran, I. U. Ahamed, A. A. Hossain, and U. D. Gupta, "YOLOv8-Based Deep Learning Approach for Real-Time Skin Lesion Classification Using the HAM10000 Dataset," in Proceedings of the 2024 IEEE International Conference on E health Networking, Application & Services (HealthCom), Nara, Japan, Nov. 2024, pp. 1–4. [7] Khan NH, Mir M, Qian L, Baloch M, Ali Khan MF, ur Rehman A, Ngowi EE,Wu D-D, Ji X-Y. Skin cancer biology and barriers to treatment: Recent appli-cations of polymeric micro/nanostructures. J Adv Res 2022;36:223–47 [8] Hussein MR. Ultraviolet radiation and skin cancer: molecular mechanisms. JCutan Pathol 2005;32(3):191–205. [9] Luo N, Zhong X, Su L, Cheng Z, Ma W, Hao P. Artificial intelligence-assisted dermatology diagnosis: from unimodal to multimodal. Comput Biol Med2023;107413. [10] Stafford H, Buell J, Chiang E, Ramesh U, Migden M, Nagarajan P, Amit M,Yaniv D. Non-melanoma skin cancer detection in the age of advanced technology:A review. Cancers 2023;15. [11] N. Khasanah and M. N. Winnarto, “Application of deep learning with resnet50 for early detection of melanoma skin cancer,” Journal Medical Informatics Technology, pp. 16–20, Mar. 2024. 81 [12] K. Mridha, M. M. Uddin, J. Shin, S. Khadka, and M. F. Mridha, “An interpretable skin cancer classification using optimized convolutional neural network for a smart healthcare system,” IEEE Access, vol. 11, pp. 41 003–41 018, 2023. [13] A. E. Minarno, A. Lusianti, Y. Azhar, and H. Wibowo, “Classification of skin cancer images using convolutional neural network with resnet50 pre-trained model,” JOIV: International Journal on Informatics Visualization, vol. 8, no. 4, p. 2013, Dec. 2024. [14] Q. Su, H. N. A. Hamed, M. A. Isa, X. Hao, and X. Dai, “A ganbased data augmentation method for imbalanced multi-class skin lesion classification,” IEEE Access, vol. 12, pp. 16 498–16 513, 2024. [15] N. Islam, M. Hasib, F. A. Joti, A. Karim, and S. Azam, “Leveraging knowledge distillation for lightweight skin cancer classification: Balancing accuracy and computational efficiency.” [16] N. Rajasekhar Reddy et al., “Enhancing skin cancer detection through an ai-powered framework by integrating african vulture optimization with gan-based bi-lstm architecture,” www.ijacsa.thesai.org. [17] B. Ahmad, S. Jun, V. Palade, Q. You, L. Mao, and M. Zhongjie, “Improving skin cancer classification using heavy-tailed student tdistribution in generative adversarial networks (ted gan),” Diagnostics, vol. 11, no. 11, Nov. 2021. [18] P. Tschandl, C. Rosendahl, and H. Kittler, “The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,” Sci Data, vol. 5, no. 1, p. 180161, Aug. 2018. [19] K. Behara, E. Bhero, and J. T. Agee, “Skin lesion synthesis and classification using an improved dcgan classifier,” Diagnostics, vol. 13, no. 16, Aug. 2023. [20] A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” Jan. 2016. [21] M. Frid-Adar, I. Diamant, E. Klang, M. Amitai, J. Goldberger, and H. Greenspan, “Gan based synthetic medical image augmentation for increased cnn performance in liver lesion classification,” Neurocomputing, vol. 321, pp. 321–331, Dec. 2018. [22] Y. Mousa, R. Taha, R. Kaur, and S. Afifi, “Melanoma classification using deep learning,” 2024, pp. 259–272. [23] A. K. Gairola, V. Kumar, and A. K. Sahoo, “Exploring multiple deep learning models for skin cancer classification,” in 2022 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS), Nov. 2022, pp. 805–810.