From Lightweight CNNs to SpikeNets: Benchmarking Accuracy–Energy Tradeoffs with Pruned Spiking SqueezeNet
Loading...
Date
Journal Title
Journal ISSN
Volume Title
Publisher
Department of Computer Science and Engineering(CSE), Islamic University of Technology(IUT), Board Bazar, Gazipur-1704, Bangladesh
Abstract
SpikingNeuralNetworks(SNNs)areemergingasenergy-efficientalternativestoCon
volutional Neural Networks (CNNs) for edge-device deployment. However, SNNs
typically exhibit a performance gap compared to CNNs, and training them remains
challenging due to the non-differentiability of spiking neurons. In this work, we sys
tematically analyze lightweight CNN-to-SNNconversionpipelinesenhancedwithpa
rameters optimization and pruning. We benchmarked models on CIFAR-10, CIFAR
100, and TinyImageNet, evaluating accuracy, F1-score, parameter count, computa
tional complexity, and energy consumption. CNNs are profiled using MAC opera
tions, while SNNs are measured with Accumulate (AC) and MAC operations to cap
ture event-driven sparsity. Our results show that SNNs achieve up to 15.7× higher
energy efficiency compared to CNN baselines while maintaining comparable accu
racy. Notably, pruning the SNN SqueezeNet improved CIFAR-10 accuracy by 6% and
reduced parameters by19%comparedtotheoriginalSNNSqueezeNet. Moreover,the
pruned SNN SqueezeNet-P exhibits only a 1% lower accuracy than CNN SqueezeNet
while achieving an88.1%improvementinenergyefficiency. Thesefindingshighlight
the potential of SNNs as low-power alternatives for edge intelligence and provide a
benchmarked evaluation framework for future studies on energy-efficient neural net
works.
Description
Supervised by
Dr. Md. Hasanul Kabir,
Professor,
Mr. Sabbir Ahmed,
Assistant Professor,
Department of Computer Science and Engineering (CSE)
Islamic University of Technology (IUT)
Board Bazar, Gazipur, Bangladesh
This thesis is submitted in partial fulfillment of the requirement for the degree of Bachelor of Science in Computer Science and Engineering, 2025
Keywords
Citation
[1] J. Acharya and A. Basu, “Neuromorphic spiking neural network algorithms,” in HandbookofNeuroengineering,N.V.Thakor,Ed.Singapore:SpringerNature Singapore, 2020, pp. 1–37, isbn: 978-981-15-2848-4. doi: 10.1007/978-981 15-2848-4_44-1 [Online]. Available: https://doi.org/10.1007/978-981 15-2848-4_44-1 [2] M.Z.Alometal.,Thehistorybeganfromalexnet:Acomprehensivesurveyondeep learning approaches, 2018. arXiv: 1803.01164 [cs.CV]. [Online]. Available: https://arxiv.org/abs/1803.01164 [3] L.DengandQ.Gu,“Optimalbrainsurgeonforspikingneuralnetworks,”IEEE Transactions on Neural Networks and Learning Systems, 2021. [4] L. Deng, Q. Gu, Y. Zhao, M. Hu, and Y. Wang, “Temporal efficient training for spiking neural networks,” in International Conference on Learning Representa tions, 2022. [5] P. U. Diehl and M. Cook, “Unsupervised learning of digit recognition using spike-timing-dependentplasticity,”Frontiersincomputationalneuroscience,vol.9, p. 99, 2015. [6] W.Fang,Y.Wang,Z.Li,C.Liu, Z. Fang, and L. Ming, “Incorporating learnable surrogate gradient for direct training of spiking neural networks,” in Proceed ings of the AAAI Conference on Artificial Intelligence, vol. 35, 2021, pp. 7920 7928. [7] T.Guide, “We’re building chips that think like the brain––neuromorphic com puting in smart devices,” Tom’s Guide, 2025, Innatera’s Pulsar neuromorphic chip for edge devices. [8] B. Han, G. Srinivasan, and K. Roy, “Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural net work,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recogni tion (CVPR), 2020, pp. 13555–13564. doi: 10.1109/CVPR42600.2020.01357 47 [9] S. Han, J. Pool, J. Tran, and W. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,” in International Conference on Learning Representations (ICLR), 2016. [10] K.He,X.Zhang,S.Ren,andJ.Sun,Deepresiduallearningforimagerecognition, 2015. arXiv: 1512.03385 [cs.CV]. [Online]. Available: https://arxiv.org/ abs/1512.03385 [11] M.Horowitz,“1.1computing’senergyproblem(andwhatwecandoaboutit),” in 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), IEEE, 2014, pp. 10–14. doi: 10.1109/ISSCC.2014.6757323 [Online]. Available: https://ieeexplore.ieee.org/document/6757323 [12] A. G. Howard et al., “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” in arXiv preprint arXiv:1704.04861, 2017. [13] F.N.Iandola, S. Han, M. W.Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <0.5mbmodel size, 2016. arXiv: 1602.07360 [cs.CV]. [Online]. Available: https://arxiv. org/abs/1602.07360 [14] F.N.Iandola, S. Han, M. W.Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, Squeezenet: Alexnet-level accuracywith50xfewerparametersand<0.5MBmodel size, 2017.[Online].Available:https://openreview.net/forum?id=S1xh5sYgx [15] S.R.Kheradpisheh,E.Ghourchian,T.Masquelier,S.J.Thorpe,andT.Masque lier, “Stdp-based spiking deep neural networks for object recognition,” in arXiv preprint arXiv:1805.11675, 2018. [16] A.Krizhevsky, I. Sutskever, and G. Hinton, “Imagenet classification with deep convolutional neuralnetworks,”NeuralInformationProcessingSystems,vol.25, Jan. 2012. doi: 10.1145/3065386 [17] Y. LeCun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” Advances in neural information processing systems, vol. 2, pp. 598–605, 1990. [18] E.Ledinauskas,J.Ruseckas, A.Jurš˙ enas, and G. Buračas, Trainingdeepspiking neuralnetworks,2020.arXiv:2006.04436[cs.NE].[Online].Available:https: //arxiv.org/abs/2006.04436 [19] H.Li,A.Kadav,I.Durdanovic,H.Samet,andH.P.Graf,“Pruningfiltersforeffi cientconvnets,”inInternationalConferenceonLearningRepresentations(ICLR), 2017. 48 [20] Z. Liu, J. Li, Z. Shen, G. Huang, S. Yan, and C. Zhang, “Learning efficient con volutional networks through networkslimming,”inProceedingsoftheIEEEIn ternational Conference on Computer Vision, 2017, pp. 2736–2744. [21] P.Merolla et al., A million spiking-neuron integrated circuit with a scalable com munication network and interface, en, 2014. [Online]. Available: https://www. semanticscholar.org/paper/A-million-spiking-neuron-integrated circuit-with-a-Merolla-Arthur/680a38e8f025685b192e9e0cf755c6b664963551 [22] P. Molchanov, A. Mallya, S. Tyree, I. Frosio, and J. Kautz, “Importance estima tion for neural network pruning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2019, pp. 11264 11272. [23] P.Molchanov,S.Tyree, T. Karras, T. Aila, and J. Kautz, “Pruning convolutional neural networks for resource efficient inference,” in International Conference on Learning Representations (ICLR), 2017. [24] E.O.Neftci,H.Mostafa,andF.Zenke,“SurrogateGradientLearninginSpiking Neural Networks,” arXiv preprint arXiv:1901.09948, 2019. [Online]. Available: https://arxiv.org/abs/1901.09948 [25] E. O. Neftci, H. Mostafa, and F. Zenke, “Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks,” IEEE Signal Processing Magazine, vol. 36, no. 6, pp. 51–63, 2019. doi: 10.1109/MSP.2019.2931595 [26] H. C. V. Ngu and K. M. Lee, “Effective conversion of a convolutional neural network into a spiking neural network for image recognition tasks,” Applied Sciences, vol. 12, no. 11, 2022, issn: 2076-3417. doi: 10.3390/app12115749 [Online]. Available: https://www.mdpi.com/2076-3417/12/11/5749 [27] M. K. Pasupuleti, “Spiking neural networks for energy-efficient edge intelli gence,” IJAIRI, 2025. [28] K. Roy, A. Jaiswal, and P. Panda, “Towards spike-based machine intelligence with neuromorphic computing,” in Nature, vol. 575, Nature Publishing Group, 2019, pp. 607–617. [29] B. Rueckauer, I.-A. Lungu, Y. Hu, M. Pfeiffer, and S.-C. Liu, “Conversion of continuous-valued deep networks to efficient event-driven networks for image classification,” Frontiers in Neuroscience, vol. 11, p. 682, 2017. doi: 10.3389/ fnins.2017.00682 [Online]. Available: https://www.frontiersin.org/ articles/10.3389/fnins.2017.00682/full 49 [30] A.Sengupta,Y.Ye,R.Wang,C.Liu,andK.Roy,“Goingdeeperinspikingneural networks: Vgg and residual architectures,” Frontiers in Neuroscience, vol. 13, p. 153, 2019. doi: 10.3389/fnins.2019.00153 [Online]. Available: https: //www.frontiersin.org/articles/10.3389/fnins.2019.00153/full [31] J.Shen,Q.Xu,J.K.Liu,Y.Wang,G.Pan,andH.Tang,Esl-snns:Anevolutionary structure learning strategy for spiking neural networks, 2023. arXiv: 2306.03693 [cs.NE]. [Online]. Available: https://arxiv.org/abs/2306.03693 [32] J. Shen, Q. Xu, G. Pan, and B. Chen, Improving the sparse structure learning of spiking neural networks from the view of compression efficiency, 2025. arXiv: 2502.13572 [cs.HC]. [Online]. Available: https://arxiv.org/abs/2502. 13572 [33] X. Shi, J. Ding, Z. Hao, and Z. Yu, “Towards energy efficient spiking neural networks: An unstructured pruning framework,” in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https:// openreview.net/forum?id=eoSeaK4QJo [34] K.SimonyanandA.Zisserman,Verydeepconvolutionalnetworksforlarge-scale imagerecognition,2015.arXiv:1409.1556[cs.CV].[Online].Available:https: //arxiv.org/abs/1409.1556 [35] K.SimonyanandA.Zisserman,Verydeepconvolutionalnetworksforlarge-scale imagerecognition,2015.arXiv:1409.1556[cs.CV].[Online].Available:https: //arxiv.org/abs/1409.1556 [36] M. Tan and Q. V. Le, “Mixnet: Mixed depthwise convolutional kernels,” arXiv preprint arXiv:1907.09595, 2019. [37] M.Tanetal.,“Mnasnet:Platform-awareneuralarchitecturesearchformobile,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 2820–2828. [38] Unnamed, “Spiking neural networks and their applications: A review,” Brain Sciences, 2022. [39] H.Wang,B.Kim,J.Xie, and Z. Han, “Energy drain of the object detection pro cessing pipeline for mobile devices: Analysis and implications,” IEEE Transac tions on Green Communications and Networking, vol. 5, no. 1, pp. 41–60, 2021. doi: 10.1109/TGCN.2020.3041666 [40] D.Wuetal.,“Buildanenergy-efficient,accuratespikingneuralnetwork,”Fron tiers in Neuroscience, 2022. 50 [41] D. Xu et al., Edge intelligence: Architectures, challenges, and applications, 2020. arXiv: 2003.12172 [cs.NI]. [Online]. Available: https://arxiv.org/abs/ 2003.12172 [42] T.-J. Yang, Y.-H. Chen, and V. Sze, Designing energy-efficient convolutional neu ral networks using energy-aware pruning, 2017. arXiv: 1611.05128 [cs.CV]. [Online]. Available: https://arxiv.org/abs/1611.05128 [43] T.-J. Yang, A. Howard, A. Smola, B. Singh, S. Chien, and V. Sze, “Designing energy-efficient convolutional neural networks using energy-aware pruning,” in ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition, 2017, pp. 5687–5695. [44] Y.-X. Zeng, J.-W. Hsieh, X. Li, and M.-C. Chang, Mixnet: Toward accurate de tection of challenging scene text in the wild, 2023. arXiv: 2308.12817 [cs.CV]. [Online]. Available: https://arxiv.org/abs/2308.12817 [45] X. Zhang, X. Zhou, M. Lin, and J. Sun, Shufflenet: An extremely efficient convo lutional neural network for mobile devices, 2017. arXiv: 1707.01083 [cs.CV]. [Online]. Available: https://arxiv.org/abs/1707.01083 [46] X.Zhang,X.Zhou,M.Lin,andJ.Sun,“Shufflenet: Anextremely efficient con volutional neural network for mobile devices,” in Proceedings of CVPR, 2018, pp. 6848–6856. [47] A. Zheng et al., “Going deeper with directly-trained larger spiking neural net works,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, 2021, pp. 6677–6685.