Advanced Classification of Agricultural Plant Insects Using Deep Learning and Explainability

Tu Hoang Vo, Nhon Thien Nguyen, Kheo Mui Chau, Phuc Tien Pham, Huan Le Lam, Vuong Phuc Nguyen, Hieu Trung Nguyen, Phuong Tan Lam

Abstract


This paper investigates the effectiveness of six pre-trained deep learning models to classify images of agricultural plant insects. We utilized the BAU-Insectv2 dataset, which includes images from nine classes. Aphids, Armyworm, Beetle, Bollworm, Grasshopper, Mites, Mosquito, Sawfly, and Stem borer. The models, namely Xception, MobileNetV2, ResNet50, EfficientNetV2B3, ResNet101, and DenseNet121, are fine-tuned by transfer learning from ImageNet. This approach significantly reduces training time while improving classification accuracy. Our experiments reveal that each model reliably distinguishes between insect species even when faced with varying lighting conditions and diverse viewpoints. To further clarify how these models make predictions, we employ Gradient-weighted Class Activation Mapping (Grad-CAM) to highlight critical regions in the images. The results demonstrate that each model focuses on unique biological features and offers clear explanations for its decisions. The research results contribute to demonstrating the potential of pre-trained deep learning architectures for agricultural monitoring and pest management, paving the way for promising future applications.

Keywords


Agricultural Insect; Explainable AI; XAI; Grad-CAM; Transfer Learning; Pre-trained model

References


REFERENCES

S. Savary, A. Ficke, J.-N. Aubertot, and C. Hollier, “Crop losses due to diseases and their implications for global food production losses and food security,” Food security, vol. 4, no. 4, pp.519–537, 2012

C. A. Deutsch, J. J. Tewksbury, M. Tigchelaar, D. S. Battisti, S. C. Merrill, R. B. Huey, and R. L. Naylor, “Increase in crop losses to insect pests in a warming climate,” Science, vol. 361, no. 6405, pp. 916–919, 2018.

S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Transactions on knowledge and data engineering, vol. 22, no. 10, pp. 1345–1359, 2009.

R. Ribani and M. Marengoni, “A survey of transfer learning for convolutional neural networks,” in 2019 32nd SIBGRAPI conference on graphics, patterns and images tutorials (SIBGRAPI-T). IEEE, 2019, pp. 47–57.

C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, and C. Liu, “A survey on deep transfer learning,” in Artificial Neural Networks and Machine Learning–ICANN 2018: 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018, Proceedings, Part III 27. Springer, 2018, pp. 270–279.

H. Liang, W. Fu, and F. Yi, “A survey of recent advances in transfer learning,” in 2019 IEEE 19th international conference on communication technology (ICCT). IEEE, 2019, pp. 1516–1523.

A. W. Salehi, S. Khan, G. Gupta, B. I. Alabduallah, A. Almjally, H. Alsolai, T. Siddiqui, and A. Mellit, “A study of cnn and transfer learning in medical imaging: Advantages, challenges, future scope,” Sustainability, vol. 15, no. 7, p. 5930, 2023.

S. Niu, Y. Liu, J. Wang, and H. Song, “A decade survey of transfer learning (2010–2020),” IEEE Transactions on Artificial Intelligence, vol. 1, no. 2, pp. 151–166, 2020..

J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, “How transferable are features in deep neural networks?” Advances in neural information processing systems, vol. 27, 2014.

A. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications. sl, sn,” arXiv preprint arXiv:1704.04861, 2017.

F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1251–1258.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.

F. Ritter, T. Boskamp, A. Homeyer, H. Laue, M. Schwier, F. Link, and H.-O. Peitgen, “Medical image analysis,” IEEE pulse, vol. 2, no. 6, pp. 60–70, 2011.

A. P. Dhawan, Medical image analysis. John Wiley & Sons, 2011.

D. Shen, G. Wu, and H.-I. Suk, “Deep learning in medical image analysis,” Annual review of biomedical engineering, vol. 19, no. 1, pp. 221– 248, 2017.

H.-P. Chan, R. K. Samala, L. M. Hadjiiski, and C. Zhou, “Deep learning in medical image analysis,” Deep learning in medical image analysis: challenges and applications, pp. 3–21, 2020.

H.-T. Vo, N. N. Thien, and K. C. Mui, “Tomato disease recognition: Advancing accuracy through xception and bilinear pooling fusion,” International Journal of Advanced Computer Science and Applications, vol. 14, no. 8, 2023.

K. Munir, H. Elahi, A. Ayub, F. Frezza, and A. Rizzi, “Cancer diagnosis using deep learning: a bibliographic review,” Cancers, vol. 11, no. 9, p. 1235, 2019.

S. Khan, N. Islam, Z. Jan, I. U. Din, and J. J. C. Rodrigues, “A novel deep learning based framework for the detection and classification of breast cancer using transfer learning,” Pattern Recognition Letters, vol. 125, pp. 1–6, 2019.

S. Guan and M. Loew, “Breast cancer detection using transfer learning in convolutional neural networks,” in 2017 IEEE applied imagery pattern recognition workshop (AIPR). IEEE, 2017, pp.1–8.

R. Fakoor, F. Ladhak, A. Nazi, and M. Huber, “Using deep learning to enhance cancer diagnosis and classification,” in Proceedings of the international conference on machine learning, vol. 28. ACM New York, NY, USA, 2013, pp. 3937–3949.

P. L. Vidal, J. de Moura, J. Novo, and M. Ortega, “Multi-stage transfer learning for lung segmentation using portable x-ray devices for patients with covid-19,” Expert Systems with Applications, vol. 173, p. 114677, 2021.

A. Z. Abidin, B. Deng, A. M. DSouza, M. B. Nagarajan, P. Coan, and A. Wismuller, “Deep transfer learning for characterizing chondrocyte patterns in phase contrast x-ray computed tomography images of the human patellar cartilage,” Computers in biology and medicine, vol. 95, pp. 24–33, 2018.

S. U. H. Dar, M. Ozbey, A. B. ¨ C¸ atlı, and T. C¸ ukur, “A transfer-learning approach for accelerated mri using deep neural networks,” Magnetic resonance in medicine, vol. 84, no. 2, pp. 663–685, 2020.

A. Ebrahimi-Ghahnavieh, S. Luo, and R. Chiong, “Transfer learning for alzheimer’s disease detection on mri images,” in 2019 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT). IEEE, 2019, pp. 133–138.

C. Srinivas, N. P. KS, M. Zakariah, Y. A. Alothaibi, K. Shaukat, B. Partibane, and H. Awal, “Deep transfer learning approaches in performance analysis of brain tumor classification using mri mages,” Journal of Healthcare Engineering, vol. 2022, no. 1, p. 3264367, 2022.

V. Perumal, V. Narayanan, and S. J. S. Rajasekar, “Detection of covid-19 using cxr and ct images using transfer learning and haralick features,” Applied Intelligence, vol. 51, no. 1, pp. 341– 58, 2021.

A. Halder and B. Datta, “Covid-19 detection from lung ct-scan images using transfer learning approach,” Machine Learning: Science and Technology, vol. 2, no. 4, p. 045013, 2021.

T. K. Sajja, R. M. Devarapalli, and H. K. Kalluri, “Lung cancer detection based on ct scan images by using deep transfer learning,” Traitement du Signal, vol. 36, no. 4, pp. 339–344, 2019.

A. Mounsey, A. Khan, and S. Sharma, “Deep and transfer learning approaches for pedestrian identification and classification in autonomous vehicles,” Electronics, vol. 10, no. 24, p. 3159, 2021.

S. Akhauri, L. Y. Zheng, and M. C. Lin, “Enhanced transfer learning for autonomous driving with systematic accident simulation,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 5986–5993.

S. Sharma, J. E. Ball, B. Tang, D. W. Carruth, M. Doude, and M. A. Islam, “Semantic segmentation with transfer learning for off-road autonomous driving,” Sensors, vol. 19, no. 11, p. 2577, 2019.

X. Liu, J. Li, J. Ma, H. Sun, Z. Xu, T. Zhang, and H. Yu, “Deep transfer learning for intelligent vehicle perception: A survey,” Green Energy and Intelligent Transportation, p. 100125, 2023.

K. Thenmozhi and U. S. Reddy, “Crop pest classification based on deep convolutional neural network and transfer learning,” Computers and Electronics in Agriculture, vol. 164, p. 104906, 2019.

W. Dawei, D. Limiao, N. Jiangong, G. Jiyue, Z. Hongfei, and H. Zhongzhi, “Recognition pest by image-based transfer learning,” Journal of the Science of Food and Agriculture, vol. 99, no. 10, pp. 4524–4531, 2019.

N. E. M. Khalifa, M. Loey, and M. H. N. Taha, “Insect pests recognition based on deep transfer learning models,” J. Theor. Appl. Inf. Technol, vol. 98, no. 1, pp. 60–68, 2020.

G. Pattnaik, V. K. Shrivastava, and K. Parvathi, “Transfer learning-based framework for classification of pest in tomato plants,” Applied Artificial Intelligence, vol. 34, no. 13, pp. 981–993, 2020.

S. Ali, T. Abuhmed, S. El-Sappagh, K. Muhammad, J. M. Alonso-Moral, R. Confalonieri, R. Guidotti, J. Del Ser, N. D´ıaz-Rodr´ıguez, and F. Herrera, “Explainable artificial intelligence (xai): What we know and what is left to attain trustworthy artificial intelligence,” Information fusion, vol. 99, p. 101805, 2023.

G. Feretzakis, A. Sakagianni, A. Anastasiou, I. Kapogianni, E. Bazakidou, P. Koufopoulos, Y. Koumpouros, C. Koufopoulou, V. Kaldis, and V. S. Verykios, “Integrating shapley values into machine learning techniques for enhanced predictions of hospital admissions,” Applied Sciences, vol. 14, no. 13, p. 5925, 2024.

J. Sun, C.-K. Sun, Y.-X. Tang, T.-C. Liu, and C.-J. Lu, “Application of shap for explainable machine learning on age-based subgrouping mammography questionnaire data for positive mammography prediction and risk factor identification,” in Healthcare, vol. 11, no. 14. MDPI, 2023, p. 2000.

E. Stenwig, G. Salvi, P. S. Rossi, and N. K. Skjærvold, “Comparative analysis of explainable machine learning prediction models for hospital mortality,” BMC Medical Research Methodology, vol. 22, no. 1, p. 53, 2022.

S. Rao, S. Mehta, S. Kulkarni, H. Dalvi, N. Katre, and M. Narvekar, “A study of lime and shap model explainers for autonomous disease predictions,” in 2022 IEEE Bombay Section Signature Conference (IBSSC). IEEE, 2022, pp. 1–6.

L.-D. Quach, K. N. Quoc, A. N. Quynh, N. ThaiNghe, and T. G. Nguyen, “Explainable deep learning models with gradient-weighted class activation mapping for smart agriculture,” IEEE Access, vol. 11, pp. 83 752–83 762, 2023.

A. B. Arrieta, N. D´ıaz-Rodr´ıguez, J. Del Ser, A. Bennetot, S. Tabik, A. Barbado, S. Garc´ıa, S. Gil-Lopez, D. Molina, R. Benjamins ´ et al., “Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai,” Information fusion, vol. 58, pp. 82–115, 2020.

R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradientbased localization,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 618–626.

M. Xiao, L. Zhang, W. Shi, J. Liu, W. He, and Z. Jiang, “A visualization method based on the grad-cam for medical image segmentation model,” in 2021 International Conference on Electronic Information Engineering and Computer Science (EIECS). IEEE, 2021, pp. 242–247.

K. Lamba and S. Rani, “A novel approach of brain-computer interfacing (bci) and grad-cam based explainable artificial intelligence: Use case scenario for smart healthcare,” Journal of Neuroscience Methods, vol. 408, p. 110159, 2024.

H. Zhang and K. Ogasawara, “Grad-cam-based explainable artificial intelligence related to medical text processing,” Bioengineering, vol. 10, no. 9, p. 1070, 2023.

A. Hamza, M. Attique Khan, S.-H. Wang, M. Alhaisoni, M. Alharbi, H. S. Hussein, H. Alshazly, Y. J. Kim, and J. Cha, “Covid-19 classification using chest x-ray images based on fusion-assisted deep bayesian optimization and grad-cam visualization,” Frontiers in Public Health, vol. 10, p. 1046296, 2022.

G. M. Idroes, T. R. Noviandy, T. B. Emran, and R. Idroes, “Explainable deep learning approach for mpox skin lesion detection with grad-cam,” Heca Journal of Applied Sciences, vol. 2, no. 2, pp. 54–63, 2024.

I. Salehin, M. R. Khan, U. Habiba, N. H. Badhon, and N. N. Moon, “Bau-insectv2: An agricultural plant insect dataset for deep learning and biomedical image analysis,” Data in Brief, vol. 53, p. 110083, 2024.

M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4510–4520.

M. Tan and Q. Le, “Efficientnetv2: Smaller models and faster training,” in International conference on machine learning. PMLR, 2021, pp. 10 096–10 106.

G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700–4708.

T. Kasinathan, D. Singaraju, and S. R. Uyyala, “Insect classification and detection in field crops using modern machine learning techniques,” Information Processing in Agriculture, vol. 8, no. 3, pp. 446–457, 2021.

A. C. Teixeira, J. Ribeiro, R. Morais, J. J. Sousa, and A. Cunha, “A systematic review on automatic insect detection using deep learning,” Agriculture, vol. 13, no. 3, p. 713, 2023.

D. Ozdemir and M. S. Kunduraci, “Comparison of deep learning techniques for classification of the insects in order level with mobile software application,” IEEE Access, vol. 10, pp. 35 675 35 684, 2022.

X. Cao, Z. Wei, Y. Gao, and Y. Huo, “Recognition of common insect in field based on deep learning,” in Journal of Physics: Conference Series, vol. 1634, no. 1. IOP Publishing, 2020, p.012034



Refbacks

  • There are currently no refbacks.


 

Indonesian Journal of Electrical Engineering and Informatics (IJEEI)
ISSN 2089-3272

Creative Commons Licence

This work is licensed under a Creative Commons Attribution 4.0 International License.

web analytics
View IJEEI Stats