Deep Learning for Arabic Question Classification: Leveraging BERT and Hybrid Neural Networks

Authors

Somia Khedimi
Department of Computer Science, Institute of Science, University of Naama, Algeria
Abdelghani Bouziane
Department of Computer Science, Institute of Science, University of Naama, Algeria
Djelloul Bouchiha
Department of Computer Science, Institute of Science, University of Naama, Algeria
DOI: https://doi.org/10.52549/ijeei.v13i4.6312
Download PDF
Published: 22-11-2025 • Issue: Vol. 13 No. 4: December 2025 • Section: Regular paper

Abstract

This paper presents a deep learning approach for Arabic question classification, leveraging the strengths of pre-trained language models and advanced neural network architectures to address the unique challenges of Arabic text processing. The proposed methodology employs BERT and Word2Vec to generate contextualized and semantic-rich representations of Arabic questions, effectively capturing their linguistic intricacies and morphological complexity. These embeddings are fed into a hybrid classification framework combining Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) networks, enabling the extraction of both spatial and sequential features from the input. Experimental results demonstrate the model’s effectiveness, achieving an accuracy of 85.12%, along with high precision, recall, and F1-score metrics. These findings highlight the potential of integrating pre-trained Arabic-specific language models with hybrid deep learning architectures, providing a robust solution for Arabic question classification. This work contributes to advancing Arabic natural language processing, offering a strong foundation for the development of high-performance question-answering systems and related applications.
Keywords: Arabic question classification; Natural Language Processing (NLP); Deep learning; CNN; BiLSTM; Pre-trained language models

References

Hamza, A., "Embedding arabic questions by feature-level fusion of word representations for questions classification: It is worth doing?", Computer and Information Sciences, 2022.

Balla, H., M.L., Salvador, and S.J. Delany, "Arabic Question Classification using Deep Learning", in CCRIS'22: 2022 3rd International Conference on Control, Robotics and Intelligent System, 2022.

Li, X. and D. Roth, "Learning question classifiers", in the 19th international conference , Association for Computational Linguistics, 2002

Bloom, B.S., et al., "Taxonomy of educational objectives: The classification of educational goals", Vol. 1, David McKay Company, INC. USA, EDWARDS BROS., ANN ARBOR, MICHIGAN, 1956.

Loni, B., "A Survey of State-of-the-Art Methods on Question Classiï¬cation", 2011.

Anhar, R., T.B. Adji, and N. Akhmad Setiawan, "Question Classification on Question-Answer System using Bidirectional-LSTM", in 2019 5th International Conference on Science and Technology (ICST), IEEE ,2019.

Hochreiter, S. and J. Schmidhuber, "Long short-term memory", Neural computation, vol. 9, no. 8, 1997, pp. 1735-1780.

Al-dihaymawee, D.T.M., A.A. Merzah, and H.M. Abdul Ridha, "The Story of Arabic Language: Historical Linguistics Study", Tasnim International Journal for Human, Social and Legal Sciences, vol. 3, No. 1, 2024, pp. 572-582.

Zulqarnain, M., et al., "A comparative analysis on question classification task based on deep learning approaches", PeerJ Computer Science, vol. 7, 2021, pp. e570.

Al-Smadi, B.S., "DeBERTa-BiLSTM: A multi-label classification model of Arabic medical questions using pre-trained models and deep learning", Computers in Biology and Medicine, vol. 170, 2024, pp. 107921.

Guo, X., et al., "Efficient Agricultural Question Classification With a BERT-Enhanced DPCNN Model", IEEE Access, vol. 12, 2024, pp. 109255-109268.

Liu, G., et al., "Chinese Question Classification Based on ERNIE and Feature Fusion", Springer International Publishing, 2020.

Alammary, A.S., "Arabic Questions Classification Using Modified TF-IDF", IEEE Access, vol 9, 2021, pp. 95109-95122.

Malkawi, R., S. Alsrahan, and A.A. Saifan, "Arabic questions classification machine learning algoritims", Journal of Theoretical and Applied Information Technology , Vol. 100, no. 20, 2022.

Faris, H., et al., "Classification of Arabic healthcare questions based on word embeddings learned from massive consultations: a deep learning approach", Journal of Ambient Intelligence and Humanized Computing, vol. 13, no. 4, 2022. pp. 1811-1827.

Mikolov, T., et al., "Efficient Estimation of Word Representations in Vector Space", arXiv, 2013.

Devlin, J., et al.," BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", arXiv, 2019.

Safaya, A., M. Abdullatif, and D. Yuret, "KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media", in Proceedings of the Fourteenth Workshop on Semantic Evaluation, International Committee for Computational Linguistics, 2020.

Antoun, W., F. Baly, and H. Hajj, "AraBERT: Transformer-based Model for Arabic Language Understanding", arXiv, 2021.

Ghaddar, A., et al., "JABER and SABER: Junior and Senior Arabic BERt", arXiv, 2022.

Abdul-Mageed, M., A. Elmadany, and E.M.B. Nagoudi, "ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic", in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing , vol. 1, Association for Computational Linguistics. 2021.

Soliman, A.B., K. Eissa, and S.R. El-Beltagy, "AraVec: A set of Arabic Word Embedding Models for use in Arabic NLP", Procedia Computer Science, vol. 117, 2017, pp. 256-265.

Schuster, M. and K.K. Paliwal, Bidirectional recurrent neural networks, IEEE transactions on Signal Processing, vol. 45, no. 11, 1997, pp. 2673-2681.

How to Cite

[1]
S. Khedimi, A. Bouziane, and D. Bouchiha, “Deep Learning for Arabic Question Classification: Leveraging BERT and Hybrid Neural Networks”, IJEEI, vol. 13, no. 4, pp. 888–896, Nov. 2025, doi: 10.52549/ijeei.v13i4.6312.