Boosting Few-Shot Text Classification in Large Language Models with Data Augmentation

Authors

Ahmed El Saeid Ali Soliman
Cairo University image/svg+xml
Reda Abd Elwahab El-Khoribi
Cairo University image/svg+xml
Basma El-Demerdash
Cairo University image/svg+xml
Ahmed Elgayar
Cairo University image/svg+xml
DOI: https://doi.org/10.52549/ijeei.v14i2.7074
Download PDF
Published: 21-04-2026 • Issue: Vol. 14 No. 2: June 2026 • Section: Regular paper

Abstract

Few-shot text classification is a challenging problem in natural language processing. Models have to generalize from a small number of labeled examples. This paper investigates the effectiveness of data augmentation methods (back-translation, paraphrasing, and noise injection) on overfitting prevention and generalization enhancement in few-shot scenario for small transformer models such as DistilBERT and DistilRoBERTa. We perform experiments on two standard benchmark datasets, DBPedia-14 and BBC News, in the 3-shot, 5-shot, and 7-shot settings. The result shows that the data augmentation substantially improves the performance of the classification. With back-translation, DistilBERT achieves 0.98 and 0.96 accuracy on DBPedia-14 and BBC News in 7-shot setting, compared to 0.88 and 0.86 accuracy without any augmentation. These results demonstrate that carefully selected augmentations can bridge the performance gap between few-shot and fully supervised learning, enabling competitive results on resource constrained hardware without the need for massive, labeled datasets.
Keywords: Text Classification; Transformer; Few-Shot Learning; Neural Network; Natural Language Processing; Large Language Model; Data Augmentation

References

D. M. Hawkins, “The problem of overfitting,†Journal of Chemical Information and Computer Sciences, vol. 44, no. 1, pp. 1–12, 2004.

C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,†Journal of Big Data, vol. 6, p. 60, 2019.

R. Sennrich, B. Haddow, and A. Birch, “Improving neural machine translation models with monolingual data,†arXiv preprint arXiv:1511.06709, 2016.

N. Edunov, M. Ott, M. Auli, and D. Grangier, “Understanding back-translation at scale,†in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), Brussels, Belgium, Oct. 2018, pp. 489–500.

C. Zhou, S. Bai, and J. Zhao, “Paraphrase generation with deep reinforcement learning,†in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, Oct. 2018, pp. 3865–3878.

M. Wieting and K. Gimpel, “ParaNMT-50M: Pushing the limits of paraphrastic sentence embeddings with millions of machine translations,†in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, Jul. 2018, pp. 451–462.

Q. Xie, E. Hovy, M. Luong, and Q. V. Le, “Data noising as smoothing in neural network language models,†arXiv preprint arXiv:1703.02573, 2017.

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,†in Proceedings of NAACL-HLT, Minneapolis, MN, USA, Jun. 2019, pp. 4171–4186.

L. Fei-Fei, R. Fergus, and P. Perona, “One-shot learning of object categories,†IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 594–611, 2006.

B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum, “Human-level concept learning through probabilistic program induction,†Science, vol. 350, no. 6266, pp. 1332–1338, 2015.

Y. Zhang, M. Cheng, L. Chen, and S. Wang, “Transfer learning for cross-dataset text classification,†in Proceedings of COLING 2015, Beijing, China, Aug. 2015, pp. 2661–2670.

J. Howard and S. Ruder, “Universal language model fine-tuning for text classification,†in Proceedings of ACL, Melbourne, Australia, Jul. 2018, pp. 328–339.

T. Hospedales, A. Antoniou, P. Micaelli, and A. Storkey, “Meta-learning in neural networks: A survey,†IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 9, pp. 5149–5169, 2022.

C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,†in Proceedings of the 34th International Conference on Machine Learning (ICML), Sydney, Australia, Aug. 2017, pp. 1126–1135.

J. Wei and K. Zou, “EDA: Easy data augmentation techniques for boosting performance on text classification tasks,†in Proceedings of EMNLP/IJCNLP, Hong Kong, China, Nov. 2019, pp. 6382–6388.

M. Gao, Y. Li, and X. Han, “A survey on data augmentation for text classification,†in Journal of Physics: Conference Series, vol. 1744, no. 3, 2021.

H. Chen, W. Xu, and Y. Lin, “MixText: Linguistically-informed interpolation of hidden space for semi-supervised text classification,†in Proceedings of ACL, Seattle, WA, USA, Jul. 2020, pp. 150–160.

L. Perez and J. Wang, “The effectiveness of data augmentation in image classification using deep learning,†arXiv preprint arXiv:1712.04621, 2017.

R. Coulombe, “Text data augmentation made simple by leveraging NLP cloud APIs,†Medium, Jun. 2018. [Online]. Available: https://towardsdatascience.com/text-data-augmentation-29aafcb60c7f

B. Gao, Y. Liu, and T. Zhang, “Data augmentation in natural language processing: A survey,†arXiv preprint arXiv:2105.03075, 2021.

X. L. Liu et al., “Back translation and paraphrasing for data augmentation in spoken language understanding,†in Proceedings of ICASSP, Barcelona, Spain, May 2020, pp. 8409–8413.

T. Yu, P. Liu, and H. Lin, “Paraphrase generation from latent-variable PCFGs for text augmentation,†in Proceedings of COLING, Santa Fe, NM, USA, Aug. 2018, pp. 106–117.

Z. Wang and Y. Yang, “Enhancing text classification with paraphrase generation,†in Proceedings of ACL, Florence, Italy, Jul. 2019, pp. 315–320.

A. E. Zirikly and M. Lease, “Improving data augmentation for low-resource text classification,†in Proceedings of NAACL, Minneapolis, MN, USA, Jun. 2019, pp. 61–66.

W. Zhao, M. Zhou, S. Li, and L. Cai, “Word swapping for text augmentation in NLP,†in Proceedings of COLING, Barcelona, Spain, Dec. 2020, pp. 2101–2110.

Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, and D. Chen, “RoBERTa: A robustly optimized BERT pretraining approach,†arXiv preprint arXiv:1907.11692, 2019.

A. Radford et al., “Language models are few-shot learners,†in Advances in Neural Information Processing Systems (NeurIPS), Vancouver, Canada, Dec. 2020.

P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre-train prompt tune: Towards generalizable zero-shot and few-shot learning,†in Proceedings of ACL, Toronto, Canada, Jul. 2023.

M. Ruder, “Neural transfer learning for natural language processing,†Ph.D. dissertation, National University of Ireland, 2019.

T. Yu, C. Lin, X. Jiang, J. Yin, S. Shah, and X. Yan, “Few-shot intent detection with pattern-enhanced learning and feature mixing,†in Proceedings of NAACL-HLT, Mexico City, Mexico, Jun. 2021, pp. 1392–1402.

Y. Feng, L. Xu, W. Ma, and X. Ma, “A survey on simple data augmentation for few-shot text classification,†in Proceedings of ACL Findings, Online, Aug. 2021, pp. 1234–1243.

Author Biographies

Ahmed El Saeid Ali Soliman
Ahmed El-Saeid Ali Soliman is a Senior Team Lead Software Engineer and Data Scientist with extensive expertise in business process automation, data engineering, and deep learning. He holds a Master’s degree in Data Science from Cairo University. His research focus lies at the intersection of Natural Language Processing (NLP) and Large Language Models (LLMs), with specific contributions in Few-Shot Learning and Data Augmentation techniques. Throughout his career, Ahmed has led and contributed to numerous high-impact projects across government and private sectors in Egypt, Saudi Arabia, and the UAE. His technical proficiency spans Python-based machine learning, Deep earning, neural networks, and advanced data visualization. He can be contacted at: aelsaeid.soliman@gmail.com.
Reda Abd Elwahab El-Khoribi
Reda Abd Elwahab El-Khoribi, Ph.D., is a Professor of Information Technology and the Dean of the Faculty of Computers and Artificial Intelligence at Cairo University, Egypt. He earned his doctorate in Electronics and Communications Engineering and has been a member of FCAI-CU since January 2008, where he has guided both undergraduate and graduate teaching across topics including machine learning, digital signal processing, pattern recognition, feature extraction, and image/video processing. Dr. El-Khoribi’s research has resulted in impactful contributions to areas such as emotion recognition via EEG using LSTM and 3D-CNN models, biosensor-based cancer cell detection, remote sensing object detection, and adaptive e-learning personalization. His work has been widely cited, with over 1,700 total citations, an h-index of 14, and an i10-index of 20 as of April 2025, placing him among the top-tier computer science researchers at Cairo University, He can be contacted at email: r.abdelwahab@fci-cu.edu.eg
Basma El-Demerdash
Basma E. El-Demerdash is an associate professor at the Department of Operations Research and Management, Faculty of Graduate Studies for Statistical Research (FGSSR), Cairo University. She graduated with a B.Sc. in Operations Research and Decision Support from the Faculty of Computers and Information, Cairo University in 2007. Her outstanding academic performance earned her a position as a teaching assistant in the department. She then went on to receive her M.Sc. degree from the same department in 2011.Her thesis focused on evaluating higher education systems in Egypt using data envelopment analysis. Her academic career began as a teaching assistant in 2007. Since then, she has accumulated 17 years of teaching experience at the Faculty of Computers and Artificial Intelligence, Cairo University. She can be contacted at email: basma.ezzat@cu.edu.eg
Ahmed Elgayar
Ahmed Abdelhamed Elgayar is a Senior BI & Data Analytics Engineer with over five years of professional experience in business intelligence, data analytics, and data visualization, along with a broader background in civil engineering and project controls. He holds a Master’s degree in Data Science and a Postgraduate Diploma in Data Science with distinction from the Faculty of Graduate Studies for Statistical Research, Cairo University. His expertise includes Power BI, SQL, Python, ETL development, data warehousing, and the Microsoft BI stack (SSIS, SSAS, and SSRS). He has contributed to large-scale business intelligence projects across governmental and private sectors in the United Arab Emirates, Saudi Arabia, and Egypt. His research interests include Arabic natural language processing (NLP), deep learning, machine learning, and AI-based language processing systems.

How to Cite

[1]
A. E. S. A. Soliman, R. A. E. El-Khoribi, B. El-Demerdash, and A. Elgayar, “Boosting Few-Shot Text Classification in Large Language Models with Data Augmentation”, IJEEI, vol. 14, no. 2, pp. 463–474, Apr. 2026, doi: 10.52549/ijeei.v14i2.7074.