Unlocking Doors: A TinyML-Based Approach for Real-Time Face Mask Detection in Door Lock Systems

Azzedine El Mrabet, Ayoub Tber, Mohamed Benaly, Laamari Hlou, Rachid El Gouri

Abstract


In response to the rapid spread of coronaviruses, including COVID-19 and seasonal common cold viruses, this article introduces a proposed system for enhancing door lock systems using TinyML technology for real-time face mask detection. The research project focuses on developing a machine learning model based on the YOLOv5 architecture to classify individuals based on their mask-wearing behavior correctly, incorrectly, or not at all in high-risk spaces prone to the transmission of coronaviruses, such as healthcare facilities, laboratories, and public settings. The study outlines the hardware and software tools utilized, including the Raspberry Pi 4, camera hardware, and the YOLOv5 machine learning model. The model is trained using a dataset containing three different classes and converted to a TFLite format for efficient implementation on the Raspberry Pi. Evaluation results demonstrate a mean Average Precision (mAP) of 0.99 and an inference rate of 10FPS for a 128-frame size input. This proposed system offers practical implications for enhancing door lock systems and promoting public health and safety during outbreaks of coronaviruses, including COVID-19 and other seasonal coronaviruses, providing a valuable approach to decrease the spread of these diseases and mitigate transmission risks in high-risk spaces, thereby contributing to the overall reduction of public health threats.

Keywords


Face Mask Detection, TinyMl, Yolov5, Door Lock Systems, Raspberry pi

References


“Coronavirus disease (COVID-19) prevention and treatment methods and effective parameters: A systematic literature review,” Sustainable Cities and Society, vol. 64, p. 102568, Jan. 2021, doi: 10.1016/j.scs.2020.102568.

A. Bassi, B. M. Henry, L. Pighi, L. Leone, and G. Lippi, “Evaluation of indoor hospital acclimatization of body temperature before COVID-19 fever screening,” Journal of Hospital Infection, vol. 112, pp. 127–128, Jun. 2021, doi: 10.1016/j.jhin.2021.02.020.

D. K. Jain, Z. Zhang, and K. Huang, “Multi angle optimal pattern-based deep learning for automatic facial expression recognition,” Pattern Recognition Letters, vol. 139, pp. 157–165, Nov. 2020, doi: 10.1016/j.patrec.2017.06.025.

N. Zeng, H. Li, and Y. Peng, “A new deep belief network-based multi-task learning for diagnosis of Alzheimer’s disease,” Neural Comput & Applic, Jun. 2021, doi: 10.1007/s00521-021-06149-6.

A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, “YOLOv4: Optimal Speed and Accuracy of Object Detection.” arXiv, Apr. 22, 2020. doi: 10.48550/arXiv.2004.10934.

J. Redmon and A. Farhadi, “YOLOv3: An Incremental Improvement.” arXiv, Apr. 08, 2018. doi: 10.48550/arXiv.1804.02767.

S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” in Advances in Neural Information Processing Systems, Curran Associates, Inc., 2015. Accessed: Apr. 09, 2023. [Online]. Available: https://proceedings.neurips.cc/paper/2015/hash/14bfa6bb14875e45bba028a21ed38046-Abstract.html

H. Law and J. Deng, “CornerNet: Detecting Objects as Paired Keypoints,” presented at the Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 734–750. Accessed: Apr. 09, 2023. [Online]. Available: https://openaccess.thecvf.com/content_ECCV_2018/html/Hei_Law_CornerNet_Detecting_Objects_ECCV_2018_paper.html

T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal Loss for Dense Object Detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 318–327, Feb. 2020, doi: 10.1109/TPAMI.2018.2858826.

Z. Tian, C. Shen, H. Chen, and T. He, “FCOS: Fully Convolutional One-Stage Object Detection,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2019, pp. 9626–9635. doi: 10.1109/ICCV.2019.00972.

V. Bazarevsky, Y. Kartynnik, A. Vakunov, K. Raveendran, and M. Grundmann, “BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs.” arXiv, Jul. 14, 2019. doi: 10.48550/arXiv.1907.05047.

M. Loey, G. Manogaran, M. H. N. Taha, and N. E. M. Khalifa, “Fighting against COVID-19: A novel deep learning model based on YOLO-v2 with ResNet-50 for medical face mask detection,” Sustainable Cities and Society, vol. 65, p. 102600, Feb. 2021, doi: 10.1016/j.scs.2020.102600.

U. Mahbub, S. Sarkar, and R. Chellappa, “Partial face detection in the mobile domain,” Image and Vision Computing, vol. 82, pp. 1–17, Feb. 2019, doi: 10.1016/j.imavis.2018.12.003.

G. Zheng and Y. Xu, “Efficient face detection and tracking in video sequences based on deep learning,” Information Sciences, vol. 568, pp. 265–285, Aug. 2021, doi: 10.1016/j.ins.2021.03.027.

S. Singh, U. Ahuja, M. Kumar, K. Kumar, and M. Sachdeva, “Face mask detection using YOLOv3 and faster R-CNN models: COVID-19 environment,” Multimed Tools Appl, vol. 80, no. 13, pp. 19753–19768, May 2021, doi: 10.1007/s11042-021-10711-8.

A. Alzu’bi, F. Albalas, T. AL-Hadhrami, L. B. Younis, and A. Bashayreh, “Masked Face Recognition Using Deep Learning: A Review,” Electronics, vol. 10, no. 21, Art. no. 21, Jan. 2021, doi: 10.3390/electronics10212666.

A. A. Abed, A. Al-Ibadi, and I. A. Abed, “Real-time multiple face mask and fever detection using YOLOv3 and TensorFlow lite platforms,” Bulletin EEI, vol. 12, no. 2, pp. 922–929, Apr. 2023, doi: 10.11591/eei.v12i2.4227.

B. Qin and D. Li, “Identifying Facemask-Wearing Condition Using Image Super-Resolution with Classification Network to Prevent COVID-19,” Sensors, vol. 20, no. 18, Art. no. 18, Jan. 2020, doi: 10.3390/s20185236.

T. Nurindini, M. N. Swacaesar, R. W. Astika, H. Purwanto, R. A. Wijayanti, and M. Taufik, “Design of Smart Door Lock System Using Face Recognition and Mask Detection Based on Viola-Jones Algorithm with Android Integration,” vol. 13, 2023.

X. Su, M. Gao, J. Ren, Y. Li, M. Dong, and X. Liu, “Face mask detection and classification via deep transfer learning,” Multimed Tools Appl, vol. 81, no. 3, pp. 4475–4494, Jan. 2022, doi: 10.1007/s11042-021-11772-5.

A. K. Gaddam and B. K. Menugonda, “Face Mask Detection and Door Unlocking System Using Deep Learning,” IJRASET, vol. 10, no. 12, pp. 651–655, Dec. 2022, doi: 10.22214/ijraset.2022.47965.

M. Loey, G. Manogaran, M. H. N. Taha, and N. E. M. Khalifa, “A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic,” Measurement, vol. 167, p. 108288, Jan. 2021, doi: 10.1016/j.measurement.2020.108288.

“The Pascal Visual Object Classes Challenge: A Retrospective | SpringerLink.” https://link.springer.com/article/10.1007/s11263-014-0733-5 (accessed Apr. 02, 2023).

T.-Y. Lin et al., “Microsoft COCO: Common Objects in Context.” arXiv, Feb. 20, 2015. doi: 10.48550/arXiv.1405.0312.

J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection.” arXiv, May 09, 2016. doi: 10.48550/arXiv.1506.02640.

G. Jocher et al., “ultralytics/YOLOv5: v7.0 - YOLOv5 SOTA Realtime Instance Segmentation.” Zenodo, Nov. 22, 2022. doi: 10.5281/zenodo.7347926.

Y. Zhang, Z. Guo, J. Wu, Y. Tian, H. Tang, and X. Guo, “Real-Time Vehicle Detection Based on Improved YOLO v5,” Sustainability, vol. 14, no. 19, Art. no. 19, Jan. 2022, doi: 10.3390/su141912274.

“Face Mask Detection.” https://www.kaggle.com/datasets/andrewmvd/face-mask-detection (accessed Mar. 08, 2023).

M. Altayeb and A. Al-Ghraibah, “Voice controlled Camera Assisted Pick and Place Robot Using Raspberry Pi,” Indonesian Journal of Electrical Engineering and Informatics, vol. 10, no. 1, pp. 51–56, 2022, doi: 10.52549/ijeei.v10i1.3636.

J. Ieamsaard, S. N. Charoensook, and S. Yammen, “Deep Learning-based Face Mask Detection Using YOLOV5,” in 2021 9th International Electrical Engineering Congress (iEECON), Mar. 2021, pp. 428–431. doi: 10.1109/iEECON51072.2021.9440346.

I. ben Abdel Ouahab, L. Elaachak, M. Bouhorma, and Y. A. Alluhaidan, “Real-time Facemask Detector using Deep Learning and Raspberry Pi,” in 2021 International Conference on Digital Age & Technological Advances for Sustainable Development (ICDATA), Jun. 2021, pp. 23–30. doi: 10.1109/ICDATA52997.2021.00014.


Full Text: PDF

Refbacks

  • There are currently no refbacks.


 

Indonesian Journal of Electrical Engineering and Informatics (IJEEI)
ISSN 2089-3272

Creative Commons Licence

This work is licensed under a Creative Commons Attribution 4.0 International License.

web analytics
View IJEEI Stats