IVFD: An Intelligent Video Forgery Detection Framework Leveraging InceptionV3 and GRU for Enhanced Forensics
Abstract
Cloud computing-like services that are great at paying for and managing multimedia are fundamental technological innovations that have made it easier for individuals and organizations to adopt multimedia content. Thanks to social media, different people with different perspectives can voice their opinions and present data through photos and videos. However, video tampering is a significant issue because illegal modification of video content can easily mislead audiences and make it difficult for them to relate to reality. This is, therefore, a serious problem, as the consequences of video forgery are dire. Several image processing-based solutions have emerged to address video forgery. Artificial intelligence has recently allowed deep learning models to be trained extensively; hence, deep learning has been frequently used for video tampering detection. However, further work is still required to refine such models or develop hybrid models to improve the existing models' capabilities in identifying video forgeries and assisting digital forensics. We introduce a framework based on deep learning to automate the detection and localization of video forgeries. We offer a hybrid deep learning model that fuses Inception V3 with a Gated Recurrent Unit (GRU) as part of our framework. We also propose a new algorithm, Intelligent Video Forgery Detection (IVFD), to detect the forgeries and their invariants based on this hybrid model. Through empirical studies applied on a standard dataset, called the Deepfake Challenge dataset, we get an accuracy of 97.21%, which makes our hybrid deep learning model outperform many existing models. Since video content is prevalent in almost all applications in today's era, our design system should be laid on top of these applications, which can facilitate detecting the tampering of the videos and thereby contribute towards digital forensics.
References
ESMA DİLEK AND MURAT DENER. (2024). Enhancement of Video Anomaly Detection Performance Using Transfer Learning and Fine-Tuning. IEEE. 12, pp.73304 - 73322. http://DOI:10.1109/ACCESS.2024.3404553
Walid El Shafai, Mona A. Fouda, El Sayed M. El Rabaie, and Nariman Abd El Salam. (2024). A comprehensive taxonomy on multimedia video forgery detection techniques: challenges and novel trends. Springer. 83, p.4241–4307. https://doi.org/10.1007/s11042-023-15609-1
P. Aberna and L. Agilandeeswari. (2024). Digital image and video watermarking: methodologies, attacks, applications, and future directions. Springer. 83, pp.1-61. https://doi.org/10.1007/s11042-023-15806-y
Achhardeep Kaur, Azadeh Noori Hoshyar, Vidya Saikrishna, Selena Firmin and Feng. (2024). Deepfake video detection: challenges and opportunities. Springer. 59(159), pp.1-47. https://doi.org/10.1007/s10462-024-10810-6
PRABODH KUMAR SAHOO, MANOJ KUMAR PANDA, UPASANA PANIGRAHI, GANAPATI PANDA, PRINCE JAIN, MD. SHABIUL ISLAM, AND MOHAMMAD TARIQUL ISLAM. (2024). An Improved VGG-19 Network Induced Enhanced Feature Pooling for Precise Moving Object Detection in Complex Video Scenes. IEEE. 12, pp.45847 - 45864. http://DOI:10.1109/ACCESS.2024.3381612
Xia Zhao, Limin Wang, Yufei Zhang, Xuming Han, Muhammet Deveci, Milan Parmar. (2024). A review of convolutional neural networks in computer vision. Springer. 57(99), pp.1-43. https://doi.org/10.1007/s10462-024-10721-6
Shobhit Tyagi, and Divakar Yadav. (2021). A detailed analysis of image and video forgery detection techniques. Springer. 39, pp.1-21. https://doi.org/10.1007/s00371-021-02347-4
Yassine Himeur, Somaya Al-Maadeed, Hamza Kheddar, Noor Al-Maadeed, Khalid Abualsaud, Amr Mohamed, and Tamer Khattab. (2023). Video surveillance using deep transfer learning and deep domain adaptation: Towards better generalization. Elsevier. 119, pp.1-34. https://doi.org/10.1016/j.engappai.2022.105698
Abhisek Ray, Maheshkumar H. Kolekar, R. Balasubramanian, and Adel Hafiane. (2023). Transfer Learning Enhanced Vision-based Human Activity Recognition: A Decade-long Analysis. Elsevier. 3(1), pp.1-15. https://doi.org/10.1016/j.jjimei.2022.100142
Sanat Ramesh, Vinkle Srivastav, Deepak Alapatt, Tong Yu, Aditya Murali, Luca Sestini, Chinedu Innocent Nwoye, Idris Hamoud, Saurav Sharma, Antoine Fleurentin, Georgios Exarchakis, Alexandros Karargyris, and Nicolas Padoy. (2023). Dissecting self-supervised learning methods for surgical computer vision. Elsevier. 88, pp.1-19. https://doi.org/10.1016/j.media.2023.102844
Armin Danesh Pazho, Christopher Neff, Ghazal Alinezhad Noghre, Babak Rahimi Ardabili, Shanle Yao, Mohammadreza Baharani, and Hamed Tabkhi. (2023). Ancilia: Scalable Intelligent Video Surveillance for the Artificial Intelligence of Things. IEEE. 10(17), pp.14940 - 14951. http://DOI:10.1109/JIOT.2023.3263725
Shraddha Suratkar, and Faruk Kazi. (2023). Deep Fake Video Detection Using Transfer Learning Approach. Springer. 48, p.9727–9737. https://doi.org/10.1007/s13369-022-07321-3
YUNG-YAO CHEN, YU-HSIU LIN, YU-CHEN HU, CHIH-HSIEN HSIA, YI-AN LIAN, AND SIN-YE JHONG. (2022). Distributed Real-Time Object Detection Based on Edge-Cloud Collaboration for Smart Video Surveillance Applications. IEEE. 10, pp.93745 - 93759. http://DOI:10.1109/ACCESS.2022.3203053
LORENZO DE DONATO, FRANCESCO FLAMMINI, STEFANO MARRONE, CLAUDIO MAZZARIELLO, ROBERTO NARDONE, CARLO SANSONE, AND VALERIA VITTORINI. (2022). A Survey on Audio-Video Based Defect Detection Through Deep Learning in Railway Maintenance. IEEE. 10, pp.65376 - 65400. http://DOI:10.1109/ACCESS.2022.3183102
Tianfei Zhou, Fatih Porikli, David J. Crandall, Luc Van Gool, Wenguan Wang. (2022). A Survey on Deep Learning Technique for Video Segmentation. IEEE. 45(6), pp.7099 - 7122. http://DOI:10.1109/TPAMI.2022.3225573
Feng Ding; Guopu Zhu; Mamoun Alazab; Xiangjun Li and Keping Yu;. (2022). Deep-Learning-Empowered Digital Forensics for Edge Consumer Electronics in 5G HetNets. IEEE Consumer Electronics Magazine, pp.1-6. http://doi:10.1109/mce.2020.3047606
Verdoliva, Luisa. (2020). Media Forensics and DeepFakes: an overview. IEEE Journal of Selected Topics in Signal Processing, 14(5), pp.1-24. http://doi:10.1109/JSTSP.2020.3002101
Stoyanova, Maria; Nikoloudakis, Yannis; Panagiotakis, Spyridon; Pallis, Evangelos and Markakis, Evangelos K. .
(2020). A Survey on the Internet of Things (IoT) Forensics: Challenges, Approaches and Open Issues. IEEE Communications Surveys & Tutorials, 22(2), 1–38. http://doi:10.1109/COMST.2019.2962586
Parveen, Azra; Khan, Zishan Husain and Ahmad, Syed Naseem . (2019). Block-based copy–move image forgery detection using DCT. Iran Journal of Computer Science, 2, pp.1-11. http://doi:10.1007/s42044-019-00029-y
Licheng Jiao; Ruohan Zhang; Fang Liu; Shuyuan Yang; Biao Hou; Lingling Li and Xu Tang;. (2021). New Generation Deep Learning for Video Object Detection: A Survey . IEEE Transactions on Neural Networks and Learning Systems, 33(8), pp.1-21. http://doi:10.1109/tnnls.2021.3053249
Unlu, Eren; Zenou, Emmanuel; Riviere, Nicolas and Dupouy, Paul-Edouard . (2019). Deep learning-based strategies for the detection and tracking of drones using several cameras. IPSJ Transactions on Computer Vision and Applications, 11(1), pp.1-13. http://doi:10.1186/s41074-019-0059-x 4
Rashmika Nawaratne, Damminda Alahakoon, Daswin De Silva, and Xinghuo Yu. (2019). Spatiotemporal Anomaly Detection Using Deep Learning for Real-Time Video Surveillance. IEEE. 16(1), pp.393 - 402. http://DOI:10.1109/TII.2019.2938527
Kuntai Du; Ahsan Pervaiz; Xin Yuan; Aakanksha Chowdhery; Qizheng Zhang; Henry Hoffmann; and Junchen Jiang;. (2020). Server-Driven Video Streaming for Deep Learning Inference . Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication, pp. 557 - 570. http://doi:10.1145/3387514.3405887
Song, Huansheng; Liang, Haoxiang; Li, Huaiyu; Dai, Zhe; Yun, Xu . (2019). Vision-based vehicle detection and counting system using deep learning in highway scenes. European Transport Research Review, 11(51), pp.1-16. http://doi:10.1186/s12544-019-0390-4
Sharma, Vipul and Mir, Roohie Naaz. (2020). A comprehensive and systematic look up into deep learning based object detection techniques: A review. Computer Science Review, 38, pp. 1-29. http://doi:10.1016/j.cosrev.2020.100301
Sergiu Oprea; Pablo Martinez-Gonzalez; Alberto Garcia-Garcia; John Alejandro Castro-Vargas; Sergio OrtsEscolano; Jose Garcia-Rodriguez and Antonis Argyros;. (2020). A Review on Deep Learning Techniques for Video Prediction . IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(6), pp.1-20. Http://doi:10.1109/tpami.2020.3045007
Weixin Luo, Wen Liu, Dongze Lian, Jinhui Tang, Lixin Duan, Xi Peng, and Shenghua Gao. (2019). Video Anomaly Detection with Sparse Coding Inspired Deep Neural Networks. IEEE. 43(3), pp.1070 - 1084. http://DOI:10.1109/TPAMI.2019.2944377
Chen, Zhihao; Khemmar, Redouane; Decoux, Benoit; Atahouet, Amphani and Ertaud, Jean-Yves . (2019). Real Time Object Detection, Tracking, and Distance and Motion Estimation based on Deep Learning: Application to Smart Mobility, pp.1–6. http://doi:10.1109/EST.2019.8806222
Sankar K. Pal; Anima Pramanik; J. Maiti and Pabitra Mitra;. (2021). Deep learning in multi-object detection and tracking: state of the art. Applied Intelligence, 51, pp.1-30. http://doi:10.1007/s10489-021-02293-7
Mittal, Payal; Sharma, Akashdeep and Singh, Raman. (2020). Deep learning-based object detection in low-altitude UAV datasets: A survey. Image and Vision Computing, 104, pp.1-24. http://doi:10.1016/j.imavis.2020.104046
Wu, Xiongwei; Sahoo, Doyen and Hoi, Steven C.H. . (2020). Recent Advances in Deep Learning for Object Detection. Neurocomputing, 396, pp.39-64. http://doi:10.1016/j.neucom.2020.01.085
Johnston, Pamela; Elyan, Eyad and Jayne, Chrisina . (2019). Video tampering localisation using features learned from authentic content. Neural Computing and Applications, 32, pp.1-15. http://doi:10.1007/s00521-019-04272-z
Kompatsiaris, Ioannis; Huet, Benoit; Mezaris, Vasileios; Gurrin, Cathal; Cheng, Wen-Huang and Vrochidis, Stefanos . (2019). Detecting Tampered Videos with Multimedia Forensics and Deep Learning. ,pp.374–386. http://doi:10.1007/978-3-030-05710-7_31
Johnston, Pamela and Elyan, Eyad. (2019). A review of digital video tampering: From simple editing to full synthesis. Digital Investigation, 29, pp.67–81. http://doi:10.1016/j.diin.2019.03.006
Quanxin Yang; Dongjin Yu; Zhuxi Zhang; Ye Yao and Linqiang Chen;. (2021). Spatiotemporal Trident Networks: Detection and Localization of Object Removal Tampering in Video Passive Forensics . IEEE Transactions on Circuits and Systems for Video Technology, 31(10). Pp.1-14 http://doi:10.1109/tcsvt.2020.3046240
Yang, Tongfeng; Wu, Jian; Liu, Lihua; Chang, Xu and Feng, Guorui . (2020). VTD-Net: Depth Face Forgery Oriented Video Tampering Detection based on Convolutional Neural Network. pp.7247–7251. http://doi:10.23919/CCC50068.2020.9188580
Kaur, Harpreet; Jindal, Neeru . (2020). Deep Convolutional Neural Network for Graphics Forgery Detection in Video. Wireless Personal Communications, 112, pp.1-19. http://doi:10.1007/s11277-020-07126-3
Bui, Tu; Thereaux, Olivier; Cooper, Daniel; Collomosse, John; Bell, Mark; Green, Alex; Sheridan, John; Higgins, Jez; Das, Arindra and Keller, Jared Robert . (2020). Tamper-proofing Video with Hierarchical Attention Autoencoder Hashing on Blockchain. IEEE Transactions on Multimedia, 22(11), 1–11. http://doi:10.1109/TMM.2020.2967640
da Costa, Kelton A.P.; Papa, João P.; Passos, Leandro A.; Colombo, Danilo; Ser, Javier Del; Muhammad, Khan and de Albuquerque, Victor Hugo C. . (2020). A critical literature survey and prospects on tampering and anomaly detection in image data. Applied Soft Computing, 97, pp.1-15. http://doi:10.1016/j.asoc.2020.106727
Feng Ding; Guopu Zhu; Yingcan Li; Xinpeng Zhang; Pradeep K Atrey and Siwei Lyu;. (2021). Anti-Forensics for Face Swapping Videos via Adversarial Training. IEEE Transactions on Multimedia, 24, pp.1-13. http://doi:10.1109/tmm.2021.3098422
Deepfake Detection Challenge Dataset. Retrieved from https://www.kaggle.com/code/kuldeepprasadds321/fakevideo-image-detection/input
Refbacks
- There are currently no refbacks.
Indonesian Journal of Electrical Engineering and Informatics (IJEEI)
ISSN 2089-3272
This work is licensed under a Creative Commons Attribution 4.0 International License.