Enhancing GRU-Based DRL with Delta-LiDAR for Robust UAV Navigation in Partially Observable Dynamic Environments

Authors

Maryam Allawi Haddad
M.Sc. Student, Department of Computer Engineering, University of Basrah, Iraq
Dhayaa Raissan Khudher
Lecturer/Researcher, Department of Computer Engineering, University of Basrah, Basrah, Iraq. Research interests: machine learning for control, robotics, UAV systems.
DOI: https://doi.org/10.52549/ijeei.v13i3.7067
Download PDF
Published: 30-09-2025 • Issue: Vol. 13 No. 3: September 2025 • Section: Regular paper

Abstract

Partial observability and sensor limitations are challenging for the navigation of autonomous Unmanned Aerial Vehicles (UAVs). Deep Reinforcement Learning (DRL) algorithms have emerged as potential tools in advancing this field. However, their effectiveness degrades in challenging environments, particularly in the presence of dynamic obstacles. Recent research trends emphasize the need for new DRL variants that guarantee robustness, real-time adaptability, and improved generalization under uncertainty. This paper proposes a lightweight DRL architecture that combines Proximal Policy Optimization (PPO) with a Gated Recurrent Unit (GRU), extended with a temporal LiDAR differencing feature called Delta-LiDAR. The difference between consecutive LiDAR scans is computed to provide the velocity and directional cues without the computational burden of Long Short-Term Memory (LSTM) networks. We evaluate three models, PPO-LSTM, PPO-GRU, and Delta-LiDAR augmented PPO-GRU in a 3D simulated UAV navigation environment characterized by noise, clutter, and dynamic obstacles. We considered several metrics, including success rate, collision frequency, trajectory smoothness, and computational efficiency, to determine the effectiveness of each architecture. The experimental results demonstrate that Delta-LiDAR improves GRU-based temporal reasoning. The deployment complexity is reduced compared with the LSTM-based architecture, which makes it ideal for real-time UAV operation in partially observable environments.
Keywords: UAV autonomous navigation; PPO algorithm; GRU network; Delta-LiDAR; Feature encoder.

References

Z. Fang and A. V. Savkin, “Strategies for Optimized UAV Surveillance in Various Tasks and Scenarios: A Review,†Drones, vol. 8, no. 5, Art. no. 193, May 2024, doi: 10.3390/drones8050193.

W. Alawad, N. Ben Halima, and L. Aziz, “An Unmanned Aerial Vehicle (UAV) System for Disaster and Crisis Management in Smart Cities,†Electronics (Switzerland), vol. 12, no. 4, Feb. 2023, doi: 10.3390/electronics12041051.

H. S. Munawar, F. Ullah, S. Qayyum, S. I. Khan, and M. Mojtahedi, “Uavs in disaster management: Application of integrated aerial imagery and convolutional neural network for flood detection,†Sustainability (Switzerland), vol. 13, no. 14, Jul. 2021, doi: 10.3390/su13147547.

Štimac, Igor and Mihetec, Tomislav, “Application of Drones in Urban Areas,†Transportation research procedia, vol. 81, pp. 84-97, 2024,

Ke Wang, and Yining Chen, “Multi-UAV Navigation for Partially Observable Communication Coverage by Graph Reinforcement Learning,â€, IEEE Transactions on Mobile Computing, vol. 22, no. 7, 2023.

M. Wisniewski, P. Chatzithanos, W. Guo, and A. Tsourdos, “Benchmarking Deep Reinforcement Learning for Navigation in Denied Sensor Environments,†arXiv preprint arXiv:2410.14616, vol. 1, 2024.

V. J. Hodge, R. Hawkins, and R. Alexander, “Deep reinforcement learning for drone navigation using sensor data,†Neural Comput Appl, vol. 33, no. 6, pp. 2015–2033, Mar. 2021, doi: 10.1007/s00521-020-05097-x.

A. P. Kalidas, C. J. Joshua, A. Q. Md, S. Basheer, S. Mohan, and S. Sakri, “Deep Reinforcement Learning for Vision-Based Navigation of UAVs in Avoiding Stationary and Mobile Obstacles,†Drones, vol. 7, no. 4, p. 245, 2023, doi: 10.3390/drones7040245..

Y. Sheng, H. Liu, J. Li, and Q. Han, “UAV Autonomous Navigation Based on Deep Reinforcement Learning in Highly Dynamic and High-Density Environments,†Drones, vol. 8, no. 9, Sep. 2024, doi: 10.3390/drones8090516.

J. Terven, “Deep Reinforcement Learning: A Chronological Overview and Methods,†AI (Switzerland), vol. 6, no. 3, p. 46, 2025, doi: 10.3390/ai6030046.

J. Hu, X. Yang, W. Wang, P. Wei, L. Ying, and Y. Liu, “Obstacle Avoidance for UAS in Continuous Action Space Using Deep Reinforcement Learning,†IEEE Access, vol. 10, p 90623, 2022, doi: 10.1109/ACCESS.2022.3201962.

T. GUO, N. JIANG, B. LI, X. ZHU, Y. WANG, and W. DU, “UAV navigation in high dynamic environments: A deep reinforcement learning approach,†Chinese Journal of Aeronautics, vol. 34, no. 2, pp. 479–489, 2021, doi: 10.1016/j.cja.2020.05.011.

Y. Zhou et al., “Deep Reinforcement Learning for Real-Time Airport Emergency Evacuation Using Asynchronous Advantage Actor–Critic (A3C) Algorithm,†Mathematics, vol. 13, no. 14, p. 2269, 2025, doi: 10.3390/math13142269.

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. K. Openai, “Proximal Policy Optimization Algorithms.†arXiv preprint arXiv:1707.06347, 2017.

S. L. Brunton and J. N. Kutz, “Data‑Driven Science and Engineering: Machine Learning, Dynamical Systems, and Controlâ€, 2nd ed. Cambridge, U.K.: Cambridge University Press, 2022, doi: 10.1017/9781108380690.

J. Zhao et al., “Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target Tracking,†Biomimetics, vol. 7, no. 4, 2022, doi: 10.3390/biomimetics7040197.

Z. Jiang and G. Song, “A deep reinforcement learning strategy for UAV autonomous landing on a platform,†arXiv preprint arXiv:2209.02954, Sep. 2022.

O. Araar, N. Aouf, and I. Vitanov, “Vision Based Autonomous Landing of Multirotor UAV on Moving Platform,†Journal of Intelligent and Robotic Systems: Theory and Applications, vol. 85, no. 2, pp. 369–384, 2017, doi: 10.1007/s10846-016-0399-z.

Z. Hu, K. Wan, X. Gao, Y. Zhai, and Q. Wang, “Deep reinforcement learning approach with multiple experience pools for UAV’s autonomous motion planning in complex unknown environments,†Sensors, vol. 20, no. 7, Art. 1890, 2020, doi: 10.3390/s20071890.

J. Zhao, J. Sun, Z. Cai, L. Wang, and Y. Wang, “Endâ€toâ€end deep reinforcement learning for imageâ€based UAV autonomous control,†Applied Sciences (Switzerland), vol. 11, no. 18, pp. 1–21, 2021, doi: 10.3390/app11188419.

F. Hêche, O. Barakat, T. Desmettre, T. Marx, and S. Robert-Nicoud, “Offline reinforcement learning in high-dimensional stochastic environments,†Neural Computing and Applications, vol. 36, pp. 585–598, 2024, doi: 10.1007/s00521‑023‑09029‑3.

B. Rubí, B. Morcego, and R. Pérez, “Quadrotor Path Following and Reactive Obstacle Avoidance with Deep Reinforcement Learning,†Journal of Intelligent and Robotic Systems: Theory and Applications, vol. 103, no. 4, 2021, doi: 10.1007/s10846-021-01491-2.

H. Samma and S. El-Ferik, “Autonomous UAV Visual Navigation Using an Improved Deep Reinforcement Learning,†IEEE Access, vol. 12, pp. 79967–79977, 2024, doi: 10.1109/ACCESS.2024.3409780.

R. Xie, Z. Meng, L. Wang, H. Li, K. Wang, and Z. Wu, “Unmanned Aerial Vehicle Path Planning Algorithm Based on Deep Reinforcement Learning in Large-Scale and Dynamic Environments,†IEEE Access, vol. 9, pp. 24884–24900, 2021, doi: 10.1109/ACCESS.2021.3057485.

P. Miera, H. Szolc, and T. Kryjak, “LiDAR‑based drone navigation with reinforcement learning,†*TechRxiv Preprint*, Jul. 2023, doi: 10.36227/techrxiv.23784246.v1.

F. S. Leira, H. H. Helgesen, T. A. Johansen, and T. I. Fossen, “Object detection, recognition, and tracking from UAVs using a thermal camera,†Journal of Field Robotics, vol. 38, no. 2, pp. 242–267, 2021, doi: 10.1002/rob.21985.

S. Zhao, W. Wang, J. Li, S. Huang, and S. Liu, “Autonomous Navigation of the UAV through Deep Reinforcement Learning with Sensor Perception Enhancement,†Hindawi, vol. 2023, 2023, doi: 10.1155/2023/3837615.

A. T. Azar and A. Koubaa, “Drone deep reinforcement learning: A review,†Electronics, vol. 10, no. 9, Art. 999, 2021, doi: 10.3390/electronics10090999.

K. Ullah et al., “Short-Term Load Forecasting: A Comprehensive Review and Simulation Study with CNN-LSTM Hybrids Approach,†IEEE Access, vol. 12, pp. 111858–111881, 2024, doi: 10.1109/ACCESS.2024.3440631.

J. Zhang, Y. Guo, L. Zheng, Q. Yang, G. Shi, and Y. Wu, “Real-Time UAV Path Planning Based on LSTM Network,†Journal of Systems Engineering and Electronics, vol. 35, no. 2, pp. 374–385, 2024, doi: 10.23919/JSEE.2023.000157.

H. Liu, Y. Shen, S. Yu, Z. Gao, and T. Wu, “Deep reinforcement learning for mobile robot path planning,†J. Theory Pract. Eng. Sci., vol. 4, no. 3, 2024.

Z. Zheng and H. Duan, “UAV maneuver decision-making via deep reinforcement learning for short-range air combat,†Intelligence and Robotics, vol. 3, no. 1, pp. 76–94, 2023, doi: 10.20517/ir.2023.04.

B. Ghojogh and A. Ghodsi, “Recurrent neural networks and long short‑term memory networks: Tutorial and survey,†arXiv preprint arXiv:2304.11461, 2023.

Y. Zhang, W. Li, H. Wang, and M. Chen, “Memory-based deep reinforcement learning for COLREGs-compliant obstacle avoidance in USV with limited environmental knowledge,†Ocean Engineering, vol. 297, p. 121978, 2025. doi: 10.1016/j.oceaneng.2025.121978.

Z. Niu et al., “Recurrent attention unit: A new gated recurrent unit for long-term memory of important parts in sequential data.†Neurocomputing, vol. 517, pp. 1–9, 2023, doi: 10.1016/j.neucom.2022.10.050.

Author Biographies

Maryam Allawi Haddad
M.Sc. Student, Department of Computer Engineering, University of Basrah, Basrah, Iraq. Research interests: deep reinforcement learning, UAV navigation, LiDAR perception

How to Cite

[1]
M. A. Haddad and D. R. Khudher, “Enhancing GRU-Based DRL with Delta-LiDAR for Robust UAV Navigation in Partially Observable Dynamic Environments”, IJEEI, vol. 13, no. 3, pp. 798–813, Sep. 2025, doi: 10.52549/ijeei.v13i3.7067.