Autonomous Underwater Navigation Utilizing Computational Fluid Dynamics Guided Reinforcement Learning

Abstract

Coral reefs, home to over 25% of marine biodiversity, have declined by over 50% in the last 30 years due to climate change and pollution. They provide habitat and shelter for over 4,000 species of fish and protect coastal communities by reducing wave energy by 97%. Traditional monitoring methods like diver-led surveys, satellite imaging, and pre-programmed AUVs struggle with efficiency and adaptability in turbulent conditions, often resulting in incomplete data collection. This project evaluates whether utilizing computational fluid dynamics-guided (CFD) reinforcement learning agents (RL) can enhance AUV navigation in such environments, focusing on the effectiveness of incorporating flow pressure sensor data for improved performance. A high-fidelity CFD environment simulated realistic turbulent currents, reef obstacles, and dynamic conditions. With the Soft Actor Critic (SAC) algorithm, two RL agents were trained: one equipped with standard position-velocity feedback and another augmented with pressure-based hydrodynamic force feedback. Performance metrics included episode length, cumulative reward, and final position/heading error, with statistical tests assessing significance. Results indicated that while both agents successfully navigated turbulence, the pressure-augmented agent demonstrated superior performance, consistently achieving longer episode durations and higher rewards, indicative of faster convergence and increased stability. In rigorous tests, this agent significantly outperformed the baseline, maintaining near-zero steady-state position errors (~0.02±0.01 m compared to 0.20±0.05 m for the baseline, p<0.05) and smaller heading deviations. Integrating RL with CFD facilitated effective AUV navigation in complex flows, with pressure feedback enhancing control, precision, and robustness. This approach could lead to safer, more efficient AUV operations in challenging marine environments.