A Reinforcement Learning Framework for Flood Zoning in Mountainous Areas using the Deep Q-Learning Model

Introduction and Goal 
Understanding flood risk efforts to reduce it are top priorities for researchers and policymakers. Flood susceptibility mapping is an important tool for risk management in mountainous regions. Because, it is possible to identify prone areas and provide a platform for spatial planning, preparedness, and preventive measures. Traditional flood mapping approaches, such as hydrological modeling and historical data analysis, often face limitations including data scarcity, computational complexity, and the dynamic of environmental systems. Therefore, modern artificial intelligence (AI) methods—particularly machine learning (ML) and reinforcement learning (RL)—have been employed to process diverse datasets and identify complex patterns. Among AI approaches, reinforcement learning is promising due to its capacity to interact with the environment and progressively improve decision-making. The Deep Q-Learning (DQL) model, as an advanced reinforcement learning technique, enables operation in continuous and high-dimensional state spaces without requiring discretization. This study aims to apply the DQL model to produce a flood susceptibility map in the Chaharmahal and Bakhtiari Province, a mountainous and semi-arid region prone to flash floods caused by snowmelt and intense rainfall. The main objective is to provide a precise framework for flood hazard assessment and to support policymakers and crisis managers in reducing risk and enhancing resilience, particularly in vulnerable counties such as Ardal and Lordegan.
Materials and Methods
To conduct this research, a list of flood events including 545 spatial points (346 flood and 199 non-flood locations) from 1983 to 2023 was compiled. In a random and stratified manner, 71% of the data was used for training (389 points) and 29% for testing (156 points). Initially, 21 environmental variables were extracted from multiple sources, including a Digital Elevation Model (DEM), climatic time series, Landsat 9 imagery, and the SoilGrid database. After multicollinearity analysis, four variables were removed, and 17 final variables (elevation, flow accumulation, flow direction, stream connectivity, aspect, slope length, plan and profile curvature, Topographic Wetness Index (TWI), depth to bedrock, maximum 24-hour precipitation, mean and maximum temperature, snow depth, NDVI, surface sand percentage, and distance from residential areas) were selected. The Deep Q-Learning model was applied to process continuous variables and generate a flood susceptibility map. This map was classified into five categories (very low to very high). The accuracy of the model was evaluated by indicators such as AUC, Kappa, Recall, Precision, Specificity, and LogLoss metrics.
Results and Discussion
The classification map was prepared using the Deep Q-Learning (DQL) model and based on the Natural break method. In this map, the area of the study area (1,655,300 hectares) was divided into five flood susceptibility classes: very low (6%, 97,221 ha), low (20%, 336,190 ha), moderate (26%, 429,453 ha), high (26%, 435,293 ha), and very high (22%, 357,140 ha). Accuracy assessment indicated very good model performance. The results of the AUC index (0.93) indicated excellent discriminative ability, the Kappa coefficient index (0.72) indicating acceptable agreement between predictions and observations, and the Recall index (0.87) indicated high identification of flood points. The Precision index and was calculated to be 0.90, Specificity was 0.85, and LogLoss was 0.65. Analysis of the significance of the variables showed that the greatest effect was related to snow depth, which has a significant impact on snowmelt in spring floods. This was followed by flow accumulation (topographic characteristics), distance from residential areas (human and urban development impacts), NDVI (vegetation cover and soil permeability), mean temperature (climatic conditions), and slope length. Areas classified as very high susceptibility were mainly concentrated in low-lying areas, valleys, and along major rivers such as the Karoon and Zayandeh-Rud Rivers, particularly within Ardal and Lordegan counties. Comparison the results of this study with similar studies based on supervised machine learning models showed that using the DQL model, flood risk in heterogeneous mountainous environments can be predicted more reliably and flood risk can be managed more accurately.
Conclusion and Suggestions
The results of this study demonstrated that the Deep Q-Learning (DQL) model, as a deep reinforcement learning approach, has high capability for flood susceptibility mapping, especially in complex mountainous regions such as Chaharmahal and Bakhtiari Province. By utilizing this model, processing continuous environmental variables and accurately identifying nonlinear relationships, valid and practical maps were produced, and based on that, high-risk areas were accurately identified. The findings of this study confirm that the main flood drivers in the region include snowmelt, topographic characteristics, human development, and vegetation degradation. These findings were also consistent with the field realities of the region. The generated maps can serve as a practical basis for land-use planning, strengthening stormwater infrastructure, protecting residential areas against flash floods, and restoring vegetation ecosystems to enhance soil permeability. Overall, the results of this study provided a novel approach and a detailed framework for sustainable flood risk management in similar regions in Iran and the world, which policymakers can benefit from in improving the resilience of local communities. To improve the accuracy of the model in future research, it is recommended that a broader list of flood events be compiled with more accurate seasonal and temporal data, and that dynamic variables such as daily temperature changes and snowmelt cycles be examined. Additionally, it is recommended to conduct more extensive field validation, especially in high altitudes and urban areas. It is also suggested to combine DQL with other reinforcement learning methods or hybrid models to increase its efficiency.