یادگیری تقویتی چهارچوبی به منظور پهنه بندی سیل در مناطق کوهستانی (با استفاده از مدل Deep Q-Learning)

Extended Abstract
Introduction and Goal
Understanding and reducing flood risk are among the primary priorities of researchers and policymakers. Flood susceptibility mapping is considered a key risk management tool in mountainous regions, as it identifies prone areas and provides a foundation for spatial planning, preparedness, and preventive measures. Traditional flood mapping approaches, such as hydrological modeling and historical data analysis, often face limitations including data scarcity, computational complexity, and the dynamic nature of environmental systems. Therefore, modern artificial intelligence (AI) methods—particularly machine learning (ML) and reinforcement learning (RL)—have been employed to process diverse datasets and identify complex patterns.
Among AI approaches, reinforcement learning is promising due to its capacity to interact with the environment and progressively improve decision-making. The Deep Q-Learning (DQL) model, as an advanced reinforcement learning technique, enables operation in continuous and high-dimensional state spaces without requiring discretization. This study aims to apply the DQL model to produce a flood susceptibility map for Chaharmahal and Bakhtiari Province, a mountainous and semi-arid region prone to flash floods caused by snowmelt and intense rainfall. The main objective is to provide a precise framework for flood hazard assessment and to support policymakers and crisis managers in reducing risk and enhancing resilience, particularly in vulnerable counties such as Ardal and Lordegan.

Materials and Methods
Chaharmahal and Bakhtiari Province, covering approximately 1,655,300 hectares in southwestern Iran within the Zagros mountain range, is characterized by steep slopes, deep valleys, and an extensive drainage network that plays a significant role in the region’s hydrological dynamics. Intense precipitation events in late winter and early spring frequently result in flash and riverine floods, causing considerable damage to infrastructure and agricultural lands.
A flood inventory consisting of 545 spatial points (346 flood and 199 non-flood locations) from 1983 to 2023 was compiled. The dataset was randomly and stratifiedly divided into training (389 points, 71%) and testing (156 points, 29%) subsets. Initially, 21 environmental variables were extracted from multiple sources, including a Digital Elevation Model (DEM), climatic time series, Landsat 9 imagery, and the SoilGrid database. After multicollinearity analysis, four variables were removed, and 17 final predictors were selected: elevation, flow accumulation, flow direction, stream connectivity, aspect, slope length, plan and profile curvature, Topographic Wetness Index (TWI), depth to bedrock, maximum 24-hour precipitation, mean and maximum temperature, snow depth, NDVI, surface sand percentage, and distance from residential areas.
The Deep Q-Learning model was applied to process continuous variables and generate a flood susceptibility map classified into five categories (very low to very high). Model performance was evaluated using AUC, Kappa, Recall, Precision, Specificity, and LogLoss metrics.

Results and Discussion Using the Deep Q-Learning (DQL) model, the resulting raster map—classified using the Natural Breaks algorithm—divided the study area (1,655,300 hectares) into five flood susceptibility classes: very low (6%, 97,221 ha), low (20%, 336,190 ha), moderate (26%, 429,453 ha), high (26%, 435,293 ha), and very high (22%, 357,140 ha).
Accuracy assessment indicated strong model performance. The AUC reached 0.93, confirming excellent discriminative ability. The Kappa coefficient was 0.72, indicating substantial agreement between predictions and observations. Recall was 0.87, demonstrating a high detection rate of flood-prone locations. Precision and Specificity were 0.90 and 0.85, respectively, while LogLoss was 0.65.
Variable importance analysis revealed that snow depth had the greatest influence, highlighting the critical role of snowmelt in spring floods. This was followed by flow accumulation (topographic characteristics), distance from residential areas (human and urban development impacts), NDVI (vegetation cover and soil permeability), mean temperature (climatic conditions), and slope length. Areas classified as very high susceptibility were mainly concentrated in low-lying areas, valleys, and along major rivers such as the Karun and Zayandeh-Rud rivers, particularly within Ardal and Lordegan counties. Comparison with similar studies based on supervised machine learning models suggests that DQL provides more balanced predictions in heterogeneous mountainous environments and demonstrates strong potential for flood risk management.

Conclusion and Suggestions
This study demonstrated that the Deep Q-Learning (DQL) model, as a deep reinforcement learning approach, has high capability for flood susceptibility mapping in complex mountainous regions such as Chaharmahal and Bakhtiari Province. By processing continuous environmental variables and accurately capturing nonlinear relationships, the model produced reliable and practical maps that effectively delineate high-risk areas.
The findings emphasize that the main flood drivers in the region include snowmelt, topographic characteristics, human development, and vegetation degradation—results that are consistent with field observations. The generated maps can serve as a practical basis for land-use planning, strengthening stormwater infrastructure, protecting residential areas against flash floods, and restoring vegetation ecosystems to enhance soil permeability.
For future studies, it is recommended to expand the flood inventory with more precise seasonal and temporal data, incorporate dynamic variables such as daily temperature fluctuations and snowmelt cycles, and conduct broader field validation, particularly in high-altitude and urban areas. Additionally, integrating DQL with other reinforcement learning approaches or hybrid models may further improve predictive performance.
Overall, this innovative framework provides a robust approach for sustainable flood risk management in similar regions of Iran and worldwide and can significantly support policymakers in enhancing community resilience.
Keywords: Iranian Highlands, Flood Risk Assessment, Reinforcement Learning, Machine Learning.