SavedAdd your journal

Journals / Gazi University Journal of Science Part A: Engineering and Innovation / 2026 / Volume: 13 Issue: 2

Reinforcement Learning-Based Self-Healing Framework in IoT Sensor Networks

Tunahan Timuçin(DÜZCE ÜNİVERSİTESİ)0000-0003-0332-4118
Pages
—
Publication Date
2026-06-30
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The DQN agent had the best cumulative reward and the most stable performance in the cascading failure case; scalability experiments proved that DQN can work with a stable performance even in the 500-node networks.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:The fast increase of devices in Internet of Things (IoT) networks, which is projected to grow to over 21 billion devices by 2025, will require more sophisticated management paradigms, as current rule-based and reactive frameworks cannot support networks of this scale.
📊
Key Findings & Data:The proposed framework has been tested in a custom Python/Gymnasium simulation framework, with six failure modes (single node failure, sensor drift, gateway failure, concurrent multiple failure, network congestion, cascading failure) in cluster topology networks, using 50 to 500 nodes.
💡
Conclusion & Impact:The DQN agent had the best cumulative reward and the most stable performance in the cascading failure case; scalability experiments proved that DQN can work with a stable performance even in the 500-node networks.
🔬Design:Modeling & Simulation
📍Scope:2025 Verileri
⚙️Variables:Internet of Things · Self-healing · Reinforcement Learning

Abstract

The fast increase of devices in Internet of Things (IoT) networks, which is projected to grow to over 21 billion devices by 2025, will require more sophisticated management paradigms, as current rule-based and reactive frameworks cannot support networks of this scale. Rule-based systems are capable of processing only of predefined failure patterns and when it comes to complex situations like simultaneous multiple failures and cascading failures, they cannot work. The proposed paper suggests a self-healing framework of reinforcement learning (RL) with respect to an IoT sensor network. The original innovation of the framework is that it brings the MAPE-K cycle (Monitor-Analyze-Plan-Execute over Knowledge) framework, the fundamental reference model of autonomous computing, to which a learning element is added to form the MAPE-K+L model. This aspect provides the system to be able to enhance its policy gradually through learning on the past failures. The proposed framework has been tested in a custom Python/Gymnasium simulation framework, with six failure modes (single node failure, sensor drift, gateway failure, concurrent multiple failure, network congestion, cascading failure) in cluster topology networks, using 50 to 500 nodes. The Q-Learning and Deep Q-Network (DQN) agents were fully contrasted with random (RND) and rule-based (RB) baselines. The Q-Learning agent in the multiple failure scenario decreased the mean recovery time (MTTR) by 51.9 and 32.8 percent relative to random selection and rule-based approach respectively (p<0.001, Cohen d=1.424). The DQN agent had the best cumulative reward and the most stable performance in the cascading failure case; scalability experiments proved that DQN can work with a stable performance even in the 500-node networks.

🔖

Cite This Article

Timuçin, T. (2026). Reinforcement Learning-Based Self-Healing Framework in IoT Sensor Networks. Gazi University Journal of Science Part A: Engineering and Innovation, Volume: 13 Issue: 2. https://doi.org/10.54287/gujsa.1914900

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (30)

  1. Adeniyi, O., Sadiq, A. S., Pillai, P., Taheir, M. A., & Kaiwartya, O. (2023). Proactive self-healing approaches in mobile edge computing: A systematic literature review. Computers, 12(3), 63. https://doi.org/10.3390/computers12030063
  2. Albrecht, S. V., Christianos, F., & Schäfer, L. (2024). Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press.
  3. Aldrini, J., Chihi, I., & Sidhom, L. (2024). Fault diagnosis and self-healing for smart manufacturing: a review. Journal of Intelligent Manufacturing, 35(6), 2441-2473. https://doi.org/10.1007/s10845-023-02165-6
  4. Alhanaf, A. S., Balik, H. H., & Farsadi, M. (2023). Intelligent fault detection and classification schemes for smart grids based on deep neural networks. Energies, 16(22), 7680. https://doi.org/10.3390/en16227680
  5. Aliu, O. G., Imran, A., Imran, M. A., & Evans, B. (2013). A survey of self organisation in future cellular networks. IEEE Communications Surveys & Tutorials, 15(1), 336-361. https://doi.org/10.1109/SURV.2012.021312.00116
  6. Aminikhanghahi, S., & Cook, D. J. (2017). A survey of methods for time series change point detection. Knowledge and Information Systems, 51(2), 339-367. https://doi.org/10.1007/s10115-016-0987-z
  7. Asghar, M. Z., Nieminen, P., Hämäläinen, S., Ristaniemi, T., Imran, M. A., & Hämäläinen, T. (2017). Towards proactive context-aware self-healing for 5G networks. Computer Networks, 128, 5-13. https://doi.org/10.1016/j.comnet.2017.04.053
  8. Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41(3), 15. https://doi.org/10.1145/1541880.1541882
  9. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd Ed.). Lawrence Erlbaum Associates. https://doi.org/10.4324/9780203771587
  10. Devi, S. K., Thenmozhi, R., & Kumar, D. S. (2024). Self-healing IoT sensor networks with isolation forest algorithm for autonomous fault detection and recovery. In: Proceedings of the International Conference on Automation and Computation (AUTOCOM) (14-16 March 2024, pp. 451-456), Dehradun, India. https://doi.org/10.1109/AUTOCOM60220.2024.10486184
  11. Feng, X., Wu, J., Wu, Y., Li, J., & Yang, W. (2023). Blockchain and digital twin empowered trustworthy self-healing for edge-AI enabled industrial Internet of things. Information Sciences, 642, 119169. https://doi.org/10.1016/j.ins.2023.119169
  12. Hagberg, A. A., Schult, D. A., & Swart, P. J. (2008). Exploring network structure, dynamics, and function using NetworkX. In: G. Varoquaux, T. Vaught, & J. Millman (Eds.), Proceedings of the 7th Python in Science Conference (SciPy 2008) (19-24 August 2008, pp. 11-15), Pasadena, California. https://doi.org/10.25080/TCWV9851
  13. Johnphill, O., Sadiq, A. S., Al-Obeidat, F., Al-Khateeb, H., Taheir, M. A., Kaiwartya, O., & Ali, M. (2023). Self-healing in cyber–physical systems using machine learning: A critical analysis of theories and tools. Future Internet, 15(7), 244. https://doi.org/10.3390/fi15070244
  14. Karamthulla, M. J., Arasu Malaiyappan, J. N., & Prakash, S. (2023). AI-powered self-healing systems for fault tolerant platform engineering: Case studies and challenges. Journal of Knowledge Learning and Science Technology, 2(2), 327-338. https://doi.org/10.60087/jklst.vol2.n2.p338
  15. Kephart, J. O., & Chess, D. M. (2003). The vision of autonomic computing. Computer, 36(1), 41-50. https://doi.org/10.1109/MC.2003.1160055
  16. Lei, L., Tan, Y., Zheng, K., Liu, S., Zhang, K., & Shen, X. (2020). Deep reinforcement learning for autonomous internet of things: Model, applications and challenges. IEEE Communications Surveys & Tutorials, 22(3), 1722-1760. https://doi.org/10.1109/COMST.2020.2988367
  17. Manju, & Srivastav, V. K. (2025). Review of self-healing IoT networks based AI-driven fault detection and recovery. International Journal of Applied and Behavioral Sciences, 02(01), 230-244. https://doi.org/10.70388/ijabs250121
  18. Mann, H. B., & Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statistics, 18(1), 50-60. https://doi.org/10.1214/aoms/1177730491
  19. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., & Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529-533. https://doi.org/10.1038/nature14236
  20. Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., & Dormann, N. (2021). Stable-Baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22(268), 1-8.
  21. Riegler, M., Sametinger, J., & Vierhauser, M. (2023). A distributed MAPE-K framework for self-protective IoT devices. In: Proceedings of the IEEE/ACM 18th Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS) (15-16 May 2023, pp. 202-208), Melbourne, Australia. https://doi.org/10.1109/SEAMS59076.2023.00034
  22. Rosenberger, J., Urlaub, M., Rauterberg, F., Lutz, T., Selig, A., Bühren, M., & Schramm, D. (2022). Deep reinforcement learning multi-agent system for resource allocation in industrial internet of things. Sensors, 22(11), 4099. https://doi.org/10.3390/s22114099
  23. Sinha, S. (2025, October 28). State of IoT 2025: Number of connected IoT devices growing 14% to 21.1 billion globally. IoT Analytics Research. URL: https://iot-analytics.com/number-connected-iot-devices/
  24. Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd Ed.). MIT Press.
  25. Towers, M., Kwiatkowski, A., Terry, J., Balis, J. U., De Cola, G., Deleu, T., Goulão, M., Kallinteris, A., Krimmel, M., KG, A., Perez-Vicente, R., Pierré, A., Schulhoff, S., Tai, J. J., Tan, H., & Younis, O. G. (2024). Gymnasium: A standard interface for reinforcement learning environments. In: 39th Conference on Neural Information Processing Systems (NeurIPS 2025). https://doi.org/10.48550/arXiv.2407.17032
  26. Vankayalapati, R. K., Pandugula, C., Ganti, V. K. A. T., & Mishra, G. (2022). AI-powered self-healing cloud infrastructures: A paradigm for autonomous fault recovery. Migration Letters, 19(6), 1173-1187.
  27. Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S. J., Brett, M., Wilson, J., Millman, K. J., Mayorov, N., Nelson, A. R. J., Jones, E., Kern, R., Larson, E., … van Mulbregt, P. (2020). SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nature Methods, 17(3), 261-272. https://doi.org/10.1038/s41592-019-0686-2
  28. Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine Learning, 8(3-4), 279-292. https://doi.org/10.1007/BF00992698
  29. Younis, M., & Akkaya, K. (2008). Strategies and techniques for node placement in wireless sensor networks: A survey. Ad Hoc Networks, 6(4), 621-655. https://doi.org/10.1016/j.adhoc.2007.05.003
  30. Zinn, J., Vogel-Heuser, B., & Gruber, M. (2021). Fault-tolerant control of programmable logic controller-based production systems with deep reinforcement learning. Journal of Mechanical Design, 143(7), 072004. https://doi.org/10.1115/1.4050624