Q-Learning-Driven Adaptive Intrusion Detection for Malicious Node Classification in Mobile Ad-Hoc Networks

Abstract

Mobile Ad-hoc Networks (MANETs) remain highly exposed to routing-layer attacks such as Sybil, Wormhole, Blackhole, and Denial-of-Service (DoS) due to their decentralised structure and constantly shifting topology.  Intrusion detection systems built on static machine learning models struggle to keep pace with these shifting attack patterns, leading to a growing number of missed detections as network conditions change. This work proposes an adaptive intrusion detection framework that embeds Q-Learning reinforcement within an Artificial Neural Network-based Intrusion Detection System (ANN-IDS), with classification finalised by a Random Forest ensemble. The Q-Learning agent refines its detection policy through a composite reward signal that jointly rewards true positive rate and packet delivery ratio while penalising false positives and end-to-end delay. Tested across four attack types (Sybil, Wormhole, Blackhole, and DoS) and eight node density settings (25 to 200 nodes) in NS-3, the framework attains an overall accuracy of 99.21%, precision of 98.51%, recall of 99.68%, and an F1-score of 99.09%. The agent's detection policy stabilised within roughly 140 training episodes and sustained this performance at the 200-node scale. Across every evaluated condition, the proposed approach surpasses Static ANN-IDS with Random Forest, Support Vector Machine, K-Nearest Neighbour, and an unprotected MANET baseline, supporting its use in large, dynamic MANET deployments.