Agent-Based End-Host Monitoring in IPv6 SDN Intranets: A Hybrid Machine Learning Approach by using UEBA Framework

Abstract

SDN architectures prioritize centralized network control but often lack visibility into host-level activity, resulting in blind spots for insider threats and stealthy attacks. A key challenge lies in developing a scalable, secure, and low-overhead host-monitoring solution that integrates seamlessly with SDN infrastructures. This paper addresses that gap by proposing an agent-based end-host monitoring architecture designed for security-critical SDN-enabled IPv6 intranets. Our method involves deploying lightweight monitoring agents on each host, which collect system and process metrics as time-series data and transmit them securely to a centralized analytics engine via mTLS-protected gRPC streams. The core analytical framework relies on User and Entity Behavior Analytics (UEBA), a behaviorbased cybersecurity paradigm that establishes multidimensional behavioral baselines for users, machines, and processes. UEBA employs machine learning techniques to detect deviations from these baselines, capturing both abrupt and subtle anomalies. In our implementation, the system adopts a hybrid detection pipeline that combines the isolation forest and the LSTM autoencoders. Detected anomalies are relayed to the SDN controller through its northbound APIs, enabling dynamic enforcement of flow rules and real-time response. Evaluation in a controlled testbed confirms that the system maintains low CPU and memory overhead per agent while achieving F1-scores of 0.70 and 0.799 for hardware and software-level anomalies, respectively. With sub-second telemetry latencies and near-realtime anomaly signaling, the proposed architecture demonstrates practical feasibility for deployment in security-critical SDNmanaged environments.

Keywords

Acknowledgments

Supported by JSPS KAKENHI (Grant No. 23K05416) and University Grants Commission, Nepal (CRG-078/79-ENgg-01).