Measuring the Effectiveness of Autonomous Network Functions that Use Artificial Intelligence Algorithms

  • Premnath K. Narayanan BDGS SA OSS PDU OSS S&T RESEARCH & PCT LM Ericsson Ltd., Athlone, Ireland
  • David K. Harrison Glasgow Caledonian University, Glasgow, United Kingdom
Keywords: Network assurance, AI, ML, counters, stochastic algorithms, KPIs


Autonomous network functions such as Software Defined Networks (SDN), Self-Organizing Networks (SON), and virtual function network orchestrator plays a crucial role in 5G and beyond 5G wireless telecommunication network. Advancements in Artificial Intelligence (AI), Machine Learning (ML) algorithms, and frameworks have led to adequate adaption of stochastic algorithms for autonomous network functions, aimed at performing better than human capability. Measuring the effectiveness of such autonomous network functions is a challenge since stochastic algorithms are fundamentally generalized models and could potentially make malicious proposals. Traditionally effectiveness of network is measured through network assurance Key Performance Indicators (KPIs). Autonomous network functions are kept active when the KPIs are in the acceptable limit, and the network is showing improvement over time. This paper introduces

• Factors that are to be considered beyond KPIs for effective measurement of autonomous network functions that use stochastic algorithms.

• Adopting the right scales for measuring the effectiveness of autonomous network functions using the grading system from medical practices which are used for the treatment of critical illness.


Download data is not yet available.

Author Biographies

Premnath K. Narayanan, BDGS SA OSS PDU OSS S&T RESEARCH & PCT LM Ericsson Ltd., Athlone, Ireland

Premnath K. Narayanan is a seasoned Software Engineer (System Engineering, software architecture and development) with 22 years of practical experience in realizing Commercially of the shelf (COTS)/cloud products for ICT (Information and Communications Technology) industry. He has designed and developed products, trained users and mentored employees. Working as a master engineer at Ericsson primarily focused on researching and developing autonomous network functions for telecommunication network products.

David K. Harrison, Glasgow Caledonian University, Glasgow, United Kingdom

David K. Harrison is currently Professor of Design and Manufacturing at Glasgow Caledonian University where he has held a range of managerial roles. He has spent his working career in manufacturing industry or industry facing academia. A graduate of UMIST, he has edited several books and conference proceedings and has published his work widely. He has supervised 81 PhD students through to graduation. Around half of these students have been based outside the United Kingdom.


Scott Mayer McKinney, Marcin Sieniek, Varun Godbole, Jonathan Godwin, Natasha Antropova, Hutan Ashrafian, Trevor Back, Mary Chesus, Greg C. Corrado, and Ara et al. Darzi. “International evaluation of an ai system for breast cancer screening”. Nature, 577.7788 (2020), pp. 89–94. DOI: 10.1038/s41586-019-1799-6.

Premnath K Narayanan and David K Harrison. “Explainable AI for Autonomous Network Functions in Wireless and Mobile Networks”. International Journal of Wireless and Mobile Networks, 12.3 (2020), pp. 31–44. DOI: 10.5121/ijwmn.2020.12303. URL:

Scott M. Lundberg and Su-In Lee”. “A Unified Approach to Interpreting Model Predictions”. In: 31st Conference on Neural Information Processing Systems. NIPS, 2017.

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. “Why Should I Trust You?” Explaining the Predictions of Any Classifier”. In: 22nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 2016.

Gregory Plumb, Denali Molitor, and Ameet Talwalkar. “Model Agnostic Supervised Local Explanations”. In: 32nd Conference on Neural Information Processing Systems. NeurIPS, 2018.

CJ Willmott and K Matsuura. “Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance”. In: Climate Research 30 (2005), pp. 79–82. DOI: 10.3354/cr030079.

Lonnie Magee. “R2Measures Based on Wald and Likelihood Ratio Joint Significance Tests”. In: The American Statistician 44.3 (1990), pp. 250–253. DOI: 10.1080/00031305.1990.10475731.

Arnaud de Myttenaere et al. “Mean absolute percentage error for regression models”. Neurocomputing 192 (2016), pp. 38–48. DOI: 10.1016/j.neucom.2015.12.114.

Shen Yi. Loss Functions for Binary Classification and Class Probability Estimation, 2005. URL:

2018. URL:

Peter J. Rousseeuw. “Silhouettes: A graphical aid to the interpretation and validation of cluster analysis”. Journal of Computational and Applied Mathematics 20 (1987), pp. 53–65. DOI: 10.1016/0377- 0427(87)90125-7.

2019. URL:

Diego Kreutz et al., “Software-Defined Networking: A Comprehensive Survey”. In: Proceedings of the IEEE 103.1, pp. 14–76. DOI: 10.1109/jproc.2014.2371999.

Graham Teasdale and Bryan Jennett. “Assessment of Coma and Impaired Consciousness”. In: The Lancet 304.7872 (1974), pp. 81–84. DOI: 10.1016/s0140-6736(74)91639-0.

Florence C.M. Reit et al. “Differential effects of the glasgow coma scale score and its components: An analysis of 54,069 patients with traumatic brain injury”. In: Injury 48.9 (2017), pp. 1932–1943. DOI: 10.1016/j.injury.2017.05.038.

Mieczyslaw Finster and Margaret Wood. “The Apgar Score Has Survived the Test of Time”. In: Anesthesiology 102.4, pp. 855–857. DOI: 10.1097/00000542-200504000-00022.

Anders Bjartell. “The 2005 International Society of Urological Pathology (ISUP) Consensus Conference on Gleason Grading of Prostatic Carcinoma”. In: European Urology 49.4 (2006), pp. 758–759. DOI: 10.1016/j.eururo.2006.02.007.

Premnath K Narayanan and David K Harrison. “BIAS mitigation methods for autonomous network functions in telecommunication networks (a presentation)”. In: Wireless World Research Forum. Wireless World Research Forum, 2019. URL:

Premnath K Narayanan and David K Harrison”. “Measuring the Effectiveness of Autonomous Network Functions that Use Artificial Intelligence Algorithms (a presentation)”. In: Wireless World Research Forum. Wireless World Research Forum, 2020. URL: