Telco industry further standardizes autonomous network assessment

Leading communications service providers (CSPs) and network vendors are collaborating to tighten standardization around how the industry evaluates levels of network autonomy, according to speakers at TM Forum’s ‘Masterclass: How to Assess Autonomous Network Levels – a Global Perspective’, held during Innovate Asia in Bangkok.

TM Forum members established the Autonomous Networks Project in 2019 to define what it means for networks to be “autonomous”. Since then, more than 60 CSPs worldwide have signed TM Forum’s AN Manifesto, and dozens are participating in TM Forum’s AN level evaluations. In addition, several are adopting AN Level 4 “high-value scenarios”, aiming to reach Level 4 for operational in some domains from 2025 to 2027, as the recently published AN journey guide Autonomous Networks: Level 4 industry blueprint– high-value scenarios explains in detail.

Autonomy refers to using intelligent systems to make independent decisions, with Level 4 AN defined as representing a major shift towards autonomous decision-making as opposed to automation processes defined by humans. The graphic below shows TM Forum’s six-level taxonomy for autonomous network evolution.

Investment confidence

As CSPs advance in AN adoption, however, they increasingly want “to track evolution of autonomous projects [and] regularly measure the level they are at,” using an agreed, industry-wide gauge, said Fernando Camacho, Product Manager, Huawei, speaking during the AN masterclass.

This will allow them to capture and benchmark their progress internally and in comparison to other service providers with greater confidence, which in turn promises to deliver business benefits like operational efficiency, improved customer experience and faster time to new revenue.

As speakers from China Telecom, Detecon, Ericsson, Huawei and TM Forum pointed out, CSPs’ main goal in adopting AN is to increase business value and operational efficiency. Those that can objectively evaluate and quantify the incremental financial and operational gains from AN are much more likely to make investments that result in either lower costs and / or new services.

A detailed examination of progress using an independent benchmark helps ground CSPs in hard business realities.

“If a capability reaches Level 4 or 5 but no one wants to use it or it costs too high, what's the point? asked Qian Bing, R&D Director, China Telecom.

He believes, for example, that an assessment of AN level “should not just focus on the degree of intelligence, but also consider change in customer metrics. For example, we should look at whether our user experience has improved, how many users utilize certain features and the user frequency,” he explained.

Objective evaluations also limit internal overestimations of progress, according to Bing.

“If we assess ourselves, then by 2025 I believe every CSP will reach Level 4, unless there’s external review,” said Bing. He believes that the “three main beneficiaries [of a standardized assessment], are first managers or leaders, who can get a clearer and more objective picture of AN level, rather than the results given by the staff. The next is our developers [who] can be clear … on the direction and … efficiency [goals], and last but not least … our frontline workers and our customers, finding the best in capabilities from [TM Forum] assessment to be more effective.”

A shared baseline

Common agreement on how to measure the progress of CSPs’ different Level 4 scenarios and use cases will make it easier for them to meaningfully share what they have learned and to draw on each other’s experiences. This in turn helps CSPs address uncertainty about where and whether they should focus AN development.

“The best way to reduce … risk and to unlock that investment is by pooling our knowledge,” pointed out Andy Tiller, TM Forum’s EVP, Member Products & Services.

The recently published AN journey guide explains that CSPs measure their AN level using an assessment questionnaire that evaluates cognitive activities in the network such as intent handling, awareness, analysis, decision and execution (IAADE), each of which contains a set of tasks to be measured. The questionnaire is based on the professional grading standards of SDOs such as 3GPP and ETSI with added weighting and detailed scoring criteria.

During 2024, two dozen CSPs including AIS, Antel Uruguay, China Telecom, Elisa, Globe Telecom, IOH Indonesia, MTN Group, Orange, stc, Telecom Argentina, Telefónica, Telkomsel and others that wished to remain anonymous, used the tool to assess their AN implementations of fault management in the RAN and core network.

Participants presented preliminary AN level evaluation results in June 2024 at DTW-Ignite. At this stage it was clear that the questionnaires needed refinement to enable normalized scoring.

As a result, TM Forum published two standardized AN Level Evaluation Tool (ANLET) questionnaires; one for fault management on the RAN and one for core networks. Most of the phase 1 participants and some new CSPs repeated evaluations using the new questionnaires.

AN solution library

TM Forum members now plan to put fault management, IP optimization and change management through the phase two process and add the resulting questionnaires to its AN level evaluation tool library.

“We'll gradually build a library of the questionnaires that are standardized and used by the wider industry to assess their AN levels,” explained Tiller.

In addition to generating and standardizing further questionnaires, including for high-value scenarios (use cases that combine an operation flow and a network domain like fault management on the RAN), TM Forum intends to package AN Level questionnaires with information on how to develop a solution in what Tiller described as “a library of reusable solution packages.”

TM Forum will also make training available to help CSPs’ employees evaluate and validate their progress.