How AT&T and Verizon are scaling AI

AT&T and Verizon have learned much since they first rolled up their sleeves to evaluate the business value of GenAI more than two years ago, as executives from AT&T and Verizon discussed during keynotes at TM Forum’s Innovate Americas.

One important lesson is that AI has shown its worth in pilots. Making it scale, however, can be another matter.

Within the industry “we see a challenge, a big chasm in terms of shifting from successful pilots ... and then on the other side, operationalizing at scale,” says Sudharsan Srinivasan, Senior Vice President, Technology, Security, Governance & Infrastructure Verizon, during his presentation at Innovate Americas.

And as Andy Markus, Chief Data Officer, AT&T points out during his discussion with Nik Willetts, CEO, TM Forum: “It's hard to get value unless you truly achieve scale.”

Both AT&T and Verizon have built internal AI platforms, which have delivered operational benefits and helped identify pitfalls. It's a route other large telcos have taken. Orange, for example, built Live Intelligence as an internal platform to avoid the mushrooming use of shadow GenAI applications. It proved to be such a popular problem-solving and optimization tool with 60,000 of Orange’s employees that the company made it a commercial product.

AT&T’s GenAI and agentic AI platform is called Ask AT&T and more than 100,000 users across the company have access to the tool, according to Markus.

“There are over 750 million API calls to the system, driving the things that truly matter. We're doing it at a scale where now we're probably averaging about 5 billion tokens a day,” said Markus.

Success, however, can come with a price. “At that scale, using commercial LLMs (large language models) becomes a little bit daunting,” pointed out Markus.

During the early days of GenAI experimentation the emphasis was on making the technology work. However, as the company progressed “we really started focusing on cost. How do we drive cost down and become more efficient? How do we move to the SLM (small language model) approach over an LLM approach?”

AT&T now uses proprietary knowledge to accurately fine tune SLMs and “when we do that, we're seeing almost a 90% reduction in cost,” stated Markus.

The company also carefully calculates ROI for use cases. “We estimate the compute, the token cost, and the data storage as part of that business case … so we're looking at a fully encompassed ROI,” explained Markus.

“We have a measure of 2x ROI for every dollar we invest of free cash flow impacting value in [the] year. And these are multiple year business cases. So that 2x will grow to 4, 5 ,6, X over the life of the business case,” according to Markus.

Building on flexible foundations

ROI is not the only area where telcos can trip up. Operating AI at scale demands a flexible, composable IT architecture, according to Verizon’s Srinivasan.

“As you dream up new AI experiences … [if] the architecture is not flexible to incorporate [them] you will have to go back to the drawing board. So, modernizing the stack is one thing, but having a composable IT architecture behind it is essential,” said Srinivasan. Verizon was accredited as 'Running on ODA' by TM Forum earlier this year, added Srinivasan. "There was a deliberate reason for that," he explained. "When we talk about operating AI at scale, [ODA] is an essential building block for us, and that is one of the reasons why we went down that path."

Both companies stress the importance of TM Forum members collaborating to ensure the industry can make good use of accurate data.

Markus, for example, believes an important question for the industry is “how do we create telco-like data that can create telco-like use cases without using live customer data?”

AT&T's answer is to create synthetic data, which he suggested will “be super important to drive the TM Forum mission" and notably as telcos collaborate on agentic AI.

“We all want to work together but really can't contribute our own data. It’s our customers' data, not the company's data. So we have to think synthetically. “

In the meantime, AT&T and Verizon are already busy working on AI agents.

“We are now shifting towards autonomous agents. We are moving thinking beyond just applications and the capabilities within the application to how do we truly create a 5x, 10x version of our employees by leveraging agents to elevate the overall capability,” said Verizon’s Srinivasan.

Much of the groundwork has already been done, according to Markus.

Agentic AI “just builds on all the great things that we've done to date … traditional AI, generative AI rules, and human decisions. It's all baked in there. But it does allow us to take more complicated use cases, solve them more accurately, and embed more automation to the process," he said.

What Markus would like to see is inter-agent collaboration, a concept that Vodafone has worked on within TM Forum’s Innovation hub.

“The true magic is going to be how we take those agents and connect them to agents that are outside our ecosystem. So, it could be a SaaS platform. Could be a CSP, it could be just another vendor … to make these agents … interoperable and have them be able to communicate.”

Watch these and other sessions from Innovate Americas.