AI needs a new networking core. Are we ready for it?

The demand on the network core to deliver and establish reliable and secure connectivity for end users is enormous. AI pushes boundaries but can also clog networks. Scaling GenAI deployment requires network and compute reconfigurations for balancing centralized training and rapid edge AI inference. How the network is built to support AI-enabled applications needs revisiting urgently – but are we ready?

Analyst firm Omdia predicts that by 2025, most network application traffic will involve AI content generation, curation, or processing. By 2030, it expects nearly 75% of network traffic to incorporate AI, with rapid video and image content growth. Bandwidth aside, AI imposes new requirements such as real-time response, lossless delivery, distributed east-west traffic exchange, compute intensity and high sensitivity to data privacy.

AI demands a different network design

AI has differing needs to traditional cloud computing applications. Real-time AI applications mimicking human decision-making processes require fast model inference which is infeasible with cloud-based architectures. The emergence of multi-model applications that require processing text, images, videos, audio, and sensory data – much generated at the edge – and centralization strains backhaul and backbone networks, imposing serious challenges regarding scalability and cost. Sensitive data used in the model training and inference raises privacy concerns. In addition, AI questions the robustness and reliability of today’s networks designed to tolerate re-transmissions.

In contrast, GenAI traffic is significantly different from typical web-based apps for which today’s networks are designed. GenAI request sizes are very large (up to 2 million tokens for Gemini 1.5 Pro model), fed with multi-modal data, resulting in long processing times. Take an example of an insurance use case which feeds medical data, car telemetry, video capture, geo-location, etc. to let the LLM assess a claim. Also, there is no ‘caching’ in GenAI; the content is dynamically generated for every request. No surprise that by 2025, Gartner predicts AI will be the top driver for infrastructure decisions due to the maturing AI market, resulting in a tenfold growth in compute requirements. Opening the floodgates to AI traffic will necessitate a network design rethink.

Building a future-ready platform

Network service providers must lay the foundations, standards, and roadmap for a network for AI that is distributed, scalable, secure, and energy efficient. This is why several TM Forum members are collaborating on an industry-specific data reference architecture, encircling both emerging AI-enabled business models and supporting networks. The TM Forum Modern Data Architecture for Telecom Operations Project comes from a need to modernize data architectures to support the rapid evolution of AI and help define how cutting-edge AI-enabled telecom operations should look.

A big consideration is bandwidth to transfer larger models with increasing real-time interactions. The network needs to be adaptable to different data types. Instead of network control and data planes, hypothetically, you might end up with an inference, fine-tuning, RAG and training plane.

AI has enormous possibilities, but responsible design is critical to handle sensitive data. Ethical AI techniques, such as federated and split learning, where AI is trained in multiple independent and decentralized locations, are used to respond to security and privacy concerns over one centralized location.

Future networks must expand cloud-centric architectures toward the edge, bringing LLM closer to data sources, enabling low-latency inference, improving data transfer by processing data locally, while maintaining user data privacy. Arguably such design opens opportunities for greener AI and overcomes the challenges of compute-intensive centralized models. This change of design paradigm from centralized cloud to edge represents a transformative approach for future networks acting as a catalyst to accelerate enterprise AI adoption.

Partnership through open platforms

We must invest in a resilient industry-wide AI network considering future business models and growth. We recommend platformizing secure and scalable networks and creating a consumer/producer economy among service providers, hyperscalers and other players via Open APIs. Co-creating software solutions and network function virtualization providing complete automation, alongside adaptability for AI traffic from edge to cloud.

Hyperscalers and telcos see AI at the edge as an innovation opportunity to add value, but each is missing a a cog. Telecom operators need access to cloud platforms, hyperscalers need a backbone and physical locations, including base stations. Even if operators control the edge stack, they need the hyperscalers to maintain a seamless, distributed, cloud computing estate. Partnerships accelerate the creation of AI networks.

A partnership ecosystem with an open platform approach using APIs as an ‘ecosystem orchestrator’ is pivotal in creating infrastructures to satisfy business objectives and AI. These heavyweight partnerships will accelerate innovation, develop new use cases, and encourage enterprise adoption.

Planning for the future

AI will augment human processes in every industry and for every job. The hyper-automated world powered by the new network core demands high resilience, availability, and ethical responsibility. However, the opportunities for collaboration and innovation are massive. Now is the time to start.