Deutsche Telekom and SK Telecom target telco CX with large language model

Jonathan Abrahamson, Chief Product & Digital Officer at Deutsche Telekom, explains to Inform why the company is collaborating with long-standing partner SK Telecom to develop a telco-specific large language model (LLM) for digital assistants in customer service. And why there could be potential to sell it to other telcos.

The companies' LLM partnership is the first initiative of the "Global Telco AI Alliance", which Deutsche Telekom, e&, Singtel and SK Telecom announced in July.

An LLM underpins generative AI (GenAI) applications. Like many large communications service providers (CSPs), Deutsche Telekom has been working on artificial intelligence (AI) for several years. But the eruption of GenAI applications marks a sea change.

“It's difficult to envisage a world where this doesn't change everything and certainly in the context of what we do as a telecommunications company,” says Abrahamson.

But despite the impressive capabilities of off-the-shelf LLMs such as OpenAI, there is a “not insignificant amount of plumbing and orchestration that you have to build around the models to make them work for our [telco] context and … use cases,” explains Abrahamson. “What became very clear for us is to get really good with this … and to use it at massive scale we need to tune the model to make it work for our use cases.”

For a start, an LLM needs to understand the specific needs of a telco’s customers, and adhere to its brand, its tone of voice, and business rules, explains Abrahamson.

Off-the-shelf LLMs “have a huge amount of parametric knowledge which they've been learning from the open internet, but they don't know telco data all that well,” explains Abrahamson, adding: “What we're doing is sending the models to summer school to learn telco.”

Once an LLM is able to operate within the parameters of one telco business, it should be able to work in any other.

“We all have the [same] concept of prepaid or postpaid or broadband or TV," says Abrahamson. "This knowledge is in ChatGPT but … not to that level where it can give a perfect answer [in a telco context], and that's the sort of stuff we need to build in."

The partners will use Anthropic (Claude 2) and Meta (LLaMa2) to co-develop a multilingual tool for use in different countries, with the aim of unveiling the first version of the telco-specific LLM in the first quarter of 2024: Telekom Deutschland’s Ask Magenta service bot, which has been used as a digital assistant since 2016, will be an initial beneficiary.

Divining customer intent

An LLM’s reasoning capabilities mean it’s very good at analyzing customer intent and searching a telco’s back-end systems for the answer, explains Abrahamson. This is one of the reasons why it's well cut-out to help with customer support.

“When a customer says 'my internet machine doesn't work' a human would know it’s the router,” says Abrahamson. Traditional chatbots, however, would not – unless somebody physically writes a branch in a decision tree that states an internet machine equals router.

Deutsche Telekom’s chatbot Ask Magenta, for example, is a legacy system based on a long decision tree, where somebody has manually programmed each branch to answer a specific question.

“For the thousands of questions that a customer might ask, there are hundreds of thousands of things that could happen, and each of those [would have to be] … manually written line by line. Large language model replaces that [decision tree] architecture, to understand the customer intent and to craft the answer,” says Abrahamson.

Deutsche Telekom is already using the LLM to augment Ask Magenta's capabilities: today, for example, the chatbot is 80% dependent on a decision tree, and 20% dependent on an LLM.

Even for a large telco, developing an industry-specific LLM is a huge undertaking, however, which is why Deutsche Telekom values its partnership with SK Telecom.

Yet it is also because building a telco-specific LLM is so resource heavy that Abrahamson sees a potential to sell it to CSPs that lack the in-house capacity to build their own - even if for now its priority is to provide the LLM to operators within its own group.

Indeed, the task is such that “if this was available from someone, then I probably wouldn't go to the effort and expense of doing it,” says Abrahamson. Nonetheless, he also sees “massive value coming [from] this … and we like the idea of owning this IP. It's so fundamental for us.”