DTWS: Avoiding AI pitfalls is all in the cards

The 'Laying the AI foundation' session on October 21 at the Digital Transformation World Series featured a panel made up of industry leaders from BT, IBM and Vodafone, and is available to watch on-demand now. As part of the Forum's AI, Data & Analytics theme, moderator Aaron Boasman-Patel, Vice President of AI and Customer Experience, TM Forum, led a fast-paced discussion on how operators can manage AI across the software lifecycle from procurement to decommission. Keeping AI from running amok has been a favorite sci-fi movie narrative for decades, but today it’s no joke for telecoms operators racing to automate their networks and keep up with the faster, better, cheaper demands of B2B and B2C services. Indeed, operators have a huge appetite for AI and want to know how to exploit it. But they lack the best practices and standards to avoid common traps and pitfalls, according to Rob Claxton, Chief Researcher, BT, who spoke on the panel during the event.

“It’s all new,” he said. “People are having to adjust to a world that’s slightly different from the world that they’ve become used to… There are lots of traps that need to be avoided, and the processes and best practices don’t yet exist.”

Checking it out

BT’s Claxton has been working closely with Boasman-Patel and TM Forum members A1 Telekom Austria Group, Amdocs, Axiata, Comarch, Ericsson, Ernst & Young, IBM, Microsoft, Netcracker, Orange, PCCW, Prodapt, STC, Tecnotree, Vodafone and Zenith to develop the best practices and processes operators require to be able to handle issues related to AI regulatory requirements, robustness, explainability and risks.

“What we wanted was literally something that practitioners could put in front of them and use day-to-day – a bit like the checklists you’d find in the cockpit of an aircraft,” said Claxton. “They help people across different parts of the lifecycle be prompted and reminded about [important tasks].”

The result is a new low-tech tool called AI Checklist Cards that offer guidance from procurement to development, deployment, and all the way through to end-of-life. These downloadable cards are available to TM Forum members and non-members, and are intentionally simple and easy to use. A new white paper about AIOps is also free to download. If cards aren’t your game, there is also a new AI Checklist Poster that summarizes the steps. The poster and the cards work hand-in-hand with TM Forum’s AI Readiness Check, an online tool that allows AI practitioners to identify gaps between current and target capabilities across six dimensions of a communications service provider’s (CSP’s) business.

Driving AI quality

TM Forum’s online AI Readiness Check – announced last year at Digital Transformation World in Nice, France – and its accompanying AI Readiness Check Poster are the first steps in starting adoption, followed by the AI Checklist Cards and poster and then AI Model Data Sheets coming early next year, according to Boasman-Patel. “Together they help drive the quality of AI management and governance practices,” he said.

“There are a lot of things we can learn from best practices,” said Michael Hind, Distinguished Research Staff Member, IBM Research AI Department, IBM, during the AI governance panel. “For the data scientists and modelers and people who deploy AI, they can focus on their expertise and use the checklist to [monitor] the governance.”

Another panelist, Soyini Taylor, Enterprise Data Architect, Vodafone Group, agreed with Hind on the helpfulness of the checklists.

“It’s important for us to be able to note down what we have done for explainability and fairness so that if I were to give my AI model to another team, or to an outside organization, they can open the box and see what’s in it without guessing.”

Claxton echoed this comment, noting that being able to document processes and steps taken in a standardized way is important for AI, especially for auditing and learning purposes. “This way we are learning about making the use of AI more effective and safer at the same time,” he said. You can watch the full session on-demand now. Watch the rest of Digital Transformation World Series content live and on-demand now too! Not registered for DTWS yet? There’s still time. Join 12,000 of your peers online through November 12. CSPs receive complimentary passes. Sign up here.