China Unicom rebuilds its big data analytics platform in the cloud, realizing over US$1 billion gains

• Who: China Unicom & BONC
• What: Implemented a single, centralized cloud-based analytics platform for all of its 24 departments in the parent group, 31 provincial branches, 358 municipal branches and 26 subsidiaries
• How: Redesigned the platform using various elements of TM Forum’s Open Digital Framework, including Open APIs and the Business Process and Application frameworks, as well as real-time data streaming enabled by Apache Kafka
• Results: Daily operational analysis reports arrive seven hours earlier; production efficiency up 70%; accumulated net profits up 458% in first year

Like most communications service providers (CSPs) around the world, China Unicom – one of China’s ‘big three’ mobile operators – is keen to harness the power of big data analytics to improve efficiencies and gain valuable and actionable insights.

However, China Unicom also had a somewhat unique challenge in the sense that it’s not simply a single operator with a single network. China Unicom is a complex organization comprising 24 departments in its parent group, along with 31 provincial branches, 358 municipal branches and 26 subsidiaries, all with huge databases and specific needs for analyzing all that data.

The other key challenge is that the size of its operations also meant Unicom was becoming increasingly swamped with data.

“With the widespread use of 4G networks, the data has grown explosively,” says Wang Zhijun, Vice General Manager of Information Technology Department, China Unicom. “The scalability, service performance and other aspects of the oldsystem were unable to meet our requirements of business processing efficiency.”

Rather than a simple upgrade of its analytics capabilities, Unicom decided to embark on something more ambitious: a centralized analytics platform that could serve the needs of every department, every branch and every subsidiary of the business, which would improve efficiencies across the organization and keep the cost of usage low.

In preparation for the project – billed as an “enterprise-level big data analysis platform” – Unicom identified several key challenges that it wanted this centralized platform to address:

Distributed cloud architecture Unicom started by adopting a distributed cloud architecture, using Apache Spark as the computing engine and a vectorized computing module to enable the platform to crunch massive amounts of data faster and more efficiently. That’s key because as mentioned above, the platform integrates data from across all of the “BOME” domains – that’s a ginormous amount of data, says Wang.

• How to access the latest data and generate daily operational analysis reports as early as possible
• How to aggregate and unify all operation and management data across four key domains: business support systems (BSS), operation support systems (OSS) management support systems and e-commerce systems (which Unicom refers to collectively as the BOME domains)
• How to respond quickly to analysis requirements via things like artificial intelligence (AI) functions, online self-support capabilities and component-based agile applications.

China Unicom's Wang Zhijun “On a daily basis we are now processing data volumes of around 17 Terabytes,” he says. “So the centralized big data analytics platform has to be capable of analyzing billions of records in just seconds. This enables more high-value and empowered decisions for business management, marketing management, business management, network operations and customer perception.”

Wang adds that Unicom has achieved this panoramic analysis of cross-domain data via a combination of technologies like big data mining, AI and geographic information systems (GIS).

GIS is a key component because Unicom’s organizational structure is not only complex, but spread across China’s vast geography.

“With the help of GIS, we have better visualization of large scenes using heat maps,” Wang explains. “It also enables us to combine accurate geographic location with business data for our millions of network base stations. We can also use a spatial search engine to search massive spatial data, which is convenient for accurate analysis and marketing assistance.”

While the cloud architecture is distributed, the platform itself is designed to be centralized at an ultra-large-scale, which not only saves on construction costs, but also facilitates the rapid development of cross-domain integration and centralized analysis of big data across the group.

“On the platform, all of the group’s data can be viewed, cross-domain data can be analyzed intelligently, abnormal data can be flagged early, information can be accessed efficiently and users can improve their analysis skills through recommendations and the accessible knowledge base,” Wang says.

The platform also enables real-time data streaming (using Apache Kafka) to support multi-dimensional real-time data monitoring and analysis for apps such as 5G and number portability. With the page refresh frequency at 1 minute/time, users can observe business management development and off-grid trends as they happen.

China Unicom's enterprise-level big data analysis platform adopts a microservices architecture based on TM Forum’s Open Digital Framework such as the Business Process Framework (also called eTOM) and Application Framework (also called TAM). The data warehouse creates more than 100,000 data tables based on the Information Framework (SID) and meets the Forum’s best practice requirements for data application presentation. The platform’s sharing interfaces use TM Forum Open APIs, including the Forum’s API Design Guidelines and specifically the Service Inventory API, Service Ordering API and Service Problem API.

Faster report generation

China Unicom’s platform project took around one year to complete – work began in May 2017, and the platform went live in May 2018. Each and every one of Unicom’s departments, provincial branches and subsidiaries can access the platform, which is currently serving 10,000 PC users with 200,000 page views per month and 8,000 mobile users with 30,000 page views per month.

The first and most obvious benefit of the revamped analytics platform is that daily operational analysis reports are available first thing in the morning rather than at the end of the business day. “Previously, for example, we would have to wait until the end of Tuesday to receive the daily operational analysis reportfor Monday, but now because we are processing data more efficiently, we can receive that same report at 9:00am Tuesday, even though we are now processing more data than we were before,” Unicom’s Wang explains.

Moreover, he adds, the platform has improved production efficiency by 70%, helping business departments to analyze early, deploy early and act early. “The marketing department and customer service department can carry out more targeted marketing, customer maintenance and other operations, including new customer development, old customer maintenance, existing customer production package upgrade and so on, to enhance our competitiveness and improve profit.”

In fact, Unicom credits the new analytics platform with boosting its operating performance a whopping 458% year-on-year to RMB10.2 billion (around US$1.5 billion) in annual accumulated net profits in 2018, and up another 12% year-on-year to RMB9.8 billion (around US$1.4 billion) as of September 2019.

The multidimensional self-service reports enabled by Apache Kylin (a big data analytics engine with online analytical processing [OLAP] on Hadoop) has cut the cost of service by 50%, and currently covers 53% of self-service demands of Unicom’s business divisions – which for now comprises monitoring and analysis of key businesses, Wang says. “Because new businesses keep emerging, the statistical caliber is diverse, and the self-service model construction has a certain period, it is impossible to cover current business demands 100%,” he explains. “At present, we are doing our best to improve the self-service demand coverage rate, which is planned to reach 70%, and continue to reduce the cost of manual service.”