Pity poor big data. Pundits of all shapes and sizes are increasingly denouncing big data efforts, characterizing them as a ‘for data geeks only’ sandbox exercise. And the topic is sliding toward the Trough of Disillusionment in Gartner’s Hype Cycle. Yet in several industries we see success stories — so why are some companies successful while others struggle?
The answer isn’t always a simple one, and to be sure there are clear obstacles to success, but there are some clear indicators of successful paths as well. We’ve been studying this area for nearly five years at TM Forum, and have seen many struggles with big data. Some of the barriers include:
- Immaturity of tools: though these have come a long way in the last few years
- Lack of skilled personnel: some companies started off looking for “unicorn” data scientists, who were few and far between, rather than dividing the role into multiple people.
- Executive and cross-organizational buy-in to data-driven management has been spotty
- Focus on technology: many companies started by playing with the tools, rather than trying to apply them to meaningful applications
- Difficulty extracting high integrity data from multiple data silos
- Making the business case for big data: many have struggled to make a case for the investment
The last point is particularly important. The driver of most IT investments is of course ROI — why spend the money if you don’t get a return of some type? Many could not make the case, and the organizational resistance and other factors just contributed further to this. Moreover, many business case ROIs are not necessarily straight forward. For example, was a company’s success during the last holiday due to that whiz-bang new analytics-driven offer management application, or to a general rise in demand. Or was it driven by improvements in service quality? Sometimes it’s tough to allocate credit for each factor.
What is more clear, though, is that some companies have positioned themselves for success and in fact have achieved degrees of success. So let’s focus on the characteristics of those companies.
- First and foremost companies need to focus on the business value of their IT investments. The ‘big data sandbox’ has a place, but to get management’s attention (and investment) and to create momentum, you need to produce at least promising results. Picking the right problem to solve is critical.
- Solicit and gain top management support. Without this, you will have a steep hill to climb. Successful big data initiatives require a data-driven culture, and this needs to start from the top to be effective.
- Manage your big data initiatives like a portfolio. As we said there’s a place for a sandbox, but that should not be your primary strategy. Starting with smaller but achievable projects can show promise and gain momentum. Acquiring increasingly available COTS analytics platforms can speed adoption. Using experienced consultants or integrators can boost project speed. Doing multiple projects with the intention of consolidation down the road can help (provided you purposely design for later consolidation).
- Continuous improvement is critical. Make sure you have a strong change management program. This is as much a cultural phenomenon of getting organizations to adopt a data-driven strategy as it is a technology strategy.
- Manage data as a corporate asset. Silos are a continuing impediment to success of any analytics initiative. You may never each Nirvana here, but at least find a way to continuously improve the integrity of the data.
- People are key. Finding or developing the right combination of technical ability, business acumen and intellectual curiosity are not easy, but really important if you are going to be successful, and not just for the IT organization. Talented business analysts and executives will need to interpret the data to put it to use. Acquire and nurture the right talent.
In summary, demonstrating a clear ROI for big data is not easy. Following the path of organizations achieving early success (and there are many across industries) can be very helpful in improving your chances of success.