AI can help data scientists generate hundreds or thousands of variations of models with different prediction features and create iterative simulations to finally choose the best variation. In fact, the best iterations involve both AI and data scientists.
Can artificial intelligence replace data scientists?
At Digital Transformation Asia 2018 in Kuala Lumpur this November, Pedro Uria Recio, Head of Axiata Analytics Center, Axiata Group will be discussing which AI technologies CSPs should invest in and implement. Read his pre-event blog below. Steve Urkel from the world-famous ABC’s sitcom Family Matters was not a data scientist because in the 90’s we did not call them that. But the nerdy teenage Urkel is no doubt the archetype of the data scientist. In one of the episodes, Urkel built a robot whose intelligence doubled every two minutes. This sounds pretty cool but was a terrible idea. In no time, Urkelbot overthrew its master and locked him in the basement… McKinsey estimates that about 64-69% of the total time spent on data collection and processing can be automated. As technology advances, many are questioning whether or not articifical intelligence (AI) has the potential to work faster and cheaper than data professionals in the near future. There are two main kinds of data professionals: Machine learning is by definition part of AI. Additionally, AI can automate many of the tasks that data scientists and data engineers perform. For starters, AI can be applied to the following tasks typically performed by data engineers: Moreover, AI can also automate some of the work of data scientists in the following ways:
The combination of AI with human problem-solving has actually empowered, rather than threatened, the jobs of data scientists.
AI can help data scientists generate hundreds or thousands of variations of models with different prediction features and create iterative simulations to finally choose the best variation. As a matter of fact, the best iterations involve both AI and data scientists. A dynamic, multifaceted decision process obtained through automation will outperform any single algorithm, no matter how advanced, by automatically testing, iterating and monitoring data quality; incorporating new data points as they become available; and making it possible to respond wisely to events in real time. Additionally AI can also assist data engineers preparing raw data, cleansing it and checking for correctness. This is not something that AI can handle entirely yet. It still requires human judgment to turn raw data into insights that make sense for a complex organization. AI cannot yet truly understand what specific data means for an organization, its business and the context of the industry. AI can help automate lower-level steps in data preparation and visualization, leaving data scientists to walk decision-makers through what the insights really mean. Lower-level tasks, which are typically performed by data engineers will be first impacted by AI For example, in the 80’s as computer programming languages advanced, demand for lower-level programmers did indeed decrease. However, demand for developers in general increased as companies adapted to these new languages. The same evolution is happening right now in analytics, with AI automating lower-level tasks. This leads to the transition of more complex, problem-solving tasks to humans. As a result, the combination of AI with human problem-solving has actually empowered, rather than threatened, the jobs of data scientists.
But data scientists and data engineers need to adapt. For starters, as AI automates lower-level data processing, data engineers will need to migrate towards data science. Even among data scientists, only the youngest have been trained in the more advanced deep learning approaches. As technology continues to advance, the skillset of data scientists will be rendered useless in 12 to 18 months. Data professionals will need to either learn new AI tools or to get left behind.
Instead of posing a threat to data science jobs, it is much more likely AI will become extremely intelligent assistants to data scientists, allowing them to run more complex data simulations than ever before. Analytical skills will soon be required in many more traditional roles. This transition is expected to create a new class of data scientist – let’s call them ‘citizen data scientists’ – that bridges the gap between business roles and strictly analytical roles. Fortunately our friend Steve Urkel was able to finally overthrow the tyrannical Urkelbot whose intelligence doubled every two minutes, just 22 sitcom minutes after being locked in the basement. Contrary to being a menace to humankind, at the moment when AI surpasses human intelligence, data scientists are likely to continue working with AI or develop newer AI systems.