Recently, I was invited to present on the impact of big data and advanced analytics on the insurance industry at the NCSL Legislative Summit. This talk couldn’t have been timelier, as the insurance sector now tops the list of most disrupted sectors. Some of the culprits and causes for this top spot are related to the speed of technological change, changing customer behavior, increased investments in the insurtech sector and new market entrants, such as homeowners and renters insurance startup Lemonade. A significant driver of this disruption is technological change – especially in big data and advanced analytics.
See also: Why to Refocus on Data and Analytics
Here are 10 key trends that are affecting big data and advanced analytics – most of which have a hand in disrupting the insurance industry:
- Size and scope – Big data is getting bigger and faster. With connected cars, homes and buildings, and machines, the amount of data is increasing exponentially. Investments in IoT and Industrial IoT, 5G and other related areas will only increase the speed and amount of data. With this increased volume and velocity, we will not be able to generate meaningful insights from all of this data without advanced analytics and artificial intelligence.
- Big data technology – Big data technology is moving from Hadoop to streaming architectures to hybrid “translytical” databases. While concepts like “data lakes” and NoSQL databases mature, new technologies like Apache Spark, Tez, Storm, BigTop and REEF, among others, are creating a constant flow of new tools, which adds to a sense of “big data in flux.”
- Democratization – The democratization of data, business intelligence and data science is accelerating. Essentially, this means that anybody in a given organization with the right permissions can use any dataset, slice and dice the data, run analysis and create reports with very little help from IT or data scientists. This creates expectations for timely delivery, and business analysts can no longer hide behind IT timelines and potential delays.
- Open source movement – The open source revolution in data, code and citizen data scientist is accelerating access to data and generation of insights. Open source tools are maturing and finding their way into commercial vendor solutions, and the pace of open source tool creation is continuing unabated; the Apache Software Foundation lists more than 350 current open source initiatives. This steady stream requires data engineers and data scientists to constantly evaluate tools and discover new ways of data engineering and data science.
- Ubiquitous intelligence – Advanced analytics – especially various types of artificial intelligence areas (reference to my AI report post) – is evolving and becoming ubiquitous intelligence. AI can now interact with us through natural language, speak to us, hear us, see the world and even feel objects. As a result, it will start seamlessly weaving itself into many of our day-to-day activities, such as using a search engine or sorting our email, recommending things to buy based on our preferences and needs, seeing the world and guiding us through our interaction with other people and things without our even being aware of its doing so. This will further heighten our sense of disruption and constant change.
- Deep learning – Deep learning, a subset of the machine learning family (which itself is just one area of AI), has been improving in speed, scale, accuracy, sophistication and the scope of problems it addresses. Unlike previous techniques, which were specific to the different type of data (e.g., text, audio, image), deep learning techniques have been applied across all different types of data. This has contributed to reduced development time and greater sharing and broadened the scope of innovation and disruption.
- MLaaS – Machine learning, cloud computing and open source movement are converging to create Machine Learning as a Service (MLaaS). This not only decreases the overall variable costs of using AI but also provides large volumes of data that the machine learning systems can further exploit to improve their accuracy, resulting in a virtuous cycle.
- Funding – Big data funding peaked in 2015. However, funding for artificial intelligence, especially machine learning and deep learning, has continued to attract increasingly significant investments. In the first half of this year, more than $3.6 billion has been invested in AI and machine learning. This increased funding has attracted great talent to explore difficult areas of AI that will be disruptors of the future economy.
- Center of Excellence: As organizations continue to obtain good ROI from their initial pilots and proof-of-concepts in analytics, automation and AI efforts, they are increasingly looking toward setting up centers of excellence where they can train, nurture and grow the talent. The exact role of the center changes based on the overall organizational culture and how the rest of their business operates – centralized, federated or decentralized.
- Competitive landscape – The big data landscape continues to grow, and the AI landscape is expanding rapidly. Deep learning companies are growing the fastest across multiple sectors. Competition among startups – as well as incumbents that want to stay ahead of potential disruption – is creating a vibrant ecosystem of partnerships and mergers and acquisitions that further the disruptive cycle.
