Big Data Analytics Trends for 2018

On this last day of an exciting year 2017, it is time to reflect what is in store for the New Year – 2018 for Big Data Analytics space. As a precursor to what is in store, it is great to see that the companies which adopted Big Data technologies as a result of the hype cycle and due to the fear of getting left behind are now thinking in terms of Value derivation from the investments. When it comes to getting value out of the data assets they have created over quarters and years now, the real excitement starts. So, at this juncture let us see where all of us (in the field of Big Data Analytics) are heading towards in 2018.

1. Cloud First Strategy – Most of the enterprises who have already built their Enterprise Data Hubs on-premise or somewhere in virtual shared space, will start migrating to “Data and Analytics as a Service” on cloud. The core technical competencies will be built by the implementation teams and the art of implementation of Analytics as a service on cloud will be mastered by most of the service vendors in the market (yes, a successful cloud implementation is an Art along with science). In terms of prominent players in the market, Amazon with AWS and Microsoft with Azure will definitely consolidate their positions but Google with it’s cloud platform will fully catch-up with front runners by end of 2018. The new enterprises who are initiating their Big Data Analytics initiatives in year 2018, will have Cloud First strategy instead of later migrating to Cloud.
2. Commoditization of ML / AI and evolution of “Data Science Engineer” – Once it was clear to the community that “Data Scientist” is the sexiest job of the 21st century, everyone wanted to become a Data Scientist (if not everyone, but most of us..). During the hype cycle, there were numerous articles on “Who is a Data Scientist” and a list of skills and expertise which qualify someone as a Data Scientist. Just like most of the new year resolutions, “I will become a Data Scientist in 2017” also could not become a reality for majority of the community (this is based on some first hand interactions). We also started seeing articles on “How to identify a fake Data Scientist during an interview“. While it is true that Data Scientists have a great role to play in the next evolution cycles of Data Analytics, it is a lot of hard work to really qualify as a Data Scientist. However, the bad news ends there !!! We are going to see commoditization of ML and AI knowledge. There will be numerous tools and APIs to utilize underlying algorithms and science part of the spectrum and easily leverage those within applications. The “Data Science Engineer” as a role will fully evolve in 2018 and will be one of the most sought after skill in the Big Data Analytics community.
3. AI in wrong hands – While we will see some amazing applications of AI within some of the critical use cases including cyber security, there will be some instances in 2018 where something as powerful as AI falls in wrong hands and our dependence on digital data assets (which is also a vulnerability) will be exploited. As a result, we are going to start to see evolution of AI within the boundaries of careful governance.
4. IOT centric implementations and maturity of Edge computing – With each passing day, we are equipping more machines to generate more and more data assets. While we have virtually unlimited storage and compute capabilities with Cloud, we are going to see a paradigm shift towards Edge computing. The storage and compute will move towards the machine or instrument which is generating data. As per Wikipedia “Edge computing is a method of optimising cloud computing systems by performing data processing at the edge of the network, near the source of the data. This reduces the communications bandwidth needed between sensors and the central data centre by performing analytics and knowledge generation at or near the source of the data. This approach requires leveraging resources that may not be continuously connected to a network such as laptops, smartphones, tablets and sensors. Edge computing covers a wide range of technologies including wireless sensor networks, mobile data acquisition, mobile signature analysis, cooperative distributed peer-to-peer ad hoc networking and processing also classifiable as local cloud/fog computing and grid/mesh computing, dew computing, mobile edge computing, cloudlet, distributed data storage and retrieval, autonomic self-healing networks, remote cloud services, augmented reality, and more.” The implementation patterns within Big Data Analytics space will evolve to leverage a combination of Edge Computing and Cloud Computing.
5. Amygdala Hijack phase is over – We are going to move away from impulsive, non-pragmatic and “me too” adoption phase of Big Data Analytics towards a more ROI and value driven approach within the enterprises in 2018 and beyond. We have passed the amygdala hijack phase of Big Data Analytics at the end of 2017.

In any case, 2018 sounds to be an exciting year for the Big Data Analytics space and I would like to wish a successful year for everyone who is part of this journey !!