Traditionally, Big Data has been geared towards three characteristics, famously expressed as the 3 V’s i.e Volume, Velocity, and Variety to support Structured, Semi-Structured and Unstructured Data. As businesses started adopting more of Big Data, new V’s aimed towards maintaining data emerged: Vulnerability, Volatility, Veracity, and Validity. Accurate management of these new V’s can ensure the underlying Big Data is risk-free, secure and valid.
Big Data is heavily employed in Analytics where they predominantly use Visualization / Business Intelligence and Variability (Dimensions). Classical Map-Reduce has enabled Big Data to process more copious amounts of Structured, Semi-Structured or Unstructured Data. However, Classical Map Reduce heavily relied on I/O which caused latency and could not support real-time data analytics. With the advent of In-Memory Engines like Apache Spark, Storm or Flink, data can now be processed in the memory, and it’s streaming data can be processed in real time. Hence an active field of Big Data based Architectures viz Lambda Architecture, and Kappa Architecture has been evolved to solve more complex Big Data problems like the Internet of Things.
However, businesses are increasingly ravenous for data. They are continually seeking to combine real-time analytical insights with old Data Sets to uncover new Insights. This Analytical FIEWdom (For(Data), If (Data), Else (Data), While (Data)) is ever on a Hyper Time Scale, i.e., a continuous process with multiple time-scales implicit.
Artificial Intelligence (AI) and Machine Learning (ML) techniques offer businesses more valuable insights and enables them to develop a knowledge-base that can be trained to automate decision-making and minimize frequent errors. This phenomenon has added a new set of V’s in the Big Data ecosystem namely Value, Visuals, Verbatim, and Voice which enables processing of Data concerning Numbers, Text, Speech or Images respectively.
Collectively, these 13V’s of Big Data (see figure) represent the Big Data Maturity Model that businesses need to adopt so that their organizations become more Data Driven and the basics of incorporating this involve Analytics/AI/ML, Big Data and Cloud.