What’s Wrong with Big Data?

We know that the whole world led by tech sector is surfing the Big Data wave. Billions of bytes of data is being collected, cleaned up, indexed and mined by the time I will finish this sentence. According to Wikipedia, as of 2012, roughly 2.5 exabytes (2.5×1018 bytes) of data is getting collecting every day. The promises of mining that data range from cloud intelligence health care to driverless automobiles.

What could possibly be wrong with Big Data anyway?

There is nothing inherently wrong. But there are two problems I can point out.

  1. Big Data is too big. It is incomprehensible. Big Data remains infertile unless you can redact Big Data to Small Data, which can be summarized, visualized and put to use. Right there is the metaphorical big elephant of our Big Data living room. How do we handle that complexity? Google, Facebook and other tech giants are already scratching their corporate heads to unscramble this puzzle. Doable, but expensive and often could be erratic.
  2. secondly Big Data convinces us that we all can become mini-statisticians. “Show me the data” is the new mantra. We often mistake correlation to causation. With Big Data, this tendency can get even more accentuated.

There are plenty of avenues for making use of “small data” available to us. One of them is our on personal data. What do we eat, how much we exercise, what are our biometric readings etc. Data collection is going to be easier than ever with the emergence of wearable technology. If we could collect our own personal data and dive a little deeper, we can derive intelligence about ourselves and it can have a positive impact to our lives. On a personal level, it is more worthier a quest than crunching mega-billion abstract data points.

A big shout out to all those Life Logging ninjas! And to the good folks at Quantified Self.