Big Data or Smart Data? The RapidMiner Perspective

Share on twitter
Share on facebook
Share on linkedin

What’s the big deal about big data? 

Sure, it’s got those 3 Vs Gartner talked about – Volume, Velocity, and Variety. But is that really the best way to think about what big data can do for you?  I think that the things worth mentioning are these:

1. Yes, the volume.  It’s BIG – hard to store, hard to retrieve. That’s actually the least interesting thing about it, though. The technology needed to store and retrieve this massive amount of data is coming along quickly, building on past developments like relational database systems and storage clusters. Now NoSQL databases and Hadoop are considered solid solutions for distributed data storage and retrieval, and I predict that they’ll see use in more and more companies as underlying data delivery frameworks.

2. It can be structured AND unstructured.  That adds another difficulty on top of storage and retrieval: organization. Big data is fundamentally hard to organize.  You need text analytics to transform unstructured information like text, images, or videos into structured data.

That leads me to the main point:

3. Big data alone has no value! To extract value from it, you have to employ advanced analytics. You can only unlock the true power of big data when you learn how to transform it into smart data, data that helps you understand why the world works as we know it.

It takes advanced analytical methods to find the correlations and underlying patterns that help us understand the present and predict the future.  If a company understands what customers will quit their contracts, or when a machine will break down, they can be proactive, solving problems before they occur and making decisions that will guard the company’s interests long-term.

Everyone seems to be trying to figure out how best to tap into Big Data to achieve their goals. I think we’ve found the answer: the true power of big data lies in creatively using advanced analytics to transform it into smart data.  Is the world ready to move forward with this goal? I certainly hope so!

Ingo Mierswa

Ingo Mierswa

Founder & President of RapidMiner