Big Data or Smart Data? The RapidMiner Perspective

Share on twitter
Share on facebook
Share on linkedin

What’s the big deal about big data? 

Sure, it’s got those 3 Vs Gartner talked about – Volume, Velocity, and Variety. But is that really the best way to think about what big data can do for you?  I think that the things worth mentioning are these:

1. Yes, the volume.  It’s BIG – hard to store, hard to retrieve. That’s actually the least interesting thing about it, though. The technology needed to store and retrieve this massive amount of data is coming along quickly, building on past developments like relational database systems and storage clusters. Now NoSQL databases and Hadoop are considered solid solutions for distributed data storage and retrieval, and I predict that they’ll see use in more and more companies as underlying data delivery frameworks.

2. It can be structured AND unstructured.  That adds another difficulty on top of storage and retrieval: organization. Big data is fundamentally hard to organize.  You need text analytics to transform unstructured information like text, images, or videos into structured data.

That leads me to the main point:

3. Big data alone has no value! To extract value from it, you have to employ advanced analytics. You can only unlock the true power of big data when you learn how to transform it into smart data, data that helps you understand why the world works as we know it.

It takes advanced analytical methods to find the correlations and underlying patterns that help us understand the present and predict the future.  If a company understands what customers will quit their contracts, or when a machine will break down, they can be proactive, solving problems before they occur and making decisions that will guard the company’s interests long-term.

Everyone seems to be trying to figure out how best to tap into Big Data to achieve their goals. I think we’ve found the answer: the true power of big data lies in creatively using advanced analytics to transform it into smart data.  Is the world ready to move forward with this goal? I certainly hope so!

Additional Reading

Ingo Mierswa

Ingo Mierswa

Ingo Mierswa is the founder and president of RapidMiner and an industry-veteran data scientist since starting to develop RapidMiner at the Artificial Intelligence Division of the TU Dortmund University in Germany. Mierswa, the scientist, has authored numerous award-winning publications about predictive analytics and big data. Mierswa, the entrepreneur, is the founder of RapidMiner. Under his leadership RapidMiner has grown up to 300% per year over the first seven years. In 2012, he spearheaded the go-international strategy with the opening of offices in the US as well as the UK and Hungary. After two rounds of fundraising, the acquisition of Radoop, and supporting the positioning of RapidMiner with leading analyst firms like Gartner and Forrester, Ingo takes a lot of pride in bringing the world’s best team to RapidMiner.