by Rick Kawamura
Today’s big IT hype is “Big Data” — the massive volumes of digital information created and delivered by blogs, social networks, forums, financial transactions, emails, documents, log files and unlimited number of other sources across the web or behind the firewall. The Internet is a huge driver in the rapid growth of Big Data, allowing businesses the opportunity to consume and access information from B2B partners and suppliers’ portals, competitor websites, government web-based applications, and customer transactions and reviews.All this data significantly changes businesses by increasing profitability through process optimization, growing sales with predictive analytics based on buying behaviors, or saving cost by foreseeing changes in market conditions.
Big Data = Big Opportunities
There is plenty of anecdotal evidence of where data-driven decision making has made a significant impact. One of the better known is the movie “Moneyball“, based on the true story about how the low-budget Oakland A’s major league baseball team leveraged data analytics to extract intelligence and competitive advantage from years of historical data to build a championship team. But there are many other examples.
Shipping companies use data on delivery times and traffic patterns for route optimization, while retailers are increasing store sales by data-driven targeting to ultimately increase the number of visitors, number of visits, and spend per visit. Financial institutions are turning identified market trends to actionable predictors, allowing them to make significant economic gains by being the first to buy certain stocks before prices go up. According to Professor Erik Brynjolfsson from MIT's Sloan School of Management, data-driven decision making achieved productivity gains that were 5 to 6 percent higher than could be explained by any other factors. This type of productivity increase is significant enough to separate the winners from the losers in most industries.
It’s More than Just Big
In reality, despite all this big promise, many companies are taking a wait-and-see approach because of the enormity and complexity surrounding Big Data. According to Gartner Predicts 2012 research, more than 85 percent of Fortune 500 organizations will be unable to effectively exploit Big Data by 2015. What often gets lost in the Big Data discussion is that, in order to obtain substantial value, companies really need to access relevant data without getting overwhelmed with the need to collect and store every piece of data. The size (volume) can be daunting, but it’s often the integration of data from multiple sources and formats (variety) and the rapid real-time capture of the data (velocity) that contributes the most substantial value.
In a Big Data report from January 2012, Aberdeen Research shows that over the past three years, the number of unique data sources that companies manage is increasing. According to Aberdeen, Best-in-Class companies are those who successfully bridge the gap between their expanding sources of data and the analytical processes and systems they use to transform the data into timely business insight.
Focus on the Data that Matters
It is now possible to access and integrate data from virtually any source across the Internet or behind your firewall in CRM, product lifecycle management or ERP systems. According to research conducted by the Economist Intelligence Unit, “Data are so abundant and so readily available that [corporations] are having trouble keeping up, [but] there is a disconnect between the ability to collect data and the ability to base decisions on them.” Accordingly, 31 percent of survey respondents “admit they have no formal processes around data management but are loath to stop collecting them.”
Don’t get caught up in the volumes -- taking incremental, easily manageable steps when embarking on a big data project is perfectly acceptable and even recommended. Start by clearly outlining the objectives of your Big Data initiative. What data is needed, why, and who will use it. Think about what types of unique insights you are trying to get from the data and for what purposes: outwitting your competition, increasing sales through supply chain and procurement optimization, or growing revenues through trend analysis and predictive analytics. This will also impact the data sets required. Selecting the data needed can often be a challenge, but start by identifying several indicators that can have a strong influence on future performance. Just focus on the data that provides the most value, in-house or outside your firewall.
Let’s use, for example, the 250 million tweets sent per day that equal 8 terabytes of data. Only 1,000 of those tweets relate to your company or product brand. So you do not need to store and analyze all 8 terabytes every day. Extracting meaningful insights from data is much more dependent on the quality and relevancy of the data than on the quantity of the data.
Real-time Access and Relevancy
There is an unlimited and ever growing number of data sources that may be relevant to your organization and can add substance to any analytical effort. These include a long trail of social feeds, review sites and news sources, your cloud applications, as well as government web-based applications (Federal regulations, public data on housing, marriages, foreclosures etc.), channels, suppliers and competitor’s sites. A majority of these data sources are difficult to access and the data they contain is constantly changing.
You will need the ability to access a wide variety of data, and to access it in real-time. With a real-time integration platform, you can flexibly define and update your desired data sources and access any data you can see on a website. You can just as easily transform that data, perform an operation on it, and automate a resulting action.
Imagine a scenario where you could identify habits and purchasing patterns of your buyers and increase sales by heightening focus on items and categories that appeal to specific customer segments. Or turn blogs, forums and social media commentary into predictors for stock performance. Or perhaps get the right information into the hands of your partners or channels, thus enabling them to sell more of your products.
If you had the opportunity to automatically access any applications or web-based data source, load the data into another application, database, or other data store of your choice, how many strategic Big Data initiatives could you think of that will make a significant impact on your business growth?
Put Your Big Data into Action
Big Data is only valuable if it leads to meaningful actions. Irrespective of how big your data sets are, the key point is to extract intelligence from the data and then be able to take action based on the insights it provides. Therefore, being able to access relevant data, regardless of its source, is critical for any data mining effort. As an example, had the Oakland A’s simply monitored and identified trends but failed to act on them, they wouldn’t have been able to transform into the winning team they became.
For the organizations that will strategically embrace Big Data, the possibilities for innovation, business agility and increased profitability are endless. Don't be intimidated by the volumes; start your Big Data initiatives by focusing on the data that matters regardless of the data source, type or format in order to immediately start generating meaningful intelligence and taking action.
Rick Kawamura is the vice president of marketing at Kapow Software (Palo Alto, CA).www.kapowsoftware.com