Computerworld has released a study by Computing, a UK company, regarding the state of Big Data in the real world. Big Data Review 2015 holds not a lot of surprises, but certainly a lot of confirmation that Big Data tools need careful handling to get the right results. In the early days of Big Data there was a lot of experimentation just to understand the capability of the tools. There will certainly continue to be plenty of experimentation just given the nature of the process, but success depends on the business execution.
Computing’s 2015 findings show the rapid evolution of the industry based on a shift in the responses in survey questions from just last year. The number of survey respondents concerned about the different topics has grown significantly, suggesting that Big Data methodologies is rapidly moving into the mainstream. Nobody wants to be left behind, of course. In terms of tools, data warehouses and analytics databases were tied with cloud based Big Data services, broadly speaking. Big leaps in survey respondents considering these tools were seen.
This should come as no surprise to those practicing the art of Big Data. You probably spend most of your time importing, moving, scrubbing and preparing data for analysis. Indeed, just finding the data that is important to your analysis can take quite a bit of work. Garbage in / garbage out still applies. This report takes time to understand why projects have been perceived as successful, not wasn’t limited to just looking at the latest tools.
For instance, 76% of respondents focused on operational data to improve efficiencies. 24% used Big Data for external opportunities. Why? Pragmatic business decision making. The fastest route to Return on Investment will be to refine and improve operations that you already know. Operational savings go to the bottom line as increased profit. New sales only net what the gross margin will allow. Improving the gross margin impacts all sales. One of the frustrations that business leaders have with Big Data is the ability to speculate or predict future events when planning business spending.
The power of predictive analytics is immense. Descriptive analytics may allow you to refine existing operations with less risk but prescriptive analytics that model how things could be, allows you move to disruptive capabilities but higher risk of success. This is traditionally the domain of the entrepreneur, the ability to remake markets and disrupt the status quo. The larger challenge for existing companies is how they decide to manage risk and failure.
This tension between the business decision maker and the analytics professional has been true from the start, of course. The difference now is that decision makers have seen how analytics can improve their business, and funding for analytics is increasing based on that success. The most visible expression of this is the democratization of Big Data with more self-service tools for business professionals. The counter trend is reluctance by some departments to share data or cooperate with broader Big Data projects.
To be successful with Big Data projects, the survey identifies a number of factors, the top 3 being: 1) business buy in 2) knowing your data and 3) Core understanding of the business. The implications of just these three is important. Business is moving more aggressively into analytics, but with a purpose. High return on investment objectives will keep projects focused, and tends to lean to the operational side where there is a more likely return. Knowing your data is another critical aspect of a Big Data project. Integrating several sources of data and preparing it for analysis is not trivial and takes a lot of time and effort. Most survey respondents felt that between 80-90% data accuracy is “good enough” for most projects. The diminishing returns of further improvements may not make any changes to the decision making. Finally, this isn’t an academic exercise. The project’s success will depend on deep understanding of the business. A big part of analytics is deciding what problem you want to solve. A poorly formed premise or problem will lead to unsatisfactory results.
Computing’s 2015 report has some great information in it, and it highlights the changes in the industry just in the last year. As analytics becomes more mature it will find application in more companies and more projects, and that’s good for Big Data and the economy.
See the report at: http://resources.computerworld.com/ccd/assets/84539/detail
Big Data Perspectives 