Get in Touch
Get in Touch


Right needle in right haystack: from ‘Big Data’ to ‘Big Insights’

August 21, 2019

Jayakrishnan P   

Director of Americas, Tata Communications

Finding the right approach to glean valuable insights from Big Data can be a daunting task but these steps may make your journey a whole lot easier. 

The great promise of Big Data is to help organisations find the proverbial ‘needle in the haystack’, to understand what their customer wants next before they even know they want it. By analysing ever-greater volumes of information we can know more, and become ever-more precise in the ways we identify the best new idea, new approach, or new solution.

“The challenge, of course, is that we don’t necessarily know what the ‘needle’ we’re looking for looks like. And just because we can find it today, does that mean we’ll still need it tomorrow? Or that the whole ‘haystack’ itself won’t get blown away by the winds of market change?”

These are the questions facing many enterprises as they attempt to pre-empt the disruption that technology and changing service dynamics will have on their industry. Some view this change and uncertainty as a great opportunity (even if it comes in the daunting form of commoditisation, as the Boston Consulting Group argued); some view it as a threat, which is to be counter-attacked.

Either way, it’s important for organisations to place themselves in their customers’ shoes in order to move as swiftly as possible ahead of the market. While Big Data can act as the ‘Ignition’ point for future product or service development, I believe it’s only the very start of a process that might ultimately lead to successful future delivery.

My view is that there are seven steps on the ‘Customer Thinking Framework’:

Step 1: Ignition: what can we understand from the Big Data available, and who is best-placed to act upon this opportunity?

Step 2: Identification: how can we focus the trends and shifts identified within our data to be really specific about the new opportunity at hand?

Step 3: Ideation: how can we flesh-out this initial grain of an opportunity to become something unique and with valid potential, and are there any close customers we can involve at this early stage?

Step 4: Insights: how can we tie the objective, numerical trends of our Big Data into the emotive, subjective, ’softer’ elements at play among our customer base?

Step 5: Implementation: how do the lessons we’ve learned from our ‘Insights’ allow us to iterate and adjust our initial idea, and how can we get customer input as early as possible?

Step 6: Impact: how can we road-test our idea with customers in a way that allows us to pinpoint its likelihood for success, and to course-correct as appropriate?

Step 7: Infrastructure: once we are confident about the approach on a small scale, how can we resource the broad delivery to take it to market at volume?

The principles of user-centred design (UCD) have become honed in the fields of app development and online delivery, as UX specialists highlight the virtues of thinking from an end user’s perspective at every stage of production.

“The opportunity for many of the system integrators we work with is to apply this thinking to a broader set of technology use cases and vertical market requirements.”

At Tata Communications, we see plenty of technologies, services and product lines which are awash with Big Data (the ‘Ignition’) and, crucially, can be linked to ‘Big Insights’. B2B social media tools and API-based collaboration tools, for example, are suitably mature for tech solution providers to understand the data underpinning usage trends; it’s critical now to build as holistic a picture as possible around the human insights which determine whether they will succeed or fail.

It can be easier, with Big Data already available to businesses in ways that may have been overlooked, to consult with industry experts who can advise where to discover and how to utilise these sources efficiently and effectively. Working hand-in-hand with these partners can allow collective Big Insights to be shared in a way truly founded on Customer Thinking.


Read more about data security one year on after GDPR was set in motion