Archive for September, 2013
Often experts pit Big Data and market research (MR) against each other, as if Big Data will make MR increasingly irrelevant. It’s a misguided match-up. Recently, At AbsolutData, in my capacity as SVP, Analytics, I had a discussion with a CPG senior executive in charge of insights, market research, for a large international CPG company. He is vitally interested, as with many leaders in the industry, in how to best combine Big Data with market research insights: Finding the best ways of bringing the two together.
I’ve taken the liberty of organizing that discussion into a Q&A format to bring out the most useful points. The discussion addresses common concerns about types of Big Data and illustrates high impact opportunities for it when combined with MR.
Question 1: How representative is Big Data, particularly for categories like kids yogurt where the core target is mothers who may or may not be the people expressing opinion/views on social media, chats, blogs etc?
Big Data can quite well represent a population. On a high level, some countries, like the US, have very high internet penetration and social media use by moms, suggesting that geographic internet penetration is a factor in determining degree of representation for given populations. Having said that, extent of representation is not a one-size-fits-all concept but is applied differently based on the context. So to more fully address the issue, we need to step back and put this question into the context of the intended use of the Big Data –
- Will we be trying to forecast demand?
- Assess overall brand or product satisfaction?
- Decide to make a product or a product line change?
- Understand the market-wide impact of product or advertising changes?
- Trigger more in depth research activity?
- Understand trends?
- Clarify consumer language and perception, etc?
There are many potential uses of Big Data and depending on the kind of Big Data we are referring to, it can have an impact on each of these areas.
The extent of representation depends on the kind of Big Data and the brand category. Furthermore the appropriateness of a given level of representativeness depends on the application of the Big Data.
If we define Big Data as Social Media – which includes tweets, FB comments and likes, Pintrest, Instagram, blogs, reviews, chats etc. these kinds of data will be representative of a given kind of population. For a category like kids yogurt, customers who mention the category or specific brands will be those who are involved in the category and are likely heavy users. These people are also most likely influencers in their circles of friends.
In fact, these people would well represent a valid target for the purposes of such issues as
- Assessing brand image
- Success or problems with product changes
- Tracking or anticipating extremes of satisfaction (which really is the zone of interest when examining satisfaction over the longer term)
- Exploring usage occasions, and
- Identifying most compelling features and benefits for communications
As another example, innovative companies like BlueFin Labs (recently acquired by Twitter) have shown a direct correlation between twitter comments and advertising effectiveness when correlated in real time with video streams in TV.
The application has a bearing on the appropriateness of representation – but the kind of data also bears upon the issue of representativeness. As detailed below in response to the second question, Big Data means more than Social Media and the data has different properties of representativeness, especially when integrated with disparate data sources.
Question 2: Therefore where and how can Big Data be used? And where can it not be used? What, if any, are its limitations?
In addition to the Social Media aspect of Big Data, and its associated specific uses, broader use cases become relevant when considering aspects of Big Data. We often think only in terms of social or web activity but we can include other data streams such as
- Daily or hourly sales by location
- Stock out information and distribution information
- Temperature and weather
- Socio-psycho-demographic information based on location
- Daily information on promotions, coupons, mailers, etc.
Because of the velocity, resulting volume, and variety of these kinds of data they can be used in solutions which are particular to what we consider “Big Data” these days. Not only can we map, in real time, these different elements together but we can model them as well to create early warning systems, recommendation systems, and communications and operations real time interventions. Social Media monitoring with text analytics can be overlaid geographically against sales, promotion usage and distribution information to react in real time with retail partners on pricing decisions or display options. We can then include demographic and even psychographic zip+4 information, and traffic information to maximize sales or communications effectiveness.
In short, when taken in its entirety and integrated together, the different aspects of Big Data that have varying degrees of representativeness when individually applied can be blended for general representativeness and applied innovatively to very important decisions. And, innovation is the fertile ground of competitive advantage.
Big Data can further be applied when approached more assertively. Big Data is a flow of information. It is somewhat reactive (but there is more to be said about this aspect) and, if we think in terms of social media, not particularly specific in all the ways we would prefer.
It must be harvested, filtered and interpreted. Or it can be blended with more traditional data to imbue more meaning.
If you are looking to test specific ideas, new product concepts, advertising copy, new product line configuration or pricing, then Big Data generally, used passively, will not provide the answers.
If you want to dive deeply into your target market to connect lifestyles, behaviors, attitudes and needs for segmentation purposes on a strategic level, you will need more than what is commonly available in Big Data.
This does bring up an interesting point, that of using Big Data passively or more actively creating data flows to more directly address specific issues.
On a very high level this is what we do with A/B testing of offers, promotions, web site enhancements, etc. We actually put something out there that customers react to in their normal activity and we measure those reactions and mine them for insights.
Likewise in the Social Media sphere much can be done to elicit more targeted and in depth information. Contests, viral announcements, online events, and such can be used to find out what people think or how they feel about very specific kinds of issues, products, features or benefits.
Question 3: How can one integrate Big Data with primary data and what would be its applicability?
Many feel very excited about the combination of Big Data with primary data.
On one level the excitement stems from the promise of enriching Big Data flows with the intimate and more detailed information that comes from primary data.
And vice-versa, the ability to further enrich our primary data insights by appending or blending in associated Big Data information.
There are many levels of integration that are possible now and yet will be possible in the near future. Here are a few examples:
- Augmenting a large portion of brand and advertising effectiveness tracking with social media – the best solution is a blended solution now
- Sampling for primary research from people who are part of a multi-device panel so that we know their primary data as well as their web and social interactions across a period of time
- Sampling for primary research directly from social media platforms, and therefore being able to tie into that users past data on that site and maybe even be able to glean information from their social-graph
- It is more typical to use social media listening and text analytics to identify trends, problems, and opportunities that can then be further explored using the more precise tools of primary data collection.
More specifically, AbsolutData has created several tools that specifically link Big Data and primary data. We can:
- Mine the social sphere to come up with an initial set of brands, attributes and levels to offer the most realistic testing using conjoint
- Track brands allowing the issues and image items we are tracking to take form from what people are talking about
- Develop a more realistic set of brand driver models based on social media information
- Administer surveys on consumer’s mobile devices allowing us to tie into social and local data both parts of the Big Data information flow.
In short the main areas that currently can blend Big and primary data would include
- Product development
- Advertising and promotional testing and effectiveness
- Segmentation and
- U&A type landscape work.
We can further combine primary data with other kinds of geo-demographic Big Data which could also include data on sales, inventory, pricing, promotion and distribution to more fully explain patterns observed in the primary data. In this area we have specific experience in
- Combining transactional data with behavioral data from survey for a segmentation exercise undertaken for a leading US based sporting goods retailer
- Combining internal CRM data with survey data to predict travel wallet share for a leading global hospitality major
- And other studies for our global clients
These ideas represent some of the increasing ways that Big Data and MR work hand in hand to generate more valuable insights. The two types of data do not contend for the insights but complement each other in deepening those insights. Each brings its own critical elements of value.
We love search, but do we really? What we are really saying is: I wish someone would find what I want. When we make people search, are we failing?
I am cross-posting a blog entry I authored for my company, AbsolutData, this month entitled: Identifying Customer Needs with Big Data.
“By the time you search, something’s already failed,” Phil Libin, chief executive of Evernote, said in an interview recently with the New York Times.
Brands can’t afford to fail to anticipate needs. With customer interaction, brands must respond before being explicitly asked – and the window of anticipatory response is shrinking every day. Before customers put energy into searching the site or asking for a product, brands that win in this ultra-competitive environment will have already anticipated – and created – a solution. As marketers, if we want to lead, we must provide customers with a situation that cuts out uncertainty.
Most businesses think this is rocket science, but we’ve had the tools to do it for a long time. Today it’s getting easier and becoming more of a mandate with the benefits of Big Data.
Big Data is already being used to identify customer needs
For 60 years, marketers have been building products that anticipate the needs of consumers. The difference now is that we have the data and tools to understand these needs in various contexts. Customers have been segmented by groups for more than 10 years, but Big Data allows us to classify individual customer needs and act in real time.
What used to be one-way is now a two-way social channel. Digitally and socially, marketers have new opportunities for relationships to increase loyalty through trust and wonder.
Big Data also allows for geographic opportunities to anticipate customer needs. Whether they’re on the highway, visiting a shopping center or out to dinner, we have an opportunity to communicate with customers at virtually any time using a mobile device. No, it’s not rocket science. In fact, some companies are already using analytics to anticipate their customers’ next moves.
Google Now’s Google Cards are a great example of how data is already being used to understand customers’ needs before they search. Google Cards works on Android phones to search existing user content, in addition to other data patterns including geographic location, to anticipate what the user will need.
For example, when your alarm goes off in the morning the application pulls up directions with the approximate time your route will take, suggesting when to leave and even providing alternative routes, if traffic is heavy. Google knows you go to work at the same time from your appointments and knows the location from your past map searches. It will even tell you stop hitting snooze or you’ll be late! The application also pulls up other things you may need for the day, including weather and important appointments.
A wider known example is Expedia’s use of data to suggest other items you may be interested in booking based on your previous behavior and others who have taken a similar trip. Even Amazon suggests new books, based on your interests and what others have read.
How businesses can move this forward within their organization
Most companies already have what they need, but they’re either not capturing it or aren’t using it in the correct way. Businesses have access to customer behavior data as well as social and transactional data. This gives them the tools to understand their customers on a much deeper level.
The first step in using this data is figuring out the vision – then you can make an argument for collecting, aggregating, storing and even mining the data. This comes from thinking about the company over a transformative period. How do you want the customer experience to differ? How can you improve the business? Larger, strategic changes can be greatly improved by the use of data.
Once you’re ready to integrate customer data, take it a step further and ask for feedback. Then, learn from behavior and refine the process accordingly to create an assertive interaction.
The tools you need and the team you’ll build depends on your strategy. There are a number of data platforms that allow for easy data access and management. New flexible analytics workspaces allow you to discover data and patterns you might not have known are important.
Finally, having the right talent is an imperative part of the process. A third-party analytics team, such as AbsolutData, can provide additional talent and tools support, or even take on projects externally.
How will this be used in the future?
The important thing to remember is that this is already happening now as the leading edge of current applications.
I don’t think we’re necessarily at the point where it’s a failure to have a customer search for a product on a brand’s website – but that’s where it’s headed, and quickly. Businesses must rise to overcome this statement of failure.
The cold, hard facts of life in this accelerating sphere of customer interaction show that if you are not taking steps now to learn and engage more, you will likely find yourself behind the pack. On the other hand, there is ample opportunity to get in the game or deepen the efforts now and doing so will pay handsome dividends.