In 2012, New York Times journalist Charles Duhigg reported a story that most marketing students still hear about today. An angry father walked into a Target store outside Minneapolis and complained that his high-school daughter was being sent coupons for baby clothes and cribs. A few days later he called back to apologize. His daughter was pregnant. She just hadn’t told him yet.
Target hadn’t guessed. According to Duhigg’s reporting, its statisticians had found about 25 products, such as unscented lotion and certain supplements, that, looked at together, let them give each shopper a “pregnancy prediction” score. That’s big data marketing in one story. It shows what the data can do, and it also shows how it can go wrong, which we’ll come back to.
What Do We Mean by “Big Data”?
Big data refers to datasets that are too large, too fast-moving or too varied for ordinary spreadsheets and simple databases to handle. It’s usually described with the “three Vs,” an idea that goes back to a 2001 research note by analyst Doug Laney:
- Volume: the sheer amount of data being collected.
- Velocity: how quickly it arrives and needs to be processed, sometimes in real time.
- Variety: the different forms it comes in, from tidy rows of numbers to messy written comments.
To make this concrete, let’s use one example the whole way through. Imagine a regional supermarket chain called Greenway, with 40 stores, a website, an app and a loyalty card used by around 300,000 shoppers.
Every week Greenway collects millions of checkout transactions (volume). Online orders and app clicks arrive minute by minute (velocity). And it has loyalty card details, product reviews, customer service emails and comments on its social media pages (variety).
So which of that data is actually useful? On its own, none of it. A pile of checkout records doesn’t tell Greenway what to do. Big data only becomes valuable when it’s analyzed and turned into a market insight that someone can act on.
Structured and Unstructured Data
Structured data is neat and searchable, like Greenway’s transaction records. Unstructured data is messier, like reviews, social media posts and customer service chats. Structured data is easier to analyze, but a lot of what customers really think is sitting in the unstructured pile.
How Do Marketers Use Big Data?
Segmenting Customers by Behavior
Traditional segmentation might split Greenway’s shoppers into broad groups like “families” or “under 35.” With purchase data, we can segment on what people actually do.
For example, Greenway might find one group of “monthly stock-up” shoppers who spend $250 in a single visit, and another group of “top-up” shoppers who visit four or five times a week and spend $20 each time. Those two groups need quite different offers. A bulk-buy discount suits the first group. A quick, convenient weekday meal deal suits the second.
Personalizing Offers
Most of us see personalization every day, from streaming recommendations to “customers who bought this also bought” suggestions. Greenway can do the same thing with its loyalty program. Instead of sending every member the same weekly catalog, it can send each shopper offers based on what they usually buy. The shopper who buys oat milk every week gets a deal on oat milk, not on beef mince.
Predicting What Customers Will Do
This is what Target was doing. Predictive analytics uses past patterns to estimate what a customer is likely to do next.
For Greenway, the more everyday use is spotting customers who might leave. Let’s say a shopper who has spent around $150 a week for two years drops to $40 a week for a month. That pattern might mean they’ve started shopping at a competitor. Greenway could flag them and send a “we miss you” offer before they’re gone for good.
Why does that matter? Because a regular $150-a-week shopper is worth a lot over several years. This links directly to customer lifetime value, which many businesses use to decide how much they can afford to spend keeping a customer.
Reacting in Real Time
Big data also lets a business respond while the customer is still deciding. If someone adds a barbecue set to their cart on Greenway’s website and then leaves, they might see an ad for that same barbecue set the next day. That’s retargeting.
Working Out Which Marketing Worked
Imagine Greenway runs a social media ad, a search ad and an email in the same week, and sales go up. Which one worked?
Analytics tools can follow a customer’s path across several touchpoints before a purchase. Greenway can also run A/B tests, sending two versions of an email to two random groups of members and comparing which one gets more people into the store.
Listening to What Customers Say
This is where the unstructured data comes in. By analyzing thousands of reviews and social media comments, Greenway might notice a run of complaints about the freshness of its bread in some stores, well before it shows up in sales figures. This is often called sentiment analysis.
What Are the Risks and Limits?
Let’s go back to the Target story. Duhigg reported that Target learned some customers were uncomfortable when the company seemed to know about their pregnancy. So it started mixing baby offers in with unrelated ones, like lawn mowers and wine glasses, so the coupons looked random. The prediction didn’t change. The way it was presented did.
That’s the “creepy” problem. Customers who feel watched rather than helped can lose trust in a brand, even when the offer is accurate. So Greenway needs to ask more than “can we predict this?” It also needs to ask “would our shoppers be comfortable knowing we used their data this way?”
There are practical limits too:
- Data quality: if loyalty cards are shared by a whole household, Greenway’s picture of one “customer” might really be four people.
- Disconnected systems: website data, checkout data and customer service records are often kept in separate systems that don’t talk to each other.
- Skills: someone has to turn the numbers into sensible marketing decisions, and good analysts aren’t cheap.
- Correlation isn’t causation: a pattern in the data doesn’t always mean what it seems to mean.
Privacy Rules
Big data marketing also has to follow privacy law. Rules such as the European Union’s General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) give consumers rights over how their personal data is collected and used, including ways to opt out. Australia’s Privacy Act sets similar obligations.
For a business like Greenway, this makes first-party data, collected directly through its own loyalty program with the shopper’s agreement, more valuable.
So What Should Marketers Take From This?
Big data helps marketers decide based on what customers actually do, rather than on hunches.
But the best users of big data aren’t just the ones with the most accurate model. They’re the ones who use it in a way their customers are comfortable with. If you were Greenway’s marketing manager, that’s the question worth asking before every new data-driven campaign.
Sources
How Target Figured Out A Teen Girl Was Pregnant Before Her Father Did – Forbes
