Build Before You Optimize
“Are you a data driven marketer?” For the last 15 years, if you answered anything other than a enthusiastic fanatical ‘YES!!” to that question, you might as well have shat yourself in the interviewer’s chair and walked out the door. If you suggested that intuition occasionally swayed you, you could not be taken seriously as a marketer. Why listen to your intuition when you have reams of data and a former Goldman Sachs banker-turned-analyst to tell you what to do?
I think data can be incredibly helpful in developing a marketing strategy. Data helps inform your market size, can help you direct resources effectively, and can even teach you things about your business that you didn’t know.
But, it should be one factor in guiding decision-making. Not the only one. And, especially for start-ups, data may not be as useful as they would like it to be for a handful of reasons:
You don’t have enough data to make a confident decision. This term is referred to as statistical significance, and it loosely means that you need to have enough data for the outcomes of your analysis to be reliable. If you’re measuring the effectiveness of marketing campaigns, the customer journey on your website, the products your early customers buy, or any other customer/prospect behavior, you need A LOT of data for the conclusions to be reliable. Like 1000s, if not 10s of 1000s of data points. A 4% increase in CTRs for an email with a distribution of 150 people doesn’t mean anything. At all. It’s not even worth talking about. And, to put things in perspective, I worked for Wayfair back in the day, in their B2B, now Wayfair Professional group. If you’re a data-driven marketer, Wayfair is your Utopia, and even there, a good percentage of our campaigns didn’t have a large enough audience to reach statistical significance.
The tools needed to manage, aggregate, and analyze large quantities of data are really expensive, and they require their own management. DMPs, CRMs, MAPs - they’re all a big investment, and they’re not all that helpful until you have a large enough dataset to get a useful analysis for your investment.
Relying on data too heavily, too early can have really limiting consequences for your marketing team:
The time needed to do the data collection, aggregation, and analysis manually is a huge time suck for a small team. A founder recently asked me what he should do about marketing attribution. I told him nothing. He had a lot of “But, what about…” follow-up questions, and I kept telling him to do nothing. He still has a lot of growing to do before the information from a marketing attribution analysis would even be helpful, and, in the meantime, he’ll burn a ton of time pushing his small team to get a granular analysis of a too-small data set. I always say “You have to build before you optimize”. Get to the point where you have too many customers to know where they’re all coming from, or your email lists are begging to be segmented. Until then, your marketing teams should be largely focused on scaling the denominators (audience reach, prospect lists, customer base).
What’s really dangerous about a too-small data set is that it can point you in the wrong direction. Almost by definition, a dataset comprised solely of early adopters is going to be skewed. If you’re making product or messaging decisions based on the behavior of a few dozen early adopters, you risk missing the boat for the broader swath of mainstream customers that will follow. (And, if you’re saying to yourself, Well, I wouldn’t make any major decisions based on these analyses, I’ll refer you to Point #1, that the exercise is a waste of time when your team should be focused on scaling).
For marketers, it forces us to use marketing channels that deliver the best data and measurement, not necessarily the ones that are best for getting the message across to the largest, best-suited audience. I can’t tell you how many times I’ve been in channel-selection discussions, and any channel that couldn’t deliver granular behavior/engagement/purchase/interaction data was immediately dismissed. And, here’s the frustrating part: Facebook, Google, and other platforms that can deliver granular audience and performance data don’t perform well. They don’t deliver efficient customer acquisition or awareness results anymore. But, they give marketers what (they’ve been told) they need: data and measurement.
People’s intuition is silenced. And that’s a really big problem. I remember working with a guy who crowed about how much he loved being a marketer. He said The data tells me what to do, I follow it, and I don’t have to think. He thought this was a recommendation for marketing. I still recoil, remembering him saying this. I got into marketing because I thought it was a beautiful blend of art & science, not because I didn’t want to think. Not because I wanted my problems solved by a data set, but because I wanted to solve sticky problems myself.
Pushing marketers to solely rely on data for decision-making has unintended consequences that go way beyond customer acquisition and go-to-market decisions. From defining job functions to the people that apply for them to how marketing interacts with others functions within the org, it impacts every part of a marketing team function.
Data is incredibly important. Don’t read this post and think otherwise. It just shouldn’t be the only factor in decision-making, and it shouldn’t be overweighted when the data itself may not be all that reliable. There are lots of follow-up questions here, like what are the most impactful ways to use data in defining and reaching an audience, how to scale the denominators, and when is a good time to apply more data analysis. More on those in follow-up posts.