Opinions are Bad Business

A good e-commerce manager is constantly monitoring, maintaining, and (perhaps most importantly) testing and optimizing their website. How do you find areas for improvement or decide what changes to make? I’d imagine that a lot of ideas come from word-of-mouth; a blog post, a conference session, a conversation with a co-worker, a nice (or not!) email from a customer, etc. You probably also do research; look at competitor’s sites, read benchmark studies, perform usability tests, solicit feedback with surveys, and so on. All of these avenues can provide valuable ideas for site improvement, but making changes to your website costs time and money. You shouldn’t THINK making a change is a good idea; you should KNOW it.

Are you familiar with the term “qualitative data”? That’s what all of the things listed above are. Opinions. Suggestions. Ideas. Concepts. As much as we hate to admit it, our opinions (no matter how strongly held) are not facts. We also easily fall victim to confirmation bias; supporting information that validates our own opinions and more easily rejecting information that doesn’t. Confirmation bias is a huge danger area for business; decision makers that fear of being wrong about their opinions impede improvement and stifle innovation.

I have a LOT of opinions regarding website user interface/user experience (UI/UX); most of them backed up by years of testing experience, but the bottom line is that while they may be informed opinions, they’re still opinions, not facts. If I were to review your site today and say “oh, I think we could make change ABC. I’ve tested that before on five other sites and it always generated at least a 10% conversion lift.” You hear that, and think, “Well, that’s great! What are we waiting for?” Do you KNOW if the change will work on your site? Nope! It’s still just my opinion, albeit an educated one. What if you invest $1,000 into making that change, and it tanks? $10,000? $1M? How good are you at risk management?

If you could be 100% certain that making a change has a specific result, wouldn’t you want to know? Getting better at risk management is why qualitative data alone isn’t sufficient; it doesn’t provide a critical threshold of information scrutiny to be a fact. It’s the old “9 out of 10 doctors agree…” thing. That’s why I say “probably” when discussing change recommendations with clients. Change ABC might be a great idea 9 out of 10 times, but what if your site is the one time that it’s a bad idea?

This is where quantitative data comes in. Quantitative data is qualitative data at scale. Instead of “9 out of 10 doctors” it’s “990 out of 1000 doctors.” Notice the root words, quality and quantity. Qualitative data are good ideas, quantitative data are PROVEN ideas. Quantitative data is gained through split testing. Test out an idea; see how your users actually react to it. Does the change have the desired outcome, or did it tank? You can find out. Prove it one way or the other. That’s why I do split testing (in fact I wrote a few articles about it), and you should too.

Don’t invest time and money into bad changes because someone thought it was a good idea. Use qualitative data to find places for improvement. Use quantitative data to make sure the change provides ROI.