Users Cannot Prognosticate

One of the favorite usability techniques by the uninformed is to sit a prospective user down in front of an almost-finished application and ask "So...how do you like it?"

This is wrong for at least two reasons. The first is the "New Coke Mistake". Older readers will recall that the Coca-Cola Company's New Coke was released in 1985. It was a sweeter, sprightlier formulation prompted by the enormous success of the Pepsi Challenge, a nationwide taste test that legitimately proved Pepsi to be a runaway favorite of test-sippers. But Coke made a huge mistake by equating sip-tests with long-term purchase and usage. Even hardened Coke drinkers didn't like the sweeter Pepsi, can after can. They wanted their old Coke. The taste tests had panicked the suits at Coke and led to a disaster of such proportions that the whole episode is still taught in marketing classes today.

The second reason is more subtle. It results from our human inability to anticipate. We like to think we're good at anticipating things. What else is a cerebrum for? But in study after study, we humans show a remarkable lack of prognostication skills. We can't correctly predict what we'll like or dislike later, what kind of gifts we'll want down the road, or what we actually do in a crisis. We do very poorly predicting how we'll respond to emotional events. Nor can we reliably predict how we'll like something later that we've just now seen. In actual fact, our conscious mind isn't all that conscious of things, either those things generated within the skull, or from the outside. Drunks aren't aware that they're impaired, for example. Cell phone users insist their reaction times are unaffected while driving, when research plainly shows that reaction is very much diminished, turning 20-somethings into senior citizens. Marketers have repeatedly found to their chagrin that products proposed to focus group members and enthusiastically embraced at the time often fail to catch on when they're actually sold.

That's not to say that focus groups or interviews are useless. They're not, but they're best used for eliciting information about the now, not in anticipating how they will be or feel later. It's almost never worthwhile asking participants to foretell the future.