Scott Adams and the Demise of Common Sense

Scott Adams, the creator of Dilbert, has announced on his blog that he'll be blogging less often. It seems that his original common sensical expectations about how the blog would turn out aren't coming out well at all.

They original expectations included:

1. Advertising dollars
2. Compiling the best posts into a book.
3. Growing the audience for Dilbert
4. Artistic satisfaction.

Of these, only number 4 has worked out. RSS has made visitors go around the ads, the book hasn’t done all that well, and the audience for Dilbert hasn’t been correlated at all with the growth of the blog. As the blog has exploded, the benefits to him haven’t. So he’s talking about blogging less often. It’s a great illustration of how common sense is a lousy predictor of future events. Viva testing and statistics.

Numbers Aren't Always Useful

A while back I worked with a client who used a popular service that provides percentage figures for visitation of others' websites. This service contracts with ISPs to get sanitized web visitation figures from its subscribers, some ten million of them last I heard. Then it reports mass figures on who went where. The problem is that it's impossible to find out just what those numbers are - all you get from the service is percentages, which are presumably percentages of the subset of ten million that went to that particular site on any given day. How many is that total? Nobody knows, and the service isn't telling. In my client's case, they were getting figures like .0016%, which is so low that it isn't hardly worth knowing, but they were very keen on it, watching the numbers shift daily from .0016% to .0021%, and cheering lustily at the uptick. Of course, .000016 X 10^6 is only 160 (visits, users? who knows?), and the surge amounted to only 50, a number that didn't make them so cheerful when I pointed it out. And that assumes that the denominator is indeed ten million, which is probably isn't - it's doubtful that all ten million subscribers are online on any given day. The actual improvement might have been as little as 10, or less.

The problem is taking percentages as real numbers. They need to be scrutinized, and you need to know what the denominator is. We called the service to find out just what those percentage figures meant, but they either wouldn't, or couldn't, tell us. The people we talked to were frankly ignorant of simple statistics, calling the ten million subscribers a "sample". I had to tell them that it wasn't a sample, it was a sampling frame, and that without knowing the denominator for the percentage, the tiny percentages the client tracked were all but meaningless. The service had no answer, and the call ended unsatisfactorily. And the client continues using the service to this day, and happily reporting the results to higher-ups.

The service may be worthwhile for larger sites that get a sizable percentage of whatever data points the service is tracking, but any sites smaller than that are probably not getting their money's worth. I used to think that people with marketing degrees would be more conscious of the fallibility of numbers and the need to understand analytics, but I've been wrong more often than right.

Common Sense Sucks

As HCI practitioners who believe in making technology slicker to use, our biggest opponents may be budgets, but coming up strong on the outside is stupidity based on common sense. Human psychology is a weird and wonderful thing, and I love studying it because there's always something unexpected waiting to mug you around the corner. Where others love bar fights, I love getting pasted by new knowledge. And research has shown for a long time that common sense is a lousy predictor of future events. You'd think that humans would be good at prediction by now, but we're not. For one thing, we think in linear terms - one cause, one effect. But the universe is gloriously nonlinear, and every event has many causes. Further, humans tend to predict based on what's going on now. When we ask "how would you like this?" or "how would you feel about this?", the respondent has to extrapolate based solely on how he feels at the moment. When we put this to the test, we find that the respondents don't really feel that way later on. This is why I put no faith in predictive surveys.

When those in power use common sense to make sweeping laws or spend huge sums of money, they too usually screw the pooch. The Freakonomics blog has a short piece on how abstinence-only sex education has actually resulted in more teen pregnancies, something any psychologist could have predicted. The urge to merge is far too strong to educate away, especially in adolescents who haven't yet developed much control over their impulses. The victims of abstinence-only education aren't given the tools to prevent disease or pregnancy, but they're driven to give in to the mating call anyway, resulting in more pregnancies. It's obvious from studies that abstinence-only ed doesn't work, and New York has appropriately dropped it, despite losing millions in federal funds. But the US Congress stubbornly sticks to the plan. Common sense is dooming teenagers and costing millions, all with no substantial foundation, but that doesn't matter.

The lesson for us is to mistrust common sense, both our own and our clients'. I've seen clients cling to old, unusable designs simply out of faith. Analytics, psychological principles, and user testing will eliminate most of the problems if they're used, but they won't be applied if common sense has anything to say about it.

Why a Messy Room is a Good Thing

I saw a comic the other morning that featured a young man and a thoroughly trashed room, with items strewn all over the place. In doubtless a whiny voice, the youngster pleaded "but when I put things away, I can't find anything". Parents will smile smugly in silent rebuttal, but an HCI'er and part-time economics junkie like me can't help but wonder if those legions of room-messers don't have a point. When we humans do things frequently and in large numbers, there's usually something to it.

Advocates for clean rooms (like rabid inspecting drill sergeants) like to say that everything has its place, and that's where it should be. But is that really true? After you reach a certain threshold of object ownership, is there a single place for everything? Children mostly keep toys in a box, where toys on top obscure the ones underneath. Folded clothing suffers the same fate in a drawer, where you have to pull out the stuff on top to get to the items below.

In effect, it seems to me that a messy room is actually a form of shallow navigation where little is hidden badly enough to be overlooked. The same pertains to a cluttered desk. I often keep a wide, low pile of file folders, papers, notes, books, and pads on half my desk. A neat freak might object that I could just as easily keep all that in their respective drawers and bookcases, but I'm convinced (without much evidence, I have to add) that doing so is less efficient. I can riffle through the pile faster than I can flip through file folders in a file drawer, even in alphabetical order. The pile is for things I'm using frequently at the moment, and overcomes the problem of filing materials under the wrong headings. Our users tend to like shallow navigation, and I'm convinced that I do, too.

Give Me Your Huddled Masses...

When I do usability work, it's astonishing how often a client won't give me access to their user base. They're so used to protecting that base that they can't immediately swing their thinking around to working with that base, rather than just wining and dining them. Sales personnel are the most protective, in my experience. They seem to live in terror that somebody will screw up the precarious relationship they have with their customers. Even when I get to customers, I'm usually given only a small number of very carefully selected and primed representatives, and sometimes not the right people within the customer organizations.

The excuses are legion - customers are too busy, they're too far away, the right people aren't available. But perhaps the most ironic is the excuse that I'll raise customer expectations by letting them think that their wishes will become features in the next release, and if they aren't then the customers will get crabby and disappointed. A lot of salespeople leave me with the impression that their customers are generally vocal and upset about something, interspersed with short interludes of grumpy acceptance. The salesperson doesn't want that sleeping dog disturbed by the slightest zephyr.

But what they can't seem to get straight is that I'm not after user wishes; I'm after user processes. I'm not going to talk much about desires, but about business. I'm trying to understand how customers operate. They'll tell me their desires, of course, but I'm always careful to be noncommittal, with comments like "I'll see what I can do about that in the design, but it may have to be in version two".

Letting users into the planning and design stages is good business, and establishes a level of trust that can't be won any other way. Now if I can just make clients see that...