Do Searchers Search More Over Time?

A paper from 2004 recently fell into my hands. It's from the journal Management Science, so you'll have to go there to get the paper (you'll need an account). As is usual in journal papers nowadays, it has five authors, Johnson, Moe, Fader, Bellman, and Lohse.

They did several studies, partly focused on asking whether people use search more often as they get more experienced on the Web. They also looked at how much people searched for sites when they wanted to buy something.

The results might be surprising to some - even in the age of search, users don't like to check out a lot of e-commerce stores. The majority prefer to settle on a few stores and go there rather than constantly checking out new ones. And even when they get more experienced with search, they don't use it much more. And they found that users don't search as much as you'd think.

None of it surprises me, although they didn't account for some factors, like age. Marketers have long known that past a certain age, the willingness to try new brands drops like a stone. The authors didn't break out sessions by individuals, but by households, so there's likely to be a lot of slop to the data.

The question that occurs to me is whether, if such brand loyalty online is real, it's due to actual loyalty, or reluctance to tangle with a new interface. E-commerce isn't so much like brick-and-mortar shopping as it like operating software, and few users enjoy mastering new software. Is it the label, or the comfort level?

There's also the fact that many users dislike searching unless they know a strong keyword. Look for "toilet seat" and you're likely to find an online hardware store. Search for "pens" and you'll end up with specialty stores, stationers, and collector sites, all of which have to sifted. Google is good, but it's not clairvoyant.

Third-Party Perils

A lot of client sites that I evaluate have tagging problems that aren't really of their own making. We have clients "tag" their sites for analytics purposes to send data back to our mothership, which is then returned to the client as reports. As you undoubtedly know, it's been getting commonplace to "farm out" a certain part of a site to third-party suppliers. Many clients, for example, now out-source their employment pages, with just enough matching page elements to make the visitor think they're still somewhere in the same site. Same thing with newsletters and emails - other people handle it for you. The problem is that those sites usually aren't tagged, so you can't track them. No tracking, no evaluation. Again, small sites aren't deeply affected, but bigger ones are. If you can, work it out with your vendor to let you tag their pages, or have them tag the pages. It's not a new request for most of them. Don't ignore such vital functions as recruitment and marketing contacts.

Engage This!

I'm afraid that I have to take exception to yet another Web buzzword. This time it's "engagement". It's hot right now. Just ask Eric Peterson, who's making a little cottage industry out of his own "Engagement Index". Please. Make it stop. "Engagement" is no better defined than "intelligence", "happiness", or "it sucks".

I'm really a numbers kinda guy, with the heart of a researcher. That means I resist sloppy thinking. And "engagement" is just that, sloppy thoughts. Naming something and believing that you've driven to the heart of it. Peterson's various components may have merit, but he's going about this all the wrong way. Ideally, you study a big group of things and then derive patterns using standard statistical techniques. You don't just wish them into being, no matter how sure you are that they exist. Then you validate your model against a known situation and see if it holds up. If it wavers, fragments, or veers wide of the mark, then your model is faulty.

So far as I can tell, Peterson has never subjected his model to rigorous validation. His various engagement components in the index aren't weighted, so as one rises another could fall, leaving you with the same EI, but with a different situation entirely. I think anybody who relies on a single-measure EI to make expensive business decisions is playing with a loaded gun with the barrel plugged.

That's not to say that "engagement" could never be defined. It can. But it can be defined only as a series of KPIs that shouldn't be arbitrarily added together. A simple radar chart could show them all. So could time series charts. And it should be defined anew for each site. The quest for a standardized index will go on, but in the end I think it's futile. Adopt a Deming approach and keep working on your own special site. I don't think there are any shortcuts.

Old Houses and Portals

I live in an older home (1920s) in a historic neighborhood. It's not a particularly wealthy neighborhood. Most historic ones aren't. Money flees its breeding ground. But the neighborhood is comfortable and reasonably vibrant. I always wondered why I loved older homes, and finally one of my gurus, Stewart Brand, might have explained it in his book How Buildings Learn: What Happens After They're Built. He says that older buildings exude what we call "charm" or "character" because they've been altered over time to suit both changing infrastructure needs (the arrival of central heating, air conditioning, indoor plumbing, electricity) and the changing life needs of its occupants (bigger kitchens, more light, more entertainment at home). They grow, morph, and gradually conform more closely to actual human life, like an old pair of jeans. New homes are raw despite their efforts to "design for life". Brand points out that buildings can't be designed up-front for our lifestyles, because no designer can get it right the first time. That's why a home needs so much time to find its proper shape.

What's the lesson for Web designers? Alas, probably not much, despite my most earnest desires to bring the analogy across. The missing element is time. Websites don't give you time. Portals were supposed to let users modify their views quickly, compressing the decades of home conformity into minutes online. Never worked. The vast majority of visitors never knew about customization or took the time to mess with it. Personalization works to an extent, but not completely. Web users are now used to their comfortable sites changing regularly, and although they may not approve, they rarely boycott on that basis.

That said, for years I've been fascinated with the idea of a personalization engine that would track Web user behavior and subtly shift the interface to suit. I've never bothered to fully flesh out the concept, but in general it would work much like Microsoft's failed personalization functionality in Office, the one that gave you chevrons instead of full menus. It was a good idea, but possibly the wrong place to use it. Office users are almost all repeat visitors. Website visitors aren't. Amazon does a good job with personalization, but I'd extend it from "you might also like this stuff" to actually shifting controls and navigational paths. A pipe dream, certainly, but given a huge pile of cash something I'd be interested in researching.

Reading TeaLeaf

Sorry to have been away so long. Complications of various kinds. But now I'm back, and with tea.

Have you seen TeaLeaf? It's a snazzy app that sits athwart your Web traffic, sniffing and recording every user's session. A bit disconcerting, that. But its benefits are undeniable. It stores thirty days (or more, at your discretion) of user transactions, at the user level. It aggregates them too. I've long been a proponent of continual usability checking. Our profession seems to put all its emphasis on initial design and testing, while utterly neglecting Web analytics and other red-flag functionality that can signal usability leaks. Traditional Web analytics is good, but it isn't always granular, meaning that its results are en masse, not at the level of the individual user. It's great for marketing departments, but not as good for usability concerns. TeaLeaf shows the actual user transactions - where people go, what they click, what choices they make, and whether their conversions are successful.

For example, you can lose users at any turn in the road, but especially during checkout. Many visitors drop off when money becomes an issue, and understandably so, since they had no intention of paying anyway; they're just here for the experience, or the knowledge. But others experience technical problems or usability pitfalls. TeaLeaf generates a report on who converted and who didn't, and then you can track out why the failures happened, following every user's trail.

Scott Adams and the Demise of Common Sense

Scott Adams, the creator of Dilbert, has announced on his blog that he'll be blogging less often. It seems that his original common sensical expectations about how the blog would turn out aren't coming out well at all.

They original expectations included:

1. Advertising dollars
2. Compiling the best posts into a book.
3. Growing the audience for Dilbert
4. Artistic satisfaction.

Of these, only number 4 has worked out. RSS has made visitors go around the ads, the book hasn’t done all that well, and the audience for Dilbert hasn’t been correlated at all with the growth of the blog. As the blog has exploded, the benefits to him haven’t. So he’s talking about blogging less often. It’s a great illustration of how common sense is a lousy predictor of future events. Viva testing and statistics.

Another Hideous Example

Perhaps the most fun entries to write are about awful sites, and here's a real winner, courtesy of BoingBoing. The site is by the British government, and it's trying to discourage knife violence. But the site has two major failings: it's all in Flash, and it has enough stupid lawyerly language to discourage a Supreme Court justice from reading the site. The BoingBoing article is here. The awful site is here.

Bookmarks as IA

I once did a project for a client that involved talking with users about their browser bookmarks. The project was a redesign of an intranet that was built like a Wild West town, with a wacky combination of independent little plots stitched together only by virtue of being under the same corporate umbrella and having links on the central page of the intranet. Every department had its own navigation and design. It turned out that employees coped by using bookmarks to provide dependable paths back to the information they had so painstakingly located. Nothing new in that, of course; Web users still use that strategy. But interestingly, the names they gave to the bookmarked pages in the bookmarks were indicative of their own quirky needs. In effect, the bookmarks were individualized navigation schemes, or IAs. By studying the bookmarks, we got a pretty fair idea of users' mental models for information.

Whenever users create informational structures, it's worth studying them to discover what's core, and what's transitory. That's the problem with today's tag or link clouds: they can't distinguish between fad and eternity. Any given cloud today may have "Britney" as its biggest member, but that probably won't be the case next year. Clouds are intrinsically time-bounded. But it would be interesting to do some multivariate work like cluster analysis on several clouds over time to see what drops out and what stays.

Control is Everything

Scott Adams got me thinking. In his blog entry for August 17, he mentions that one of our strongest needs it to feel like we're in control. He used an old example: A genie offers you two choices. In the first choice, "You can eat at the finest restaurants in the world for free, twice a week. The only catch is that the genie picks the day, when you are not already booked, and he picks the specific restaurant." In the second choice, "You can eat at “good” restaurants, again for free, twice a week. But this time you can schedule it whenever you want, up to two places per week, and pick whatever “good” restaurant you want."

He goes on to develop the theme that the first choice probably wouldn't make many people happy, because they would eventually feel the keen sense of loss of control. The second choice, while gastronomically less appealing, is probably a better one for most of us.

It reminded me that one thing users dearly love is control, or at least the illusion of it. This is something that subconsciously irks me about lots of software and websites, I think. It's why I'm irritated with Flash so often. It just takes off and does things without asking me. The same thing annoys me about flashing ads, shifting menus, and other things that don't help me do things, but invade my locus of control. We humans don't seem to resent losing control if we don't expect it. We accept that the good guy may die at the end of the movie, but we'll shriek in fury if we can't change the channel to another movie. And we accept a loss of control when it benefits us. My car's engine does hundreds of things that I don't need to approve as they're happening. But there are some places where humans just won't accept interference. I wouldn't pay less for a car if it decided by itself when it would start. The same thing is true for software and websites, I think.

Decluttering

A group of scientists at MIT headed by Ruth Rosenholtz, a long-time researcher into vision and technology, has developed a prototype application in MATLAB that determines the amount of clutter on-screen (Link). The HCI profession has long needed something that could separate figure from ground reliably. The program is only in prototype, but apparently it's rather promising.

The problem of figuring out what's vital few from trivial many isn't trivial itself. Nuclear facility control rooms are a case in point. Rows of lights can go from being background hum to suddenly becoming extremely important. How much do you expose to an operator, or to a website user? Hicks Law was an early attempt at measuring how much stuff was too much, but the sophistication of control schemes today needs a better way of knowing when you've overstuffed the interface.

Usability as a Path to Failure? Surely Not.

Todd Wilkins at Adaptive Path has thrown down a gauntlet to usability professionals, claiming that usability is not only overrated, but even injurious and a path to failure. He cites successful artists who didn't worry about "usability" either. He says:

So, why oh why do people in this day age still hold up “usability” as something laudable in product and service design? Praising usability is like giving me a gold star for remembering that I have to put each leg in a *different* place in my pants to put them on. (Admittedly, I *do* give my 2 year old daughter a gold star for this but then she’s 2.) Usability is not a strategy for design success. The efficiency you create in your interface will be copied almost instantaneously by your competitors. Recently, I’m even coming to believe that focusing on usability is actually a path to failure. Usability is too low level, too focused on minutia. It can’t compel people to be interested in interacting with your product or service. It can’t make you compelling or really differentiate you from other organizations. Or put another way, there’s only so far you can get by streamlining the shopping cart on your website.

Ahem.

Rarely do I see a designer get this blatant. They may think this drivel, but they don't usually voice it before a plunge into happy hour. First, usability here might seem synonymous with "make stuff easy to see". We professionals know this is not anywhere close to being true. Second, it entirely overlooks that websites aren't works of art, unless they're private, non-commercial ones. Commercial (e-commerce) sites are for making money, and every visitor who snorts in frustration and leaves is a financial failure, not a failure to make a friend. Visitors don't need to be engaged, or have fun in most cases. They need to transact. They need to do the tasks they arrived to do. Much "design" merely gets in the way of that simple goal, and ought to be cut out like a splinter under the fingernail, because it provides about as much value. A big-time website isn't an opportunity to dance the visitor about. It's to enable him to act.

Of course any successful design will be copied. But then, there are only a few designs in human experience, and they're all copied every day. Graphic designers tend to think that their designs are unique and powerful. Most often the ones that are sold this way are actually glitz with no go, at their core simply reproductions of past designs with a few cosmetic changes. There are only so many ways to arrange elements on a surface.

In my view, websites are not akin to artworks, but more like cars. First you make sure the damned thing drives properly, and then you dress it up. Not the other way around. We tolerate few physical objects in our lives that are as poorly designed as "cool" or "artistic" websites, yet we complain about the physical and work our way around the virtual. This seems asinine to me.

Keeping a Usability Portfolio

When I scan the want ads for people in the design end of usability, I often see language like "Must show portfolio". Huh? Most of us would find that requirement very hard to fulfill, no matter how long we've been in the business. It's not like we're artists in a garret, and our work endures down the centuries. It may last only days or weeks. And it may be buried in the overall design of the site. Further, we may not be the graphical designer, who will get credit for the look of the site. Perhaps more importantly, websites are inherently team affairs, largely produced by committees. After the wrangling is over, any usability person might question where his or her work might be found and pointed out. Add to this the short life spans of many design companies or design departments. Even if the company name sticks around, the personnel turn over rapidly in some places. After we've been gone for a year or so, nobody there remembers us. The lesson here is that when a site goes live, we should take screen shots and put them away on a CD somewhere so that later we can make up "portfolios". Forget, and the opportunity may slip away forever. Put it into your design process.

When the Least of Us Are Ignored

I was made a convert to the cause of accessibility a few years back when I attended an STC conference with a progression that dealt with handicaps. At each table was a different handicapped person. One was blind, another deaf, another with only limited use of his legs, and so forth. It sounds like a freak show, but it was shockingly enlightening. I never forgot the lessons I learned at that session, and if you ever get a chance to be taught those lessons yourself, I suggest you take it.

One big lesson I learned was that accommodating the handicapped is not necessarily a big or expensive proposition, but simply being conscious of them. Widen aisles a little. Don't use slick flooring everywhere. Give optional paths that are not demeaning. In my view, this applies to all of us in human factors.

Then I walked on the Sakai project, and was jolted again. The one person on the whole big, extended team who was thinking about accessibility for the visually handicapped was almost literally crying out in the wilderness of Ann Arbor, Michigan. Sakai's interface was a long way from being handicapped-friendly. Tests proved that screen readers couldn't use it. It's been improved, but it's still not exactly ready for screen-reader prime time. Portals are often difficult for screen readers to use. Flash, text in graphics, and scripts can be real headaches, too.

I've since been struck several times by how little attention is paid to accessibility online by any website owner. Even e-commerce sites are often impenetrable for the blind, and unnecessarily so. (This may change. The National Federation for the Blind is suing Target Corp. to make its online suit accessible, under the Americans with Disabilities Act. Very preliminary so far.)

But I also found out something else interesting -- concern about the handicapped is generally in direct proportion to how much contact a designer or marketing manager has with the handicapped. If someone in their workplace, church, or family is blind, deaf, or has physical problems, they're usually far more interested in making the blind welcome online. If they've never run across the handicapped except in movies, then they're often not just blind themselves, but dismissive.

Eye-Tracking of Multiple Images

A user experience expert will often want to know where users’ eyes are going on a page. The early equipment for “eye tracking” or “eye gazing” was cumbersome and unpleasant for users, but I’m seeing a lot of work being done nowadays with lighter and more easily available gear. This is an example of one research report on eye-tracking I ran across recently: http://psychology.wichita.edu/surl/usabilitynews/91/eyegaze.html

Eye tracking maps are often known in the user experience trade as “heat maps”, because most of the time they’re shown as websites with superimposed patches of color that go from light blue to blazing red, depending on how long a user has stared at each spot. Current research is revealing interesting things about how people look at sites. Text almost always shows an F-shaped pattern of scan. We’ve known for a long time that visitors don’t read text online, but scan instead. This is old news. But the research reported by Usability News looked into patterns of search on pages with lots of images, and there the gaze patterns break up rapidly into individualistic styles, especially when searching for something in particular. Otherwise, when browsing visitors show much the same orderly left-right, up-and-down zigzag pattern we’d expect to see. During visual search of multiple images, the eye hops rapidly about in unpredictable patterns as the brain works in overdrive to spot patterns, just as it might in an unfamiliar room with too much furniture.

The research didn’t investigate further, but from my own experience I’d say that the hippity-hoppity effect can be neutralized with proper use of boxing, labels, heads, and other clues that let the visitor quickly narrow down choices.

Web Analytics

One of the things I wish my colleagues knew more about was Web analytics. We've ceded this important facet of usability to the marketing folks, and we need to get a piece of it back.

Web analytics is the ability to get constant data on where users have been in your website, what they looked at, where they came from, what they bought, how long they lingered, and so forth. Much of this stuff used to be in log files, but they proved to be too feeble for prediction. Current analytics packages have much more functionality than log files ever did. For example, in WebTrends I can see what proportion of users went in one of several directions from the main page, and where they went after that, and after that. I can see what keywords they used in searches, both on the site and in search engine queries to find the site.

Web analytics has become the province of marketing departments everywhere, but often they don't know how to use the data for usability or IA purposes. We do, but most of us never look at the data, or don't know what to do with it when we do. When you can see clickpaths through your site, you can begin to optimize your navigational structure. You can gradually make the site better. If you're using a WA package now, I'd suggest you start mining it for every bit of gold you can find.

What's REALLY Screwing Up Usability

After a considerable period of experience, I have come to the conclusion that most websites suck more because of lousy navigation than from bad controls placement, poor color choices, small text sizes, or anything else related to layout. I've found that users will forgive and use almost any interface mistakes, if only they can find what they're looking for. Poor navigation has left more users looking as confused as goats on Astroturf than any other single cause.

Now, it should be noted that this general rule applies more obviously to information-storage sites, not so much to interactive sites. But even if the emphasis is on interactivity, knowing where to go next in a sequence of steps is only a variant on where to go next to find materials.

It's for this reason that I wish HCI programs put more emphasis on information architecture. Many HCI'ers, even those with grad degrees, can't do card sorts or affinity exercises, nor perform cluster analysis. They have no clue about taxonomies, ontologies, or thesauri. I'm finding that the ability to organize whole site logically is both science and fine art, and deserves more class time than it's getting.

Life Poses Usability Problems

As if life didn't have enough usability problems, Second Life is apparently even worse. Over at Meta Versatility, there's a nice blog note about Second Life's usability difficulties. Linden Labs has engaged Adaptive Path to clean up the interface.

At Last I'm a Lert

As the saying goes, "Be alert...the world needs more lerts!" I'm contributing my share.

I've discovered Google Alerts, and I'm using it to watch for some terms. Google Alerts lets you know when something is published on the Web that matches your keyword in the Alert. I, for example, am using three alerts: my name, "usability", and "information architecture".

So far the results have been mixed. I've caught a couple of references to my old books, which is exceedingly gratifying. And I've caught a few good blog entries here and there about my professional subjects. But a good many notifications are less than intriguing. Signal-to-noise definitely trending toward noise, but infinitely easier than doing tedious, full-out Google searches to stay current.

Portals Will Forever Suck for Usability

In the past few years I've spent a lot of time working with portal software. I was involved for half a year with the design of Sakai, which you may know as Oncourse CL. For those of you who want to flail me about its usability flaws, be assured that we refugees from the Tools Team, who were the usability voices, are not too happy about them either. I've since become even more deeply involved in Websphere Portal and its evil little sidekick LWCM, the web content management part of the team. In between, I've been a user of one or two other portal packages. Believe me, they all stink from a usability perspective.

First let's get together on terminology. A "portal" in the old sense is just a website with a bunch of collected links that send you elsewhere. Yahoo pioneered this interface. The newer sense of "portal" is actually a software package that specializes in showing various systems to a user all at once, and supposedly giving the user the ability to customize his own interface (although no one ever does). Portal pages can have different "windows" on a page. For example, one place on the page permits Google searching, another one gives the weather, another one shows your stock market portfolio, and yet another has corporate news from your employer's PR machine. These various "windows" are actually known as "portlets", "applets", "gadgets", or "tools". Oncourse CL is an example. There, it's "tools". You can write a tool. Anybody can. It's completely open, and that's what Sakai's creators are most happy about.

But that's Sakai's Achilles heel, too. Tools don't talk well with one another, because they're designed to be secure and happy living alone. Navigation and other usability factors vary from tool to tool, so you can never just settle down to one consistent interface.

Major portal makers, like IBM and its Websphere Portal, suffer similar problems, except in greater profusion. A typical Websphere portal page may have a dozen "portlets" on a single page, all doing something different, and all potentially with a different "skin". If nothing else, they don't always fit thematically together. They're like jet cockpits. Over there is the flaps indicator, while over here is the oil pressure indicator. Just a flock of barely connected different things. There's no flow to the page, few cues, and little uniformity. And it's designed to be just that way, as if the business and computer science communities conspired in smoke-filled rooms to stick it to both users and usability professionals. Portal proponents claim that users can overcome this madness by customizing their pages, but that's just crazy talk. Users won't do it. They suffer in silence instead. In a portal, there's almost no room for conventional user testing or interface design. There is no "interface" as such, merely pages with things that do stuff. This is great for developers and business types, because portal architecture makes it much, much easier to incorporate in one place functions as diverse as weather announcements, time reporting, and CRM applications.

If you want to see examples, check out the NCAA (www.ncaa.org/wps/portal) or IBM (www.ibm.com).