FYI The perils of user testing and web page creating

Rohit Khare (khare@pest.w3.org)
Fri, 17 May 96 12:11:14 -0400


Software - Testing how easy "easy" really is
{The Wall Street Journal, 10-May-96, p. B1}

Creating multimedia pages on the Internet's World Wide Web isn't just for
computer programmers any more. Companies now sell graphical software aimed at
ordinary folk, peppering their pitches with words like "simple," "easy" and
"intuitive." Two of them - Macromedia Inc. and Dimension X - even told this
reporter that their software was so easy to master "your mom could use it."
We decided we had to see about that.
Of course, these companies aren't really testing mothers; they're after
professional artists with computer experience. And frankly, using mothers to
illustrate the height of computer illiteracy strikes us as pretty insulting.
But to find out if average people with little or no computer experience can
really make their own Web pages, we asked five companies - Microsoft Corp.,
Netscape Communications Corp., Adobe Systems Inc, Macromedia and Dimension X
- to bring their products to our offices in San Francisco. And we chose as
testers my mom, Elisa Rigdon, a 70-year-old retired clerk whose hobby is
landscape painting; and Muffrey Kibbey, a 43-year-old architectural
photographer who teaches at the San Francisco Art Institute.
Mrs. Rigdon had never used a computer before. Ms. Kibbey had used computers
to balance the books for her photography business but had never created
computer graphics or used the Internet.
Two companies that agreed to our test were aghast when they found out they
would have to face such neophytes. They agreed to go ahead only when we said
they could give half-hour tutorials before our testers spent 30 minutes trying
to create their own Web pages. To give the companies a further edge, I gave
my mother some last-minute coaching, and Ms. Kibbey explored the Web for 30
minutes before the test began.
Software for creating Web pages is a lot like graphics software for making
flyers or birthday cards. But behind the scenes, the Web programs create
documents in "html," or hyper-text markup language a programming language that
is the standard for the Web.
Macromedia, which targets computer-savvy artists, has a new $79 product
called Backstage Designer for creating Web pages. Both of our testers sat
back as Macromedia's director of corporate development, Jim Funk, steered them
through layers of directories to find pictures that could be inserted onto
their pages. Mrs. Rigdon made a few notes but looked confused. When it came
tome for the testers to fly solo, Mr. Funk declined to hand over the controls
to her completely, a blatant violation of the ground rules.
Across the table, Ms. Kibbey wanted to look at all the different background
colors and textures she could use for her Web page. Following the directions
of Macromedia's Anthony Wood, she clicked on four menus, finally pulling up
backgrounds that looked like crinkly paper and like paint applied to a rough
wall. Mr. Wood congratulated her, but Ms. Kibbey wasn't satisfied. "I want
to see them all," she said.
Eventually, she decided it took too many mouse clicks to view each option
and settled for drawing a purple square on her page. In the end, though, she
wasn't impressed. "They kept grabbing the controls away," she said.
Adobe was up next. The company's head of Internet products, Robert Seidl,
arrived to find that his rental computer had no keyboard or mouse. He raced
out to buy one, and 10 minutes later was showing off PageMill, a $99 program.
To personalize the experience, he photographed Mrs. Rigdon and Ms. Kibbey
with a digital camera and loaded the pictures into his computer. PageMill
works with a program called Scrapbook that lets you see an image before you
move it onto your Web page. Neither tester had any problem dragging her photo
from the scrapbook.
Then, suddenly, trouble. The testers wanted to change the sizes of the
photos, which required them to click on a tiny square. First, Mrs. Rigdon
failed several times. Then, Adobe's spokesman, Richard Brown, struck out.
Finally, Mr. Seidl succeeded - and promised to make the square bigger in
future releases of the software.
Next up: Dimension X's $99 product, Liquid Motion, a graphical program that
can make animations for the Web using Sun Microsystems' complex Java language.
The company's director of marketing, Munjal Shah, tried to get Mrs. Kibbey
to make a page using a picture of balloons and confetti as a background. But
Mrs. Kibbey didn't like it. After 15 minutes, with coaching, she figured out
how to make a globe spin. She had to work through several menus, filling in
blanks next to commands like "X position" and "frame rate," and checking a box
called "loop enabled." After several minutes of practice, she did it all,
mostly by herself, creating a page with a picture of the moon, twinkling stars
and bouncing words.
Across the table,Mrs. Rigdon yelled, "Uh-oh!" but then relaxed when Mr.
Brown said calmly, "Just hit delete."
Meanwhile, as Ms. Kibbey tried to save her image, her computer crashed. Mr.
Shah winced. "So close and yet so far," he said.
Mrs. Rigdon couldn't use Liquid Motion without help. She wanted to use the
balloons and confetti and then animate the words "Still Life in Confetti." To
do this, she had to go to the "edit" menu and then slide her cursor down to
choose "wander." But she had trouble getting to the right spot and suddenly
lost the entire menu. "What happened?" she asked. The Dimension X team fell
silent as she tried again, this time successfully.
Then her animation suddenly sped up and slowed down unpredictably. Mr. Shah
thought she might have found a bug. It turned out she had hit a wrong button.
Next, Microsoft presented its $149 Front Page, scheduled to ship this month.
It comes with an optional "wizards" feature that poses questions and the makes
Web page templates using the answers. "This is much easier to use," Ms.
Kibbey said.
She felt limited by the templates, though, which called for things like
descriptions of "current projects." And she she had trouble lining up a
section on hobbies with the section on projects because she had accidentally
put in a formatting code she couldn't remove. "To do it, you'd have to start
again," advised George Meng, Microsoft's trainer. "Then forget the damn
hobbies," Ms. Kibbey responded.
The testers both found Netscape's Navigator Gold product, which costs $79
and also has wizards, as easy to use as Microsoft's Front Page. But by then,
Mrs. Rigdon was so tired she had trouble locating the "OK" button she was
supposed to hit after each choice. Sometimes it was at the bottom of the
window, sometimes near the top. "Your eyes have to be like a billiard ball,"
she noted.
Ms. Kibbey made two pages with the Netscape program, one with a bright
orange and pink background, the other minimalist, with lots of empty space and
horizontal lines. But she was also tired as the test wound down.

After eight hours, the test was over. The testers' verdict: Making Web
pages isn't exactly a cinch. "I need at least six weeks to learn this," Mrs.
Rigdon said. Ms. Kibbey felt she had figured out the technology but wasn't
impressed with the Web's content. "That's the trouble with technology," she
says. "It attracts people with nothing to say."

To see the Web pages that were built by our testers, see The Wall Street
Journal Interactive Edition at <http://wsj.com