Subjects were invited to the Beltsville Community Center where is a bimonthly meeting of 100-200 seniors. The test took place in a computer room with ~10 terminals. For the test, we provided a separate laptop to avoid installation issues. Unfortunately, the laptop did not have a regular mouse port, and the center did not have USB mice, so users were forced to use the touchpad on the laptop for the test. They were given instructions on how to use the touchpad, were told that the real application might be on a terminal or touchscreen, and were asked to ignore difficulties with the touchpad while evaluating the software.
The tests ran about 20 minutes each, in a one-on-one setting, with a
PhotoGenic team member on hand to provide assistance if the user ran out
of ideas for how to proceed. Users were encouraged to "think out loud,"
so that observations could be made about how they were interacting with
the software. Since the subjects were not accustomed to usability tests,
it was emphasized that this is a work in progress and that if something
doesn't work as they expect, it is a flaw in the software, not in them.
Even with that preparation, subjects still seemed reluctant to say that
they could not figure out how to do something, because of an apparent
perception that if they can't do it, they must be at fault.
Placing a Phone Call
Searching for a Person
Look at Pictures
Send an Email Card
Two of the subjects were male, and one was female, and each received a questionnaire for demographic information. All three used a computer and email at least several times a week. One was aged 40-49, one was 50-59, and one was 60-69. When asked they or someone they know had the experience of seeing a picture of a close family member and not being able to recall their name, one chose "rarely" (the youngest one) and two chose "occasionally".
The subjects looked at photo albums ranging from a few times a month to almost never. All three buy hallmark cards, and two of three write a separate message on the card.
When asked what they liked about the program, subjects cited ease of moving between parts of the program, and the association of pictures with emails and calls. When asked about areas of the program that need work, subjects mentioned the adding ability to set a larger font size, larger picture size, and the order of operations for the Send an Email Card feature.
Subject suggested more categories for the Send an Email Card feature, selection of photos by letter to jump to a name, selection of a location by clicking an icon (cell phone, for example) in addition to clicking the radio button underneath, and having a picture associated with an incoming email.
Other comments included: "Great idea", and "This program would be nice
to have for use at home or even at work. It would make life a little more
simple for people that do not use a PC or laptop too often." One user
reacted with "Oh, I like this" when the graphics for the email card came
up on the screen.
Observations and Problems
In general, the subjects were interested in the software, and thought that the idea was a good one. They seemed to understand the tasks, and they related to the tasks as things that they do on a regular basis anyway.
Two of the three users never clicked the help button, and the third only
did it out of curiosity after filling out the post-test questionnaire.
This might have been because a PhotoGenic representative was on hand for
the test and so they did not feel the need. None of the three spent time
on the welcome screen, but they were also not expressly asked to, and
they were given verbal instructions instead.
Names were not alphabetically listed. Importance 5, Effort 1.
Subjects clicked the button on the left side of the screen in attempts
to "reset" the view if they felt disoriented, and this did
not work. Importance 4, Effort 2.
Look At Pictures
It was not clear who the four people in the thumbnail view were (in the fullsize layout, the selection matched the currently visible person). Importance 4, Effort 1.
It was not clear if the caption on the button under the pictures was a description of the current state, or an explanation of the state that would be entered by clicking the button. Importance 4, Effort 2.
There was no integration between Look at Pictures and Send an Email Card, so if you wanted to send a card to someone whose picture you were looking at, you had to change modes using the buttons on the left. Importance 4, Effort 4.
One subject wanted names to be sorted by both last name and first name. Importance 4, Effort 3.
Subjects expressed a preference for seeing one picture at a time, but
appreciated having both views to choose from, since sometimes the name
is hard to remember, and sometimes the face is hard to remember.
Send an Email Card
Changing tabs before required information was entered showed the new tab with an error message, and then went back to the previous one, which was disorienting. Importance 4, Effort 1.
There were spelling errors in text of hallmark cards. Importance 5, Effort 1.
Suggested other categories: Congratulations, Thinking of You, Sympathy,
Thank You, Invitations. Importance 3, Effort 3.
Make a Phone Call
All subjects tried to click the pictures of Home Work, and Cell rather than the radio buttons. Importance 5, Effort 1.
One subject was very frustrated by the lack of a "Back" button.
The subject missed the desired name and had to cycle through the entire
list to find it, only to miss it again! Importance 5, Effort 1.
Subjects did not seem to understand what Cancel meant in the context of search. Perhaps "Clear" was what was intended? Importance 4, Effort 2.
One subject clicked the "Choose Person" to start from the beginning,
which worked, but clicking next showed the next person from before the
user clicked "Choose Person", i.e. where the subject had previously been
in the list. Importance 5, Effort 1.
Response to Feedback
The following changes were incorporated into the different components as a result of the student suggestions and usability test results.
Look at Pictures
Send an Email Card
Make a Call & View Diary