Usability Test Process

The process of conducting the usability tests was to create a list of pre-test and post-test questionnaires along with the actual interface usability test. We were able to find six participants to conduct the usability test. Unfortunately, all six were females, but none of the males that we asked would volunteer. Two of the participants had bipolar disorder. We had a list of people who suffered from urinary incontinence, but we did not ask these people due to privacy concerns. The participants ages consisted of, two in early 20’s, one in late 20’s, one in early 30’s, one in 40’s, and one in her 50’s. When conducting the tests, users were brought into an empty and quiet room. Each participant was required to answer a list of pre-task questions.

Pre-test Questionnaire

1) Rate often do you use a computer.

Monthly 1 2 3 4 5 6 7 Daily

2) Have you ever used a mood chart before?

Yes / No

3) Rate your current level of anxiety?

Very Calm 1 2 3 4 5 6 7 Very Anxious

4) Have you used a journal before?

Yes / No

5) If so, rate how often.

Seldom 1 2 3 4 5 6 7 Daily

6) Have you ever used a journal to track a health condition?

Yes / No

7) Would you consider using a computer to make daily entries in a journal?

Yes / No

8) Do you think keeping journal would be beneficial for you?

Yes / No

The results of the pre-test questionnaire for the subjects were very similar no matter the subject’s age group or whether they were bipolar or not.

Subject #1
Female, early 20’s
Pre-task questionnaire responses:
- Daily computer user
- Has never used a mood chart
- Level of anxiety: 2 (fairly calm)
- Has used a journal before
- Frequency of journal use: 4 (irregularly)
- Has never used a journal to track health condition
- Would consider using a computer to make journal entries
- Feels keeping a journal would be beneficial


Subject #2
Female, early 30’s
Pre-task questionnaire responses:
- Uses computer 3-4 times a week
- Has never used a mood chart
- Level of anxiety: 1 (very calm)
- Has used a journal before
- Frequency of journal use: 7 (Daily)
- Has never used a journal to track health condition
- Would consider using a computer to make journal entries
- Feels keeping a journal would be beneficial


Subject #3
Female, 60’s
Pre-task questionnaire responses:
- Uses computer daily
- Has never used a mood chart
- Level of anxiety: 3 (moderately calm)
- Has used a journal before
- Frequency of journal use: 5 (often)
- Has used a journal to track health condition
- Would consider using a computer to make journal entries
- Feels keeping a journal would be beneficial

Subject #4
Female, early 20’s
Pre-task questionnaire responses:
- Uses computer daily
- Has never used a mood chart
- Level of anxiety: 3 (moderately calm)
- Has used a journal before
- Frequency of journal use: 1 (Seldom)
- Has not used a journal to track a health condition
- Would consider using a computer to make journal entries
- Does not feel keeping a journal would be beneficial

Subject #5
Female, late 20’s, Bi-Polar
Pre-task questionnaire responses:
- Uses computer daily
- Has never used a mood chart
- Level of anxiety: 5 (moderately anxious)
- Has used a journal before
- Frequency of journal use: 6 (almost daily)
- Has not used a journal to track a health condition
- Would consider using a computer to make journal entries
- Does feel keeping a journal would be beneficial

Subject #6
Female, early 50’s, Bi-Polar
Pre-task questionnaire responses:
- Uses computer a few times a month
- Has used a mood chart
- Level of anxiety: 4 (not calm/not anxious)
- Has used a journal before
- Frequency of journal use: 6 (almost daily)
- Has used a journal to track a health condition
- Would consider using a computer to make journal entries
- Does feel keeping a journal would be beneficial

After answering the pre-test questions, the users were asked to experiment with the interface. There were two versions of the interface, for physicians and patients. Each subject was first asked to use the interface as a physician, then as a patient.


Physician:
1) Initiate a dailyLog for a specific patient
- Enter patient name: Bill D. Schafer
- Condition to be tracked: Bi-Polar disorder
- Continue by selecting “add patient”


2) Add questions to be used
- Select question already created from list: “How do you feel today?”
- Create a new question: “Did you take your medication as directed today?”
- Continue by selecting “continue”


3) Select the manner in which the patient will respond to a question
- Select question, “How do you feel today?”
- Click “Define Response Style Button”
- Choose from the options offered: select “mood chart entries”
- Click Done
- Select the question: “How many times have you urinated today?”
- Click “Define Response Style Button”
- Choose from the options offered: select “Creator Defined Responses”
- Enter “0 times” as a creator defined response.
- Enter “1 to 3 times” as a creator defined response.
- Enter “4-6 times” as a creator defined response.
- Enter “7-9 times” as a creator defined response.
- Enter “10 or more times” as a creator defined response.
- Commit Creator Defined Responses: select “Done adding responses”
- Click Done


When the subject used the interface acting as a patient, the task list changed.


Patient:
1) Retrieve a dailyLog entry
- Select “Patient Login”
- Select log entry dated 3/14/2003
- Open this entry

2) Make a new dailyLog entry page
- Select “Patient Login”
- Choose today’s date
- Continue by clicking “Create New Entry”

3) Respond to questions on an entry page
- Respond to question, “How do you feel today?”
- Complete the “Mood Chart entry” form that appears
- Set Anxiety level to “mild”
- Set Irritability to “none”
- Set Hours Slept Last Night to “7.0”
- Set Symptons slide bar between “no symptons” and “ellevated”
- Make a comment in Daily Notes about experiencing aches and pains
- Respond to question, “How many times have you urinated today?”
- Select “7-9 times”
- Click “Done”
- Click “Exit DailyLog PHR” to leave dailyLog entry page.

After the interface design test was finished, subjects were asked to complete a post-questionnaire about the interface. These questions consisted of:

Post-test Questionnaire

1) How would you rate your effectiveness of using the computer to perform the tasks you were given?

Ineffective 1 2 3 4 5 6 7 Very Effective


2) Did you encounter any of these difficulties? Circle the best answer for each one.

a) Disorientation when moving from screen to screen
Very Disorienting 1 2 3 4 5 6 7 Not Disorienting

b) Inability to skip through familiar areas of activities
Never Able to Skip 1 2 3 4 5 6 7 Can Always Skip

c) Inability to know if actions you performed were accepted.
Never Able to Know 1 2 3 4 5 6 7 Always Know

d) Inability to know if task is completed
Never Able to Know 1 2 3 4 5 6 7 Always Know

e) Inability to know how to correct errors
Never Able to Know 1 2 3 4 5 6 7 Always Know

f) Inability to reverse an action
Never Able to Reverse 1 2 3 4 5 6 7 Always Reversible

g) Feeling unattached from the task(s)
Never Feel Attached 1 2 3 4 5 6 7 Always Feel Attached

h) Remembering information from one screen that is needed for another screen.
Never Remembering 1 2 3 4 5 6 7 Always Remembering

3) Suggestions for improving the interface?

_____________________________ _________________________________

_____________________________ _________________________________


4) Do you think this activity is harder or easier when using the dailyLog instead of a paper journal?

Harder 1 2 3 4 5 6 7 Easier

5) Would you be willing to use a product similar to this if it were recommended by your physician?

Yes / No Why?_____________________________________

The result for each subject and observations are shown below:

Subject #1
The subject was capable of completing all tasks quickly and accurately. She expressed her concerns with the interface openly and included these opinions in her post-task comments. She expressed dislike for the manner in which a list box performed counter-intuitively to her, counting down instead of counting up. She also did not care for the multiple “done” buttons she was required to click in order to complete some tasks.

Post-Task questionnaire responses:
- Her use of the computer very effective for completion of tasks
- Did not find the interface behavior disorienting
- Was sometimes unable to skip familiar areas
- Sometimes did not know if performed actions were accepted
- Sometimes did not know if task was completed
- Most of the time knew how to correct errors
- Sometimes able to reverse actions
- Did not understand concept of feeling attachment to task (2)
- Most of the time able to remember information required in another screen
- Suggestions:
§ “Hours slept” arrow buttons reversed
§ Fewer “done” buttons to complete tasks
- Found tasks to be easier using the DailyLog
- Willing to use such a product, because its “Easy and neat”


Subject #2
The subject was capable of completing all tasks accurately. She hesitated occasionally and struggled with using the calendar interface to find the date “March 14, 2003”. She required the instructions be read repeatedly sometimes.

Post-Task questionnaire responses:
- Felt the interface was more effective for the physician aspect
- Did not find the interface behavior disorienting
- Did not have inability to skip
- Always knew when actions were accepted
- Always knew if task was completed
- Sometimes knew how to correct errors
- Mostly able to reverse actions
- Felt attachment to task
- Always able to remember information required in another screen
- Suggestions:
§ Be able to click regions instead of drag indicator for Symptoms
§ Directions for calendar use
- Found tasks to be easier using the DailyLog
- Willing to use such a product

Subject #3
The subject was capable of completing all tasks accurately. She had difficulty view the text in the interface, but was otherwise able to accomplish her goals.
Post-Task questionnaire responses:
- Felt the interface was effective for performing the tasks
- Found the interface behavior a little disorienting
- Did not have inability to skip
- Sometimes knew when actions were accepted
- Usually knew if task was completed
- Usually knew how to correct errors
- Mostly able to reverse actions
- Felt moderate attachment to task
- Moderately able to remember information required in another screen
- Suggestions:
§ Larger screens
- Found tasks to be easier using the DailyLog
- Willing to use such a product


Subject #4

The subject was able to complete the tasks but was slightly disoriented when using the arrows to increase the number of hours slept. She also did not realize when adding a question from the list whether or not the question had been added to the journal.


Post-Task questionnaire responses:
- Felt the interface was moderately effective
- Did not find the interface behavior disorienting
- Did not have inability to skip
- Always knew when actions were accepted
- Always knew if task was completed
- Always knew how to correct errors
- Always able to reverse actions
- Almost always felt attachment to task
- Almost always able to remember information required in another screen
- Suggestions:
§ None
- Found tasks to be slightly easier using the DailyLog
- Willing to use such a product
§ Felt it was more convenient

Subject #5

The subject was able to complete all the tasks although she also did not realize whether or not the question she added from the list had been added to the journal. This subject appreciated the language used and enjoyed the experience of using this type of journal on a computer.


Post-Task questionnaire responses:
- Felt the interface was effective
- Did find the interface behavior to be slightly disorienting
- Could only skip half the time
- Almost always knew when actions were accepted
- Only half of the time knew if task was completed
- Always knew how to correct errors
- Mostly of the time able to reverse actions
- Moderately felt attachment to task
- Moderately able to remember information required in another screen
- Suggestions:
§ None
- Found tasks to be much easier using the DailyLog
- Willing to use such a product

Subject #6

The subject was able to complete all the tasks given but took more time to complete compared to the previous two subjects. She had some difficulties using the slider for the mood chart entry.

Post-Task questionnaire responses:
- Felt the interface was mostly effective
- Did not find the interface behavior disorienting
- Seldom experienced inability to skip
- Half of the time knew when actions were accepted
- Almost always knew if task was completed
- Slightly more than half of the time knew how to correct errors
- Half of the time able to reverse actions
- Almost always felt attachment to task
- Almost always able to remember information required in another screen
- Suggestions:
§ None
- Found tasks to be easier using the DailyLog
- Willing to use such a product
§ Felt it was more efficient