User testing

Overview

The project #3 is a continuation of Project #2. Its main purpose is to give you hands-on experience applying some of qualitative evaluation concepts we discussed in class.

What you have to do

For project #3, your group will design and conduct a qualitative evaluation of the system you built as Project #2. More specifically you will have to:

• Identify at least 3 major tasks you wish to evaluate,
• Design a protocol for your qualitative evaluation. Your protocol must at least include a direct observation evaluation, a post-observation survey and debriefing,
• Conduct the evaluation for at least 4 subjects (you might need to debug your protocol on member of your group before running subjects),
• Analyze the information gathered during the qualitative evaluation, and identify which part of your design should be changed.

Deliverables

Keep your project in a 3-ring binder. Major sections should be indicated by index tabs. Your final portfolio should contain: 

• A detailed explanation of the design rational behind your protocol (3 pages), including the overall goals of your study, how the tasks you picked relate to this goal, how the users you picked relate to this goal, and how your survey questions will help you understand user needs better (40 points),
• For each user, key observations during the testing session, his or her completed survey and a summary of the debriefing session (20 points),
• The key findings of your evaluation and for each of them how do you reach this conclusion from the data you collected (20 points),
• A new prototype (storyboard or Pictive) of your interface (20 points).

Grading

Grades will be based on the quality and sophistication of the design of your protocol as well as the quality of your analysis (see above). We strongly recommend that you consult either the TA or the instructor before starting to run the experiment so that you can be sure to get the most out of the data you will collect.