| Usability Test Process |
|---|
Our goal when running the test was to have the users run a series of tasks in a very relaxed environment. We broke our tests into a five step process. We began with a simple introduction of ourselves and our systems. The users then took a written pre-survey that included basic personal information such as age, weight and when they discovered diabetes. At that point we had them run through a series of pre-determined tasks and recorded thier comments, suggestions and actions. We gave them a post-survey that asked them for their thoughts and opinions on the experience and finished the process by allowing to voice and additional comments they may have. We pooled our users from the people we knew in our day to day life. They included men and women, ages 21-60, of a eclectic cultural background. One of the tests was actually administered over the internet to user in another country! We also attempted to interview users who were patients of an endocrinologist that we know, but because of HIPAA regulations, we could not find out who those potential users might be. Here is the usability test that we used with our testers: Introduce/Thank Start out by introducing yourself and thanking the tester for coming to help you. This may be a good time to offer them something to eat or drink. Try to make them feel as comfortable as possible and let them know that you appreciate them donating their time to you. Explain/Outline Explain to them what our system does, or more specifically what its supposed to do. Let them know that we hope to make the interface better and welcome their input into any potential drawbacks of the current design as well as suggestions for improvements. Make it clear that the system is being tested, not them; problem are expected/invited and they signal a fault on our side not a deficiency of the user. Warn them that some part of the interface are not complete and they may not be able to FULLY complete all tasks and that is ok. Briefly outline the process that they are about to engage in: A written pre-test, a series of observed tasks and finally a written post-test. Ask them if they have any questions and once their questions have been answered ask them to sign the paper. Pre-test(Page 2) Hand them the pre-test questionnaire and writing utensil and ask them to fill it out. Allow them to skip any questions if they feel uncomfortable and you may provide as much assistance as they require, just make sure the answers are their own. Run Tasks(Page 3 & 4) Start the system and give the tester the driver’s seat. Allow them a moment to look at the screen and become comfortable. Giving them as much time as they require, read each task to them and ask them to carry it out. DO NOT read the steps of the tasks to them, we are trying to see if the system is designed well enough that it can be used without explicit directions. At this time take COPIOUS notes on what they accomplish, what confused them, what mistakes they made, any comments they make, what they seem to like/dislike, ANYTHING of relevance. If they become completely stumped with a task you may show them the NEXT step they must take, then allow them to move on from there – again observe and record their actions and comments. Post-test(Page 5) Once they have finished all the tasks, give them the post test and writing utensil and ask them to answer the questions. This should run similar to the pre-test. Closing Thank them again for their time and ask them if there is anything they would like to add or comment about. Ask for final thoughts or suggestions. Thank them again and allow them to continue their day. Pre-test Questions 1. What is your age? 2. What is you native language? 3. Do you consider yourself techno-savvy? 4. How long have you had diabetes? 5. When you learned that you had Type I diabetes, what were some of the most difficult things to do? Why? 6. How well do you know your condition and what steps you need to take to maintain balance? 7. Do you think a tool that helps you to forecast your BGL over the next few hours would be useful? 8. Are you a vegetarian? 9. Do you exercise a lot? If so, how many hours do you average exercise a week? 10. How often do you record your BGL? 11. How relevant for you is to know your BGL? (From a scale of 1-10) 12. What are the most and least frequent functions you use with your BGL testing device? Why? 13. What are some of the most difficult experiences you have had when getting a reading? Tasks Reminder: This test focus on the usability of the device, there is no hidden or implied test on the user. The following tasks assume that you are using the DFS with normal healthy condition. It is your first time using this device, and you are trying to perform several tasks using the basic features of the device. Task 1: Change a Default Setting. You are a user in a Spanish speaking country and you need to change the language setting so that you can read the device. Please find the change the language on the device. 1. Turn on the device and enter the setup screen. 2. Locate the Language setting 3. Change the setting from English to Spanish 4. Exit the setup after saving the setting. Task 2: Update Personal Information. Go to the settings menu and update your personal information. 1. This one is hopefully fairly self explanatory. Task 3: Test Your Blood Glucose Level You are doing the routine task of obtaining your BGL with this device, how would you go about testing you blood. (Explain the hardware of the device and tell the user to tell you what steps they would take physically..ie. taking blood and where they would put it etc.) 1. Turn on the device. 2. Go to the get blood section. 3. Insert the testing strip to put blood into system 4. Wait for reading Task 4: Check the Last Record of BGL You want to see the BGL that you tested yesterday. 1. Turn on the device. 2. Go to the history section. 3. From the list of dates and time, pick the specific one you want to see. Task 5: Enter Food Information and Insulin You just ate breakfast and took both long and short insulin. Test your BGL, enter both the BGL reading and food information into the system, and get a forecast for when you need to check again. 1. Perform a Simple BGL Test 2. Enter Food menu 3. Enter Food data and exit to main menu 4. Enter Insulin Menu 5. Enter Insulin Data then press Forecast button. Did the result show up? Do you understand what the reading says? i.e. Did the system provide useful suggestion and prediction that can help you know when to check your BGL again? Task 6: Export your data to your Computer You want to send the result of the forecast to your doctor by sending the information to his email address. Suppose the device is wireless enabled. 1. Turn on the device 2. Click Export on the main screen. Task 7: Edit Exercise List You like to participate in Water Polo. This is not an option on the exercise list. Please update the list so that it includes Water Polo. 1. Enter the settings screen(or the Exercise Screen) 2. Click add exercise 3. Enter Water Polo and Some level of Calories 4. Click save Post-test Questions 1. Did you like the device? If you are given a scale from 1 (least satisfied) to 10 (most satisfied), what would you rate this device? 2. Was doing data entry easy and efficient or was it complex and time-consuming? 3. Were the graphics appropriate? Did you find the text sizes and font styles easy to read? 4. Was the task time consuming, tedious or difficult? 5. Was the prediction informative and helpful or was it too vague? 6. Were you satisfied with the number of screens that you had to pass through to get to the screen you wanted? 7. Were the instructions or the icons presented by DFS ambiguous, confusing or easy to follow? 8. Were you satisfied with the feedback you were presented after entering data? 9. Were you allowed to fix errors (e.g. change the mistyped information)? 10. Do you think that you would remember how to do similar tasks as the ones you have performed in the future? If not, please state which kind of task that would require you most practice. 11. What are the most and least useful parts of the device in your opinion? Please tell us the reasons as well. The most definitive statement that can be made about usability tests is that they take more time than expected. Between finishing the prototype and working around schedules this deliverable took almost three times as long as expected. Tight and solid scheduling is important because one delay can have a domino effect that leaves the group with dry periods of waiting followed by late nights spent furiously scampering to get things done. We felt we got a lot out of our usability tests, with a number of problems coming to light that seem blatantly obvious in hindsight. The most common and noticeable problem was the ambiguity of the icons on our toolbar, which resulted in almost every user either wasting time pondering over their usage and meaning or wasting clicks navigating down digital dead-ends. The second most common issue is almost embarrassing: the way we display our predictions is far too vague to be truly useful. That prediction is the major piece of innovation with our product and fixing this issue is of the highest importance to us. We also discovered bugs in our implementation and were offered some very useful suggestions about how to streamline data entry. We plan on implementing as many of the suggestions as possible because we found we agreed with the users in most cases. The actual running of the tests was a learning experience. We were able to run some very fun tests with the technology at our disposal, including running the prototype right off a webpage and doing an entire test with someone in another country through text messaging and video conferencing! All of the user’s were eager to help, but it is very important to start getting as much feedback as soon as possible. We found that as the experiments wore on the users got more and more comfortable expressing their opinions to us and offering suggestions. If given the opportunity to run the tests again, we’d add an emphasis to verbal questions and answers at the beginning of the test and devise some critical thinking or creativity evoking questions just to open communication lines as much as possible before we move onto questions and tasks that we consider critical. |
| To see the results of our tests, please download the following pdf (also on our downloads page): Usability.pdf |
| Diabetes Forecast System |
|
|---|---|
| Blood Glucose Level Measuring Technology |