CMSC 434 - Phase #3 - Spring 2017
Evaluation, Presentation, and Customer Attention

Phase 3 will have three parallel sub-phases.
   Phase 3.1 (PDF) due April 23rd before 11:59pm.
   Phase 3.2 (PPT) due April 17th before 11:59pm and in-class presentations
      taking place on April 18th, 20th, 25th, 27th.
   Phase 3.3 (video) due by class time on May 4th.


General Notes:

  • Parallel sub-phase 3.1 will be a heuristic evaluation of another team's Phase 2.2 prototype and the design of a user study about it. We will inform you of the "who evaluates which" soon after Phase 2.2 is turned in.
  • Parallel sub-phase 3.2 will be a team presentation about their own prototype. We have assigned teams to the dates April 18th, 20th, 25th, and 27th.
  • Parallel sub-phase 3.3 will be a short "crowd-source fundraising" video about your own team's prototype.

Overview:
The evaluation sub-phase is not connected to the presentation and fundraising video sub-phases.

The immediate purpose of the evaluation sub-phase is to give you experience at:

  • performing a heuristic evaluation of a prototype with a team
  • designing a user study
  • optionally (for extra credit on this phase) piloting your user study with a small number of participants and reporting on it
In addition to turning it in to us on ELMS, your group will also deliver your report to the team whose Phase 2.2 you are using for your Phase 3.1. However, your heuristic evaluation and usability study design (and report) will be used to determine your Phase 3.1 grade. They will not be used to alter the other team's Phase 2.2 grade. They will, however, be used by the other team as part of their Phase 4 work.


The overall purpose of the presentation and fundraising video sub-phases are to give you experience at:
  • presenting the strength of your work and your accomplishments to a team of peers and supervisor in a meeting
  • selling your team's idea and product prototype to a (hopefully) nascent customer base


For parallel sub-phase 3.1 you will need to provide us with several things, all submitted as a PDF.

  • your team members need to undertake a heuristic evaluation of the prototype provided to you by the other team; this will require each individual on the team undertaking an evaluation (the first section of the PDF), and then the team coming together to combine their findings into a single report (the second section of the PDF)
  • your team will design a user study, providing us with a detailed description of the plan (the third section of the PDF)
  • optionally (for some extra credit) your team can pilot the study you designed with two or three participants and provide us with a summary of what you learned from them about the prototype (as the fourth section of the PDF)
For the heuristic evaluation, the members of your team will undertake the role of HCI experts brought in to review a prototype. Because of the economy of these methods, you are expected to be able to apply them in your actual work practices. The other team can (and should) provide you with a list of tasks their prototype supports, but your evaluation can also address things such as any in the horizontal levels not mentioned in their tasks if you see them as common tasks or vertical aspects that you can tell aren't in the current design but you feel need to be.

The methods used in the usability study you design can include strict observation, think-aloud, constructive interaction, questionnaires, and interviews. Your team needs to determine the techniques it feels will be best used for the user study of the project you are assigned, but the expectation is a mixture of methods.

If you choose to undertake the pilot user study, in your write-up summarize what your study revealed about the prototype, compare the different techniques you used within the context of thinking about future usability studies (what worked well and what didn't in this pilot study). Then recommend the methods you wish your group to use in the future. Which was most useful? Which was least useful? What would you keep? What would you throw away? Also, please include the original observation notes and/or raw data in the form of questionnaires, etc. for context.


For parallel sub-phase 3.2 you will need to provide us with a slide set (submitted as a ppt or pptx on ELMS by April 17th) and give a live in-class 10-12 minute presentation followed by a short Q&A period. All presentations will be run from the room PC to avoid issues with personal laptops, etc. If you want to confirm things look the way you think on it, let me know in advance. If you plan to do a demo as part of your presentation (as opposed to showing screenshots in your slides) I would encourage you to also let me know in advance so you can test them on the room PC or how using the document camera works to show something on a mobile device or connecting your laptop to the secondary cable just for the demo part. Please rehearse your presentation before the submission deadline and then again before your presentation slot.


For parallel sub-phase 3.3 you will create a short "crowd-source fundraising" video around your project. It will be no more than 5 minutes long and should generate interest in viewers (as well as their money to help kickstart the theoretical full implementation phase of your project's development).

Remember that this video is not about how you made the project, but rather about why it's a useful product for the user. Users generally do not care which coding languages you used (or will use) but rather they care about how the product will benefit them.

We encourage you to explore some of the most popular kickstarter campaigns which all have example videos that can help provide your team a "feel" for what your video might be like.

Some General Tips:

  • Keep it simple/direct.
  • Avoid using long, continuous shots (use different camera angles and even locations).
  • Don't include paragraphs of text in the video, but short phrases and important words can be put in.
  • If using a smartphone, don't record your video vertically.
  • Stick with a clear message throughout the video (think of the "brand of your product") (create a logo? tagline?)
  • It's probably useful to include audio in the background of the video.

  • Some thoughts on potentially easy to use programs are Windows Movie Maker, iMovie, and Lightworks. Some more intermediate/expert programs are Final Cut Pro, and Adobe Premiere.

    Some possible sources for photos and music might be http://unsplash.com and http://www.bensound.com/royalty-free-music/


    We recommend that you divide your team into two sub-teams; one sub-team could focus on the slide show and the other could focus on the fundraising video.




    "Who did what" reports to be turned in on the same day as the video is due.
    • Public who did what: You should turn in a one page summary of who in the group did what. For each person, explain what portions of the project they worked on, what they wrote, portions of code, documents, etc. This is a public document that you must all agree upon.

    • Private who did what: Each person should individually upload to ELMS a PDF document describing the role and value of each person on the team. In this phase, different students might take on very different roles as agreed to by the team, so the question is rooted in whether everyone did the things that they agreed to do, to the level they agreed to do them. As part of this, you are to imagine that you have $100 to "pay out" to people on your team or to donate to charity. You need to tell me how much of this each of your team members will get and how much will go to charity. For example, if there were three other people on your team and they all did what they were supposed to, you could give everyone $33 and $1 to charity. If two people did what they were supposed to and one did not contribute, you might give $33 to each of the first two, only $10 to the last, and then donate $24 to charity. If one person wasn't doing their share and someone else picked up that slack (other than you) then you could give that person $40 and the person who did their share $33 and the person who underperformed $10 and $17 to charity. You get the idea... Any member of a team who does not upload this assessment and payment information will lose points on their part of the project grade, so everybody please remember to do this!


    Grading Note
    The elements of this phase will be worth 11 of the 40 percentage points that the team project makes of the semester grade.

    Updates
    If any updates to the description are required, they will be announced on places like Slack as well as the class website.








    Web Accessibility