Evan Golub Department of Computer Science Human-Computer Interaction Lab University of Maryland College Park, MD 20742 firstname.lastname@example.org
Computer-based presentation systems can be a powerful tool for faculty in the classroom. Computer note-taking systems can be a powerful tool for students. This paper explores the idea of faculty using a computer note-taking system during student presentations to support interaction, evaluation and feedback. A note-taking system designed with students in mind is introduced. Its use as a faculty tool is then discussed within the context of a presentation assigned to student teams in an undergraduate course on Human-Computer Interaction. This technique can be extended to a wide variety of student presentation assignments.
K.3 [Computers and Education]: Computer uses in Education – Computer-assisted instruction (CAI). Computer and Information Science Education – Computer Science Education, Curriculum.
Design, Human Factors.
Classroom presentations, critiquing, evaluation, HCI education.
As we have moved to a more technology-based classroom, much attention has been paid to using computers to present material. Examples include video presentations, PowerPoint slides, Web-based presentations, and electronic whiteboards. As more and more students are coming to class with laptop computers, attention has been paid to questions of the opportunities this presents . An example of a project investigating taking notes in class on top of pre-prepared lecture material using a specific presentation system is StuPad , which is part of the Classroom 2000 project [1,6]. StuPad is built around their ZenPad presentation system. Another example is Classoom Presenter [2,6,10] from the University of Washington. It is designed around pre-prepared PowerPoint presentations and also investigates live feedback to faculty during class.
The proliferation of computers on both sides of the classroom as well as in students’ homes provides an opportunity to translate existing educational techniques as well as investigate new ones. In this paper I describe the BIRD note taking system , which I am developing for use with a wider range of presentation systems and without the need for advanced distribution of notes. I look at the question of how faculty, rather than students, can use this system in the classroom during student presentations to facilitate their evaluation, feedback and grading.
It is my belief that by using a computer-enhanced note-taking system, an instructor can better take notes during a student presentation. Good annotations and records of the presentation can assist in more organized and effective feedback both during class as well as after class. It can also be beneficial in the evaluation and grade-assignment stages of a faculty member’s work.
I will focus on a presentation-based type of assignment in an undergraduate Human-Computer Interaction (HCI) class as a case study on how to use BIRD to support short-term and long-term goals. This assignment is a Hall of Fame/Shame (HoF/S) presentation in which students discuss an existing user interface and justify placing it in either a User Interface “Hall of Fame” or “Hall of Shame”. Though much of the discussion in this paper is in the context of the HoF/S presentation, the technique can be generalized to other forms of student presentation. Another example is given in Section 7. Section 8 discusses the requirements for using this technique in the classroom.
I designed the Beacon-Identified Real-time Display (BIRD) note-taking system as a tool with which I felt students could more effectively and efficiently take notes in certain classroom situations. Specifically, in classes where an LCD projection system is used to display material at the front of the room (e.g.: PowerPoint slides, HTML outlines). In these situations, in order to have a context to comments they write, students have expressed to me that they often copy what is on-screen. While this might match some students’ learning style, others might prefer to be able to simply write only their own thoughts, but still copy the presented material to give context to their comments. While the intent of the BIRD project was to create a tool for a student note-taker, this paper addresses its utility to the faculty note-taker during a student presentation.
With the BIRD system, a server (Figure 1) is run on the presentation computer. The default action is to serve copies of the current screen image. If the computer has an extended desktop, the option for Secondary Screen will be active and the presenter can select which of the two screens they would like to serve. This is useful if using a presentation system such as PowerPoint that allows for two different views – one for the presenter and one for the audience.
Figure 1: BIRD Server on presentation machine with a TCP port entered along with a brief description of the server’s audience.
When started, this server emits a beacon to the local network with its network information to facilitate automatic discovery by the note-taking clients in the room. The beacon also contains a text message specified by the presenter to help students differentiate between multiple servers being run on the same network. It is also possible that the student might be on a different network than the presentation machine or that the local network is configured to block UDP broadcasts. For flexibility, the server application window will also display the IP address or addresses for the machine so that if students are not on the same network, that information is readily available to announce or write on the board at the front of the room.
When a request arrives at the server from a student’s note-taking client, the server captures the current contents of the specified screen and returns it as an image to the client.
At the start of class, the students using the note-taking client have their client search for servers on the local network (Figure 2). A list is returned and they may select the entry for their class. If it does not show up in the list, they can manually enter the address of the server if the presenter has made it available verbally or on the board.
Once the server is selected, any time that the student sees something being projected that they would like to annotate, they request a screen via the client (Figure 3).
Figure 2: BIRD Client displaying list of detected servers from which the student may select.
To request the currently projected screen, the user can press a button on the toolbar, select an option from the menu, or use a keyboard shortcut. When they do this, the client sends a request to the server for the current screen and the server returns the image. The note-taking client then creates a new page in the student’s notes.
Each page has two panes; the top one is able to display an image and accept ink, the bottom one is able to accept typewritten notes. Any ink added to the image pane that crosses into the space occupied by the image appears on top of the image. There are two categories of ink available; pen ink and highlighter ink (see Figures 3 and 4). Pen ink is opaque while highlighter ink is partially transparent. The user can configure their widths and colors.
Moving the splitter between the panes adjusts their heights. Any image and ink on the top pane is scaled when the pane is resized. The scaled ink is kept properly positioned relative to the scaled image. The text size in the bottom pane is not resized, though the user can configure the text’s font.
Figure 3: BIRD note-taking client with a PowerPoint slide captured and annotated with a highlighter and a pen, as well as typewritten notes.
When working on a page, the ink can be temporarily hidden to reveal full access to the image beneath it. If you would like to have a second set of comments for a particular screenshot, an additional page with a duplicate of the current page’s image can be added to the document. A new page without an image in the top pane can also be added for solely freehand drawing.
Expert reviews can be an important part of an undergraduate HCI course. Over the course of the semester, if students are expected to review their own projects or another team’s project, it is good to see existing software reviewed first. An assignment that is used in some HCI classes [3,8] is to evaluate an existing product via the Hall of Fame/Shame exercise. This gives them the opportunity to perform their first evaluation on a program whose creators are not in the class with them.
Students can either work individually or in teams of three or four students. The assignment can either be in the form of a written document or an in-class presentation. The assignment style that this paper addresses is teams giving an in-class presentation. Each team needs to agree upon an application to evaluate. For the presentation, they are told to introduce the application, its target users and its purpose and to then discuss the various aspects of the user interface within the context of human-computer interaction and usability. They either evaluate the full program or, as is more often the case, a significant and related subset of the program’s features. At the end of the presentation they place the application either in the “HCI Hall of Fame” or “HCI Hall of Shame”.
Presentations are given to the class, typically aided by a PowerPoint presentation used to organize the flow of the presentation as well as to show appropriate full or partial screenshots. Presentations are given at the beginning of class – one team per class. Each team is given 8-10 minutes to present their evaluation. This is followed by 5-7 minutes of questions and comments from the student audience and then several minutes of my own questions and comments.
The first few presentations each term are typically rather harsh evaluations, and are often even antagonistic towards the developers of the software. Presentations with this tone are often unproductive or even counterproductive. Having this type of undesirable presentation occur with an external product is a good way to learn through trial and error how to discuss flaws in a non-judgmental manner, before evaluating the work of fellow students. This is a key factor in avoiding the creation of in-class tension and outright competition and/or retaliation in later assignments. The experience of considering the issues they raise, as well as those their classmates raise, will also prepare them for some of the comments they will need to address about their own work. For many of the students, this is the first time that a peer will be evaluating their work.
The HoF/S assignment can be given either before or after the students have been introduced to techniques in expert review. Having presentations before the topic of expert reviews is covered allows students to consider in detail how they and others currently evaluate software, and then use that as a foundation as they are introduced to more formal methods. Additionally, earlier presentations allow students more time to consider how their term projects will be evaluated, possibly improving them as a result.
Student presentations are often fast and furious. A challenge is taking notes during the presentations that are detailed enough to be useful for evaluation and feedback while still concentrating on the presentation itself. To meet this challenge, I use BIRD during the presentations. Students use my laptop connected to the LCD projector in the classroom to give their presentations. I run the BIRD server on the laptop while they give their presentations. I bring a TabletPC to class as well. This allows me to run the BIRD client and capture the screen being projected at will during the presentation. Students in the class who bring their own laptops with wireless cards are also able to capture screenshots during the presentations and take their own notes.
Figure 4: BIRD note-taking client with student presentation screenshot and annotations.
During the student presentation, I sit in the audience with my TabletPC (12.1” screen set at 1024x768 resolution), with its wireless card on the same network as the presentation laptop’s card. As the students give their presentation I capture slides about which I have comments and/or suggestions.
Figure 4 shows an example of a screenshot taken during a team’s presentation. I have used the tablet pen to write comments on top of that screenshot regarding one of their bullets as well as the screen capture. I have also typed a general question/comment that I can use later either during the Q&A portion of the presentation, or when grading the team’s performance and giving them additional private comments.
A great advantage of this technique is that one does not have to invest time in drawing a sketch of the displayed image or attempt to write a detailed description of what their thoughts were at that moment. The image will provide context for the notes when reviewing them. This allows the instructor to pay more attention to the presentation as well as to make more comments during the presentation.
Although some of this experience could be supported by having students submit their presentations earlier, and then printing them out to bring to class, there are aspects that would be unavailable or less convenient. Aside from the additional paper-based filing overhead associated with pre-printing, there are also issues such as running out of note-taking room on the page for a particular slide or that of needing to bring a wide variety of pens and highlighters to class. Several additional issues will be addressed in context as appropriate in the sections that follow.
After the team has completed their presentation, there is then a question and answer period. During this, other students in the class and I have the opportunity to ask the team questions about specific parts of their presentation, or ask questions we felt were unanswered. I allow the other students to ask their questions first. As they ask their questions, I flip to the appropriate page in my notes so that I can either take the opportunity to enhance the question that has been asked, or check off that question from my list. I am also able to note the team’s reply, usually using a different color “ink” so as not to confuse those notes from the ones I took during the presentation. After the students in the class have had the opportunity to ask several questions, I ask a final question from my notes.
Once the Q&A period has concluded, the team is thanked and returns to their seats. At this point I spend several minutes reviewing the style of the presentation rather than the content. I do this by starting at the first page of the notes I took, and flipping through them looking for stylistic comments I made during the presentation. These are often about the layout or speed of the PowerPoint slides, but can also address public speaking issues. For example, if the team member presenting some portion is speaking to the screen rather than the audience, I will make a quick note to myself. Then, while I am reviewing the presentation for technical points I refresh my memory on the public speaking points I wished to make.
Each team receives a numeric grade for their presentation as well as the option of receiving additional comments in private discussion. In order to avoid being overly harsh or soft on the first few teams, I wait until a few teams have presented to refresh my memory on the varying styles and quality of presentations and then assign grades. In the past I would look through the copy of the teams’ PowerPoint slides side-by-side with my handwritten notes. However, having my notes integrated with the slides from the presentation decreased the time it took to review them. It also increased my level of confidence in my memory of the presentations.
For the private evaluation sessions, teams do not always immediately avail themselves of this opportunity. If a team comes a few weeks later, having all of my comments in the context of their presentation slides helps recreate the presentation scenario as well as my state of mind. While a paper-based system of taking notes during presentations would support this as well, it would have the additional overhead of filing and tracking these printouts in my office over the course of the semester. Additionally, on occasion a team has come to see me at the end of class rather than in my office. Using BIRD rather than paper, I can always have my notes with me and be ready to discuss them with the students.
This technique can also be applied in other presentation situations. At the end of the semester each team (different teams than the HoF/S teams) presents their term project. As with the HoF/S presentations, I used the BIRD system to take notes on the final presentations. In this situation the students do a live demonstration of their program while discussing their various design decisions. Since one of the aspects of the BIRD system is that rather than serving just PowerPoint slides, it will serve whatever is on the screen, the same note-taking technique worked.
Over the course of two weeks, I had 25 teams across two sections of the course present their term projects. At the end of the two-week period I was able to take an afternoon to write detailed evaluations for all 25 teams and assign grades for the presentations. By doing them all together I felt more confident in the consistency of my comments and evaluation.
I have also used this technique more casually in a colleague’s class where I acted as an outside observer during the class’ final presentations. We started the BIRD server on the laptop connected to the LCD projector and I sat with my laptop taking screenshots of the students’ work and slides, and then annotated them. With the commented screen captures, I could then quickly and easily select a question or two for each group at the end of their presentation. The notes were also useful to refresh my memory of the details of the presentations when it came time for me to give my opinion of their success.
The range of classroom presentations for which this technique could serve as a useful aide might be as wide as the range of types of classroom presentations.
The minimum hardware requirements to use this software in a classroom are the presence of two machines with the Microsoft .NET framework installed, which are able to communicate with each other over a network, and an LCD projector. The network environment can be either wired, wireless, or a hybrid configuration. This includes the option of an ad-hoc wireless network between the presentation computer and faculty note-taking computer. I have used the BIRD software in all of these scenarios.
One seemingly limiting factor is the need for two computers; a presentation computer connected to the projector at the front of the room, and a portable computer on which to take notes while watching the presentation. However, the BIRD server application is small (~5 Mbytes) and easy to install. As a result, the presentation computer could easily be a computer already present in the classroom or even the student’s own computer.
The BIRD system is not dependent on any particular software for presentation. Students can capture information presented via PowerPoint or a Web browser equally well. It is also possible to integrate this system with other electronic systems. For example, a faculty member using a Mimio  whiteboard capture bar could allow students to capture a copy of the current whiteboard as well. To accomplish this, the computer to which the Mimio is connected would need to run the BIRD server and set Mimio’s software to display the whiteboard image full-screen. Then when a student request came in to the computer, they would receive the current whiteboard contents.
The one current restriction is the need for the presentation computer to have the .NET framework installed. However, this is a short-term limitation since the communication model is http-based. Additional servers that do not make use of the .NET framework can be written for any platform that allows screen capture and IP-based communication.
9. FUTURE PLANS
My personal experience in using BIRD in the manner described was a success. I felt I was able to keep more organized and detailed notes. I was able to select questions to ask of my students with less effort and was able to better describe the context of my question based upon my notes. I was more confident in determining an appropriate grade as well as providing feedback to students after they received their grades.
In my first semester using this technique, I did not share my notes with the students in the class directly. After one presentation I was going to connect my TabletPC to the LCD projector to review my comments. However, when I mentioned that I was going to do that, several students (not only the ones in the team that had presented) said they’d prefer if I didn’t do that, so I did not. Still, few teams take the initiative to come to office hours for more detailed discussion of their presentation, so I do want to share my notes with them. In the Fall 2003 semester I plan to e-mail a copy of my notes to the students in the team that presented that day.
One issue that this has brought to the foreground is whether this would cause me to censor myself as I took notes. I have decided to take notes as I did in the Spring 2003 semester, but then review them after class and edit or remove any comments that I do not feel appropriate to give to the students in a “raw” form. For example, I find it is useful for me to draw a large red question mark and circle it enthusiastically if there is some presentation aspect about which I want to be sure to question the team. However, there is no need for the team to see that annotation when they read my notes. I will also take that time to expand upon my comments so that the students will not need to decipher my shorthand. I might also add references to class topics and/or sections from the book that the students should review and apply to their future work.
A useful feature I might investigate for the BIRD system would be to have two types of ink – public and private – to make it easier to share the notes with students. However, taking the time to switch between those modes while taking notes could be more cognitive load and distraction than it is worth.
Future work will include both expanded personal use, as well as investigations of other faculty’s use, of the discussed techniques. The Hall of Fame/Shame example is just one of many presentation assignments in which these ideas could be used to facilitate discussion, evaluation and feedback. Student reactions to this method of note-taking during their presentations, and the use of the resulting notes will also be investigated.
My thanks to Allison Druin, Ben Bederson, and William Gasarch for inspiration and support, and to Microsoft University Relations for the Toshiba 3500 TabletPC used in the classroom.
 Abowd, G., “Classroom 2000: An Experiment with the Instrumentation of a Living Educational Experience,” IBM Systems Journal, Special Issue on Pervasive Computing, Volume 38, Number 4, October 1999, pp.508-530.
 Anderson, R., Anderson, R., Hoyer, C., Simon, B., Videon, V., and Wolfman, S., “Lecture Presentation for the TabletPC,” Workshop on Advance Collaborative Environments, 2003.
 Bederson, B., Syllabus for a course in Human Factors in Computer and Information Science, http://www.cs.umd.edu/~bederson/classes/hci/syllabus.htm
 BIRD Note-taking System, http://www.cs.umd.edu/~egolub/AVIAN/BIRD.
 Campbell, A. And Pargas, R., “Laptops in the Classroom,”
Proceedings of the 34th SIGCSE Technical Symposium on
Computer Science Education, pp. 98-102.
 Classroom 2000, http://www.cc.gatech.edu/fce/eclass/overview.
 Classroom Presenter, http://www.cs.washington.edu/education/dl/presenter.
 Landay, J., Assignments list for a course in User Interface Design, Prototyping & Evaluation, http://guir.berkeley.edu/courses/cs160/spring2002/assignmen ts.htm.
 Mimio Whiteboard Capture System, http://www.mimio.com.
Simon, B., Anderson, R. and Wolfman, S., “Activating Computer Architecture with Classroom Presenter,” Workshop on Computer Architecture Education, 2003.
Truong, K. And Abowd, G., “StuPad: Integrating student notes with class lectures,” Proceedings of CHI 1999 Extended Abstracts, pp.208-209.