Draft: July 5, 1998
Ben Shneiderman (firstname.lastname@example.org, 301-405-2680)
Department of Computer Science, Human-Computer Interaction Laboratory, and
Institute for Systems Research
Ellen Yu Borkowski (email@example.com, 301-405-2922)
Academic Information Technology Services
Maryam Alavi (firstname.lastname@example.org, 301-405-2226)
College of Business and Management
Kent Norman (email@example.com, 301-405-5924)
Department of Psychology
University of Maryland
College Park, Maryland 20742
Emergent Patterns of Teaching/Learning
Novel patterns of teaching/learning have emerged from faculty and students
who use our three Teaching/Learning Theaters at the University of Maryland,
College Park. These fully-equipped electronic classrooms have been used
by 74 faculty in 264 semester-long courses since the Fall of 1991 with
largely enthusiastic reception by both faculty and students. The designers
of the Teaching/Learning Theaters sought to provide a technologically rich
environment and a support staff so that faculty could concentrate on changing
the traditional lecture from its unidirectional information flow to a more
collaborative activity. As faculty evolved their personal styles in using
the electronic classrooms, novel patterns of teaching/learning have emerged.
In addition to enhanced lectures, we identified three common patterns:
active individual learning, small-group collaborative learning, and entire-class
Many questions concerning the use of instructional technology may be resolved as researchers and instructors begin to provide feedback from innovative pedagogical explorations. Proponents of computers in every classroom do sometimes blindly promote technology without adequate vision. Similarly, opponents of technology sometimes raise unrealistic fears of alienated students, displaced faculty, and corporate sell-outs (Noble, 1998). While instructional technology professionals may have more realistic understandings, inflated expectations and unrealistic fears are still widespread.
The turbulence is likely to subside as all stakeholders become engaged in the process of choosing appropriate technologies, designing effective courses and curricula, training teachers, and setting new goals. We can only hope that wishful thinking and fear-provoking rhetoric will give way to clearer goals with well-conceived field trials accompanied by thoughtful evaluations.
We believe the confusion is brought on by the large number of provocative new technologies and appealing philosophies that are being promoted (Hofstetter, 1995; Jonassen, 1994; Laurillard, 1993; NAS/NRC, 1996; PCAST, 1997). Some of the technologies are:
- Email for individual discussions among students and instructors and email reflectors for entire classes
- Bulletin boards, newsgroups, and listservs for archived asynchronous discussions, often with threaded topics
- Chat rooms, MOOs, and MUDs for real-time interactions
- Websites with digital libraries, learning resources, and interactive course materials
- CD-ROMs or websites with simulation games and dynamic models
- Specialized educational or general purpose software for home, lab, and classroom use
- Video/audio conferencing for synchronous remote access
- Electronic classrooms for lectures and face-to-face collaborations
The plethora of technologies is matched by the diversity of pedagogical philosophies, including:
- Distance education, or tele-learning, by which students reduce their need to travel and can participate synchronously or asynchronously with other students and professors (Alavi, Wheeler, & Valacich, 1995; Harasim, Hiltz, Teles, & Turoff, 1995; Hiltz; 1994; Neal, 1997)
- Active learning and inquiry-based education, in which students investigate issues or solve problems with varying levels of human and computer guidance (NAS/NRC, 1996)
- Collaborative and cooperative learning, in which short- or long-term teamwork supports the social construction of knowledge (Davidson & Worsham, 1992; Edelson, Pea, & Gomez, 1996; Sebrechts, Silverman, Boehm-Davis, & Norman, 1995; Slavin, 1990)
- Service learning, in which students work on projects on campus or in their communities (Jacoby, 1996)
- Individualized, or self-paced instruction, in which students work
on their own using computer software that guides their progress and gives
feedback. The old terms such as "computer-assisted instruction" and "intelligent
tutoring systems" have given way to "interactive learning environments"
based on "learner-centered design" (Soloway, Jackson, Klein, Quintana,
Reed, Spitulnik, Stratford, Studer, Jul, Eng, & Scala, 1996).
Motivations for change are equally complex, including the need to keep up with livelier media such as television and to ensure that students are sufficiently proficient with computers to be employable. These motivators are broad, but when administrators sit down to plan, they split into two camps: those who argue for improved educational quality and those who see an opportunity for lower costs and larger markets (Gilbert, 1996). Advocates of quality often emphasize active learning and collaborative methods to promote greater student engagement with higher retention rates. Administrators who worry more about budgets are often attracted to self-paced instruction and distance learning, but unfortunately these are often euphemisms for computerized education and higher student-to-faculty ratios.
It is remarkable that many decision makers are lured into the fantasy that teachers can be replaced by technology. Neither books, television, or videotapes have replaced faculty, but the seduction of "intelligent tutoring systems" has lured some commentators to believe that this technology is different. While the technology can be wonderfully empowering for teachers and students, the relationship between human beings is still the heart of the educational process. The central premise is that "knowledgeable teachers provide challenge, guidance, and evaluation. They build a motivating and supportive environment while attending to the diverse needs of each student. The successful teacher conveys enthusiasm for and competence in the subject and the process of instruction, earning trust by presenting the right level of challenges for each individual and team" (Shneiderman, 1998a).
The best teachers create relationships in which students eagerly seek greater challenges and accept responsibility for their own education. Some students learn what they need by reading books, but a key function of a university or school setting is to enable the relationship between teachers and students to flourish. Teachers can employ technology to support the relationship directly by email and indirectly by providing students with access to remarkable resources (e.g., digital libraries or simulation models) and tools (e.g., word processor or music composition packages). Technology can support and strengthen relationships, but never create or replace them.
In addition to the controversy about the role of technology, another troubling aspect is the all-or-none attitude of many discussants and administrators. They seem to promote a single approach and seek a single answer. Imagine if in 1910, you had to choose among trains, cars, or planes as the single mode of transportation for the future. Cars were riskier than trains because no system of highways was in place and airplanes were just too fragile to carry large numbers of people. Similarly, no one pedagogical formula will dominate. It is likely that instructors and administrators will mix and match technologies and philosophies as each is refined over time.
Email is likely to be the personal car of the future. Most people will have it and use it daily. Listservs and websites might be the trains, with large numbers of regular commuters. Electronic classrooms might be the airplanes for special occasions because of their higher cost and complexity.
These metaphors are playfully suggestive, but they help sort out some
of the possibilities. Email is certain to have profound influence on education,
commerce, and many aspects of daily life in the coming century. Listservs
and websites seem likely to be widely used in creative ways. Electronic
classrooms are more of a mystery (Bruce, Peyton, & Batson, 1993). Proponents
believe that in the future every classroom desk will have a computer or
at least a plug for the student's personal laptop. They envision classes
in which students use the computers for much of the time in class to cooperate,
send messages, or browse websites. Electronic classroom builders may create
showcase environments that impress visitors or donors, but their pedagogical
vision is often lacking and implementation flawed. Skeptics are troubled
by the high costs and are likely to prefer traditional lecturing with student
questions or group discussions. Given these concerns, it seems beneficial
to conduct evaluations of electronic classrooms and the way that they are
used by teachers and students.
Electronic Classroom Goals
The University of Maryland, College Park, has made a long-term commitment to develop electronic classrooms to explore how they can influence teaching and learning in a range of disciplines. Our major goal has been to provide faculty with an environment in which technology and a support staff can be used to enhance and transform teaching, from its traditional unidirectional information flow to a more collaborative teaching/learning process. Our focus is not on the technology but rather on its use as a tool for promoting effective learning (Shneiderman, Alavi, Norman, & Borkowski, 1995).
These collaborative goals were only vaguely incorporated into our early expectations of electronic classrooms, but experience has gradually refocused our vision. As in many research projects, new opportunities and ideas emerged as we learned what worked and what didnít. Our goals were ambitious in that we were seeking to change a large community of faculty in diverse disciplines. To this end we prepared announcements and brochures, sent email notices, held seminars, and organized day-long Teaching with Technology conferences, to tell the story to our colleagues on campus and off. To date, 74 faculty have taught in these collaborative electronic classrooms, and most of them have rethought their philosophies of education. Some have gained new insights that inform their teaching even when in traditional classrooms, while a few passionately refuse to teach in traditional classrooms.
Not surprisingly, most faculty have evolved personal styles in using the classrooms. While these styles reflect individual subject matters and personalities, they have in common a shift to a more collaborative approach. New patterns of teaching/learning have also emerged from the way these faculty have used the classrooms. Faculty have become guides and coaches, while students have taken a more active role in the classroom.
In addition to enhanced lectures, we identified three common teaching/learning patterns: active individual learning, small-group collaborative learning, and entire-class collaborative learning. Some faculty emphasized active individual learning and reporting through individual students use of software tools to write, draw, simulate, search, etc., and then share their products by displaying them on the large screens in the rooms. Some faculty encouraged small-group collaborative learning by organizing small teams (usually 2- 5 students), while others emphasized entire-class collaborative learning by creating whole-class experiences based on brainstorming and groupwork. After overcoming initial problems with the technology and taking at least a semester to develop their personal styles, faculty report tackling more ambitious projects, giving their students more authentic experiences, and creating a higher level of engagement (Alavi, Wheeler, and Valacich, 1995; Norman, 1994a, 1994b; Shneiderman, 1993a).
Many of the lessons learned are in harmony with the
research on computing in education in elementary and secondary schools.
Beckerís analysis of K-12 teachers found that exemplary computer-using
teachers were located in a context of "collegiality among users, school
support for use of computers in consequential activities, resources allocated
to staff development and computer coordination" (Becker, 1994). He also
found, as we did, that major positive curriculum changes accompanied the
shift to computer use.
Electronic Classroom Infrastructure
The AT&T Teaching/Learning Theater, the first of our electronic classrooms opened in the Fall semester of 1991. It was made possible by a grant from AT&T to investigate how technology could improve the teaching process. The IBM-TQ (Total Quality) Teaching/Learning Theater opened in the Fall semester of 1993. This room was made possible by a grant from IBM as part of a TQM (Total Quality Management) in education project. The third classroom, the AITS (Academic Information Technology Services) Teaching/Learning Theater, was built in 1996 with existing building funds for the new building in which this classroom is located. Two more electronic classrooms are being built as part of a major construction project.
The electronic classrooms are supervised and supported
by the Teaching Technologies group of the University's Academic Information
Technology Services. The mission of the Teaching Technologies group is
to seek out new ways to use technology to improve the quality of education
across the University campus (see also http://www.inform.umd.edu/TeachTech/).
Academic units that have used the Teaching/Learning Theaters include American
Studies, Anthropology, Art History, Business and Management, Chinese, Civil
Engineering, Computer Science, Education, Electrical Engineering, English,
Government and Politics, History, Housing and Design, Library and Information
Services, Mathematics, Mechanical Engineering, and Psychology.
The AT&T Teaching/Learning Theater has four tiered rows of five custom-designed desks, each equipped with a computer (see Figure 1). Each desk can comfortably seat two students for a total capacity of 40 students. The desks are cantilevered so that there are no supports to restrict chair movement; the front row of desks is wheelchair accessible. The computer monitors are recessed into the desk tops to permit clear sight lines with the instructor and other students. Five-caster swivel chairs promote interaction among students. The instructorís console is designed to encourage experimentation with the layout of equipment and controls. A similar arrangement was used in the IBM-TQ Teaching/Learning Theater, which used a tiered double U-shape instead of rows (see Figure 2) (Allen, Bowen, Clabaugh, DeWitt, Francis, Kerstetter, & Rieck, 1996). Feedback from faculty indicated a preference for the double U-shaped configuration, therefore, the AITS Teaching/Learning Theater is in the same configuration as the IBM-TQ Teaching/Learning Theater.
The original student computers in the AT&T Teaching/Learning Theater were 386-based AT&T 6386 WGS; these computers were upgraded in 1995 to Pentium-based AT&T Globalyst 620ís with 16 MB of RAM, 570 MB hard disks and 17" monitors (1024 x 768 pixels). The instructor has two similarly configured computers at the front of the room. The original student computers in the IBM-TQ Teaching/Learning Theater were 486-based IBM Ultimedia computers with 16 MB RAM, 200 MB hard disks, and 14" color monitors; these computers were upgraded in 1997 to IBM Pentium-based 32 MB RAM computers. The instructor also has two similarly configured computers at the front of the room. The aITs Teaching/Learning Theater has Pentium-based Gateway 2000 computers with 32 MB of RAM, 2 GB hard disks, and 17" color monitors. The computers in each electronic classroom use Microsoft Windows, are networked on a local Novell network, and are connected to the campus optical-fiber network.
Each desk in our electronic classrooms contains a
monitor, keyboard, and mouse. The CPUís in the AT&T and IBM-TQ Teaching/Learning
Theaters are housed in a separate room -- an arrangement which reduces
the noise level in the classroom, allows for better control of temperature
for people and computers, improves security, and reduces equipment in the
classroom. This setup also allows quicker and easier access for repair:
a CPU can be replaced quickly, even while a class is in session. The monitor,
keyboard, and mouse are run over long cables (up to 45í) strung under the
floors and up through the desks. This setup does not allow students to
have access to the floppy drives of the computers. Our network environment
addresses this issue: each student who is enrolled in a class held in an
electronic classroom is given an account and some storage space. These
accounts can be accessed from any of the 30 open labs on campus. Printing
support is not available in the electronic classrooms, but that capability
is also available in the open labs. A single printer in the support room
is available for faculty use.
A touch-panel control box for the audio/visual equipment
and lights controls the equipment itself (e.g., starting/stopping the VCR)
as well as what is displayed on the projectors. Available audio/visual
equipment in each Teaching/Learning Theater include:
|2 high-resolution rear screen projectors||S-VHS video tape player|
|U-Matic (3/4") videotape player||laser disc player|
|broadcast television antenna||connection to the campus video cable|
|35mm slide projector (video image)||visual presenter|
|compact disc (CD) player||audiotape player|
|stereo speaker system||closed-captioning decoders|
A video recording system with three cameras, three ceiling-mounted microphones, and a wireless microphone allows instructors to videotape their classes and allows the teaching technician to monitor the classroom from outside the room. This system is linked to the Universityís campus cable system, allowing down-linked satellite programs to be viewed as well as allowing two rooms to be linked visually.
A critical control box, called the LINK box (Applied Computer Systems, Inc., of Johnstown, Ohio), allows for shared viewing of computer monitors by providing the following capabilities:
- Any screen on a student workstation can be displayed on the instructorís monitor and broadcast to all the studentsí monitors.
- The keyboard and mouse on any student workstation can be "taken over" by the instructorís workstation. Effectively, the instructor can take over control of any workstation in the room.
- Any monitorís display can be shown on the rear projection screens.
- All student screens can be "blanked" by the instructor.
When the AT&T Teaching/Learning Theater opened in 1991, little collaborative software for educational support was available. An on-line collaborative software product, a commercial group decision support system called VisionQuest (Intellect Corporation of Dallas, Texas) was adopted to enable faculty to devise innovative ways of using the softwareís tools in support of collaborative classroom exercises. Later, another collaborative software product called GroupSystems (Ventana Corporation of Tuscon, Arizona) was installed.
Over the years, the Teaching Technologies staff have developed "lectureware" tools as faculty helped the staff to discover what worked and didnít work in the classrooms. These "lectureware" tools include the One Minute Paper, Feedback Meter, MultiChat, Class Directory, and Caprina. This software is also usable from the open labs on campus.
· Feedback Meter enables students to click on buttons to indicate whether they are following the lecture or are confused. Instructors get a summary of the number of students in each category. This has proven to be more valuable at appropriate stopping points rather than continuously during the class.
· MultiChat allows students to "chat" with each other, anonymously or identified in small groups. An art history teacher integrates verbal and textual chat discussions of images that students find on web sites.
· Class Directory combines each studentís picture, name, and a personal biographic sketch into a format searchable by picture or by name.
· Caprina makes 15,000 high-quality color images easily accessible through a multislide program or a quiz slide program. Students can individually explore these images and then present them to the whole class as a slide show.
· Two faculty have developed their
own courseware. HyperCourseware (Norman, 1994a, 1994b) is a system of interlocking
modules that serves as a complete electronic infrastructure for classroom
learning by providing templates for lectures, exams, grading, and interactive
exercises (see http://www.hypercourseware.com). The TQ Classroom Software
(Alavi & Yoo, 1995) consists of a number of integrated and easy-to-use
student and faculty tools, including electronic information display and
note taking, an electronic seating chart, and a set of classroom evaluation
and feedback tools.
To take advantage of the special capabilities of the rooms, all three electronic classrooms are scheduled through a proposal process rather than by the registrar. Courses that are scheduled into the electronic classrooms for the full semester must go through a proposal process that ensures they are used as designed -- that is, to take full advantage of an interactive, collaborative, multimedia environment to support teaching and learning. A Teaching/Learning Theaters Steering Committee reviews the proposals to ensure the goals of the classrooms are being addressed. Composed of a dozen faculty from the various colleges and administrative units on campus, the Steering Committee is chaired by the head of the campus Center for Teaching Excellence.
A call for proposals to use the electronic classrooms is distributed
to the entire campus at least one semester in advance (usually a year in
advance). Acceptance is based on how the faculty plan to use the facilities
available in the room to foster a collaborative learning environment. Faculty
with proposals that show promise, but are weak in some area, are contacted
for further discussion and refinement. Where possible, a faculty mentor
is assigned to work with the proposer and the Coordinator of Instructional
Technology & Support provides additional support and training. The
Coordinator also resolves scheduling conflicts, supports short-term usage,
and prepares demonstrations for the steady stream of visitors. Once the
courses are selected by the Teaching/Learning Theaters Steering Committee,
the faculty are contacted to arrange for training and/or software development.
Clearly one of the keys to the success of the Teaching/Learning Theaters has been the support provided to the faculty and their teaching assistants (TAs) who use the rooms. Support has been an integral part of the electronic classrooms since the beginning: the guiding principle has always been that faculty should be able to come to a high-tech environment and focus on teaching the material for the day, not on fixing a "locked" computer (Yu & Gilbert, 1992).
Instructional support is provided before and during the semester through
a general orientation for all faculty who are selected to teach in the
electronic classrooms and several individual orientation sessions. The
Coordinator of Instructional Technology & Support provides as many
individual orientation sessions as needed. Technical support is provided
in every class throughout the semester. A student technician, assigned
to each class, provides day-to-day support and is always present during
the class. In addition, weekly "faculty prep" sessions are scheduled to
provide time for faculty and their TAs to come to the electronic classrooms
to work on materials or try software.
Many universities have been experimenting with electronic classrooms and lecture halls, language laboratories, and distance learning environments. Hiltz (1994) pioneered the use and research of virtual electronic classrooms supported by a computer conferencing system. Her focus is on asynchronous usage in a distance learning environment; her extensive survey, interview, and data collection methods are a model for all researchers concerned with electronic classrooms.
The University of Arizona in Tucson has an electronic classroom equipped
with 29 student workstations and three workstations for the faculty. The
workstations are IBM 486-based computers interconnected via a Novell local
area network. The Arizona classroom is furnished with a wide array of audio/visual
equipment, including three 10-foot diagonal high-resolution rear projection
screens (Alavi, Yoo, & Vogel, 1997). Other electronic classroom projects
include Northwestern University (Guo, 1995), the University of Notre Dame
(Stuebing, 1994), the University of Delaware (Hofstetter, 1995), and Gallaudet
University (Bruce, Peyton & Batson, 1993).
Courses Taught in the Electronic Classrooms
Summary of use
A total of 264 courses (Table 1) [a current list is maintained at http://www.inform.umd.edu/TT/Schedules/Classes.html] have been taught by 74 faculty representing 26 different departments over their first seven years (Fall 1991 - Fall 1997). In Spring 1998, 23 faculty representing 14 different departments will teach 30 courses in the electronic classrooms. A total of 7,514 students have enrolled in classes taught in the electronic classrooms (Fall 1991 - Fall 1997). Average class size was 28 students.
Insert Table 1 about here
The 74 faculty and graduate students who taught in the electronic classrooms
from Fall 1991 through Fall 1997, included 16 non-tenured faculty, 31 tenured
faculty, 9 graduate teaching assistants, 8 staff, and 10 instructors. They
represent the following 26 academic units on campus:
|Art History||Human Development (Education)|
|Arts & Humanities||Journalism|
|Behavioral and Social Sciences||Library and Information Services|
|Business & Management||Maryland English Institute|
|Curriculum and Instruction (Education)||Spanish|
|Government & Politics||Urban Studies and Planning|
The initial set of faculty who taught in the classrooms were very experienced computer users. Over the years, a mix of experienced and novice computer users have taught in the electronic classrooms.
It is also interesting to reflect on those who have
not used these electronic classrooms. A majority of those faculty who do
use the Teaching/Learning Theaters come from the Colleges of Arts &
Humanities, Behavioral & Social Sciences, and Business. In spite of
our publicity, there are still many faculty who don't know about the electronic
classrooms or believe that such facilities are meant for software and computing-oriented
courses. However, computer science faculty have been limited users, wrongly
believing that it would take substantial effort to prepare programs. Resistance
to innovation is an old theme, and many faculty continue to be skeptical
or negatively pre-disposed to electronic classrooms.
Use of the Electronic Classrooms
Patterns of Teaching/Learning
As faculty have evolved their teaching styles in the electronic classrooms, we recognized four emerging patterns of teaching/learning:
- active individual learning and reporting
- small-group collaborative learning and reporting, and
- entire-class collaborative learning.
During small-group collaborative learning and reporting, groups of 2-5 students solve a problem collaboratively or create a work product. As with active individual learning, the groupís product can be shared with the rest of the class. Entire-class collaborative learning is possible with certain software that allows the teacher to engage the whole class in an exercise based on brainstorming and teamwork.
Examples of the enhanced lecture style range from using a word processor to using a presentation software package. Several faculty who were novice computer users chose to lecture in a presentation-only style using a word processor. As these faculty became more comfortable with the technology, they provided their lecture notes to the students electronically. A professor teaching Chinese poetry started in a presentation-only style but soon moved to a presentation-with-discussion style. He used PowerPoint for his presentations in each class, going through 50-60 slides each period. PowerPoint provided him the capability to use color to emphasize different aspects of poems and to show different translations side-by-side. The instructor provided the same slides to his students to follow during class and for review outside of class. As this instructor became more familiar with the technology, he added discussion to his lecture, integrating the use of the One Minute Paper tool to provide feedback to him on students' comprehension. In the past, he asked for verbal feedback, but he reported that only a few students would respond, while most sat with blank stares. In the electronic classroom, every student would compose a response, creating a much more active learning environment. The instructor could much more accurately gauge student comprehension across the class.
With a computer at every desk, students have access to a suite of software for creating their own interpretations of the material being discussed. The LINK box provides the capability to share these interpretations with the entire class for discussion. These abilities fostered the emergence of the active individual learning and reporting style. Instructors from the Maryland English Institute -- which teaches courses in English as a Second Language (ESL), used this style frequently in class. A typical exercise consisted of showing a clip from a video tape or showing a picture on the electronic overhead and asking the students to write a paragraph in a word processor describing what they saw. The instructor used the LINK box to share what each student wrote anonymously with the class and to discuss grammar and writing style. This allowed students to learn from each otherís creative leaps and mistakes.
This active learning with reporting pattern was also successful in the other courses. Similar to what was done in the ESL exercise with grammar and writing, a business instructor asked, students to revise a business letter, which was then shared with the class for discussion. Sciences and engineering faculty would have students use software specific to their disciplines to create products in response to a problem. These products were then shared with the class for peer critiquing. For example, Visual Basic was used to create novel user interfaces that were critiqued for layout, color, terminology, and completeness.
Small-group collaborative learning and reporting were enabled through the use of software such as MultiChat. In a computer science course, in-class group programming was made possible by the use of the network in the electronic classroom. The professor composed a 20-line main program that invoked three procedures. The students then worked in small groups to generate versions of the procedures, and these were examined by the instructor using the LINK box. The instructor was able to copy the version he liked best, make a few slight changes, and then copy it into his main program on his machine. The program was compiled, and it ran on the first try. The instructor commented that "while he was enthusiastic, his students seemed to think that this was the natural way to do things."
Case-study exercises in business courses provide another example of small-group collaborative learning. Groups of 5-6 students were assigned business-oriented roles and given individual case studies to analyze. VisionQuest was used to support these exercises, enabling the group members to propose solutions based on their role. Then the solutions were refined, rated, and ranked before each group shared its decision with the entire class for discussion.
The collaborative software available in the electronic classrooms, especially VisionQuest and GroupSystems, has allowed for entire-class collaborative learning. VisionQuestís "brainwriting" and "comment cards" tools were used successfully in many classes. A business school professor frequently starts her classes by having students use the brainwriting tool to identify concepts and issues from the assigned readings that, in their view, need further discussion or clarification in the classroom. Once the issues are identified, she asks each student to pick the two or three issues that are central to understanding the topic. She then focuses her lecture and class discussion on the issues most frequently picked by the class as well as on those that, in her judgment, are necessary to complete and enhance student comprehension. This process of issue identification and prioritization takes about 10 minutes. In a similar manner, a Spanish professor used VisionQuest to have students list troubling vocabulary words and then elicit feedback about how troubling the words were by asking the entire class to rank them electronically. Students seem to like knowing how well or poorly their classmates are doing. As another means of beginning the class discussion, a speech communication professor asked students to list metaphors for the concept of conflict in VisionQuest which she then used as illustrations during the class.
A Spanish instructor used MultiChat to foster a combination of small-group
collaborative learning and a form of entire-class collaborative learning.
Groups of students had been assigned countries in South America to study
throughout the semester. At the end of the semester, they used the MultiChat
program to support a United Nations trade negotiation exercise: a UN room
was set up and each country had its own chat rooms. Students could chat
with the students in their group and make decisions on negotiation trade-offs
and then go to the UN room (where the entire class participated) to negotiate
with the other countries.
Empirical analyses of courses taught in the Teaching/Learning Theaters are limited and exploratory. However, many faculty are eager to understand how the Teaching/Learning Theaters affect their courses and they are eager to improve their teaching methods. Although such studies fall outside the approach taken in rigorous studies of instructional design, they are helpful to faculty and they provide provocative insights into uses of electronic classrooms.
One exploratory study involved a semester-long graduate business (MBA) course on management information systems (MIS) (Alavi & Yoo, 1995). The 20 students met once a week in the electronic classroom for two hours and forty minutes. Prior to each class, the instructor used the note taking tool to create and distribute an electronic version of her lecture notes to the students. During the class, each student was able to retrieve and annotate the instructorís lecture notes on his/her workstation. The student was then able to save the annotated notes on his/her private disk space for future reference. The classroom evaluation tools were used regularly throughout the semester to obtain anonymous student feedback on the classroom for the purpose of continuously improving the course content and delivery mechanism. VisionQuest software was frequently used by the class for brainstorming, evaluating alternatives, and collaboratively analyzing business cases.
At the conclusion of the course, the students were asked to respond to a questionnaire consisting of 25 structured items and two open-ended questions. The results indicated that the students rated their learning effectiveness in the electronic classroom significantly higher (significant at .05 level) than in the traditional classroom on all the items. Furthermore, the students were highly satisfied with their experience in the electronic classroom, indicating that they would take another course there. They reported that the computer activities in the classroom were well-planned, well-organized and enjoyable.
The students identified electronic note taking as the most popular feature. They responded favorably to the room's allowance for interactivity, idea sharing, brainwriting, and multimedia. Students also appreciated the computer/technology taking care of such mechanical aspects as saving notes, displaying the syllabus, etc.
Alavi (1994) conducted another study that compared the electronic classroom and a traditional classroom according to the learning outcomes and student evaluation of the learning process. The study involved 127 graduate business students, 79 of whom attended class in the electronic classroom (2 classes) and 48 of whom attended class in a traditional classroom setting. All three classes were taught by the same instructor. The course met once a week for a period of two hours and forty minutes. The instructor followed an identical set of classroom protocols and procedures was in each of the three classes. The primary difference between the courses was the use of VisionQuest in the electronic classroom to support student interactions during the collaborative learning group exercises.
A post-course questionnaire with 28 items was used to measure studentsí
perceptions of their learning and classroom experience. Five-point Likert-type
scales were used to measure all items. The learning and evaluation items
were subjected to separate principal component analysis, followed by varimax
rotations. Results indicated the presence of five factors (70.9 percent
variance explained) for the learning items and three factors (67.1 percent
variance explained) for the evaluation items. Means, standard deviations,
and correlations of the five coherent scales with acceptable alpha reliabilities
(3 from learning and 2 from evaluation) are presented separately by experimental
condition in Table 2. Alpha reliability coefficients for each scale are
provided in the diagonal. Means, standard deviations, and intercorrelations
with other scales for the expected grade measure are also presented in
Insert Table 2 about here
The findings of the study indicated that technology-mediated collaborative learning in the electronic classroom can lead to statistically significantly higher levels of perceived skill development, self-reported learning, and evaluation of classroom experience in comparison to collaborative learning in a traditional classroom. Furthermore, the final test scores of the group of students who were in the electronic classroom were statistically significantly higher (electronic classroom M=88.23, traditional classroom M=83.97, t(125)=-3.92, p<.001) than those of the other group of students who were in the traditional classroom.
Other exploratory assessments of electronic classroom effectiveness looked at the dynamics of the classroom lecture environment to determine ways in which multimedia can assist in the learning process and make education a more active and engaging pursuit (Alonso, 1995). In an experiment on learning statistical concepts, the instructor varied the form of control (learner-controlled versus instructor-controlled materials during the lecture) and the forms of interaction (passive/simple versus active/complex interaction). Although scores on a quiz indicated that subjects in the learner-controlled, complex-interaction group performed relatively well, they did not do significantly better than students in the other conditions. However, students in this group felt that the feature of student control and interaction were very important and did enhance their learning.
Borkowski (1997) conducted a case study to describe what happens in a technology-rich classroom (IBM-TQ Teaching/Learning Theater) in the Spring 1997 semester. Data were collected through participant observation, with the observer taking on the role of an active participant serving as an observer, technical support person, and, for one class, teacher. All classes were videotaped except for exam days. The videotapes were transcribed wherever possible.
The 28 students in the class completed a preliminary questionnaire that was used to gather demographic information, technology experience, and initial reactions to being in a technology-rich classroom. A final survey was given at the end of the semester to determine if the studentsí attitudes had changed over the semester in regard to the advantages or disadvantages of being in a technology-rich classroom. In addition, a mail reflector with only the studentsí and observerís email addresses was used during the semester to share student reactions/reflections on activities that occurred in class.
The student feedback from the surveys provided some insights into what needs to be taken into consideration when teaching a course in a technology-rich environment. Although the majority of students offered a very positive response in the preliminary survey to the question of anticipated advantages, the majority also anticipated disadvantages including issues of time (teaching time interrupted to address technology issues) and frustrations with having to learn technology. Studentsí expectations need to be addressed by informing them that the course they are enrolled in is going to be held in a technology-rich environment. In this study, not informing the students of this may have provoked some apprehension in those students who were not technology literate. We also had expected that the students would possess technology literacy skills, which was not the case. These reasons may explain many of the studentsí comments from the final survey in regards to the issue of the computers being a challenge to learn and that the students felt that the technology took away from learning the subject material.
One interesting observation is that the changes in the level of engagement
of the students increased when technology was used to support an activity.
Based on a review of the videotapes, the participation inherent in a technology-based
activity appears to stimulate more student participation in the face-to-face
discussions that follow. This is contrary to some findings in CMC (computer-mediated
communication) research, which showed no increase in face-to-face discussions
after CMC use (Kern, 1995; Ruberg, Moore, & Taylor, 1996). Further
research is required to understand how the combination of technology-based
activities with verbal face-to-face activities available in the Teaching/Learning
Theaters results in higher interaction levels and participation from all
students during face-to-face discussions that follow technology-based activities.
Surveys of Faculty.
Initially, the majority of faculty feedback was gathered through informal discussions held at the end of each semester. Later, a more formal questionnaire was developed and refined as the number of Teaching/Learning Theaters increased.
From the Spring semester of 1995 through the Fall semester of 1996, faculty were emailed surveys with 13 open-ended questions asking about what was successful and what was not. The survey probed for troubling aspects of class preparation as well as insights to teaching/learning opportunities.
Responding to these surveys was not mandatory. Response rate averaged 36% for the surveys collected from Spring 1995 through Fall 1996. Surveys submitted were forwarded to the Teaching/Learning Theaters Steering Committee for review. Starting the Spring 1997 semester, the format for faculty feedback was changed from an emailed questionnaire to a request for one-page self reflections on the semester. Faculty were asked to include a brief response to the following questions:
2. How did it go? Re: your goals/hopes
3. What were the strengths and weaknesses of your teaching experience in the theater?
Results of Faculty Surveys: Positive Aspects
Faculty reported that the level of engagement among students was very high. Allowing students to see each otherís work seemed to lead to a better expression of ideas among them. The technology seemed to capture studentsí imaginations and serve as a motivating force (Norman & Carter, 1994).
The sharing of student work was reported by the majority of faculty as a "successful" activity. A psychology professor found that when students typed their responses rather than spoke them, their answers were more well-thought out and better articulated. A computer science professor also commented that the quality of student work had improved. Faculty from all disciplines liked the ability to have students look at each otherís work. Some found the ability for the instructor to take control over studentsí work when necessary to be helpful in instruction. An English professor commented that being able to integrate media is "great": going from video to network to audio to oral to white board allowed the instructor to vary presentation mode according to content considerations and student needs.
Feedback from a mathematics professor, who taught a computer graphics course, noted that the network of computers, which is integrated with an array of audio/visual equipment, made teaching "easier and fun." He felt that the room was designed in such a way that the students and the instructor felt good to be in it. He claimed an attendance close to 100%. The instructor would come 30 minutes before class to play a VCR or laser disc with interesting computer graphics. Soon, students started coming to class early to watch the movies, play with the computers, and discuss the assigned projects amongst themselves.
Several of the government and politics faculty noted that familiarizing students with using the software happened fairly quickly. The LINK box enabled instructors to share student examples of what to do or not to do. Also, the Teaching Assistants could monitor the students from the front of the room electronically.
All the faculty agreed that additional time is required to prepare to teach in the electronic classrooms, but that it is well worth the effort. One education professor commented that it is "well worthwhile in terms of greater learning efficiency in the Theater." For those who have used the electronic classrooms several semesters in a row, preparing for class has become less demanding.
An education professor found that using VisionQuest and WordPerfect
along with the LINK box was most successful when there was a clear assignment
for students to prepare in advance and when the task was complex enough
to allow creativity but not so complex that brainstorming lists were too
long or had too much to be displayed and read. Especially successful were
her exercises on choosing a daycare center, applying research results,
and discussing gender roles. She also used the interactive capabilities
to construct the study list for the final and got into the habit of putting
lecture outlines on disk, loading them, and displaying them for the students.
Results of Faculty Surveys: Negative aspects
During the first several years of use, most of the feedback on the AT&T Teaching/Learning Theater related to physical setup of the room. For example, it was noted that certain color whiteboard markers were not visible from the back of the room (red and yellow in particular). A common comment was that the large instructorís console restricted movement. In the IBM-TQ Teaching/Learning Theater, which was built two years later, the instructorís console was designed to be smaller.
Lighting was, and is still, a problem: if it was dark enough to see the projection screens, it was too dark to see the whiteboard. There were complaints about the performance speed of some programs running in the IBM-TQ Teaching/Learning Theater; the upgraded machines have reduced this problem.
With all the technology in the electronic classrooms, it is a challenge for the support staff to keep all the equipment running smoothly. A government and politics professor cautioned, "be careful technology doesn't become the focus." Too many technical problems can disrupt a class. In addition, many of the faculty noted that they prepared backup lesson plans in case the technology did not work.
There were complaints about the poor resolution of the electronic overhead device, which has been replaced with a better unit. A professor who has taught in the electronic classrooms for several semesters commented that the touchscreen control panel for the media and lighting had enough complexity to make it unsettling even to long-term users. Several of the math professors complained that there was not enough whiteboard space. Faculty in non-technical disciplines had to grapple with the issue of how much class time to dedicate to learning the software in the electronic classrooms.
During the first couple of years that the Teaching/Learning Theaters were open, access to files from outside the electronic classrooms had been a problem. The process of attaching to the appropriate file server took too many steps and students experienced problems accessing their information. When the AT&T Teaching/Learning Theater opened in Fall of 1991, floppy disks with the appropriate files on them were given to students to use to login and have the correct access setup for them. This introduced many problems with wrong files being copied onto the floppies (different floppies for each course), floppies not working, etc. This process has been streamlined through the development of a software program which takes care all the necessary steps once the user enters their login id and password. Current problems with accessing files remotely relate to the complexity of having several Windows platforms available (Windows 3.11, Windows 95, and Windows NT) in the open labs for students.
Trying to do a traditional lecture in the electronic classroom proved
to be unsuccessful for one professor. A math professor didnít try to alter
what he did in a traditional classroom and tried to use the electronic
overhead and whiteboard combination for lectures and then class exercises
on the computer. The professor did not succeed because of the lack of whiteboard
space and the low resolution of the electronic overhead. In this case,
the instructor felt that lecturing in a traditional classroom was better
than in the electronic classroom.
During the first several years of use, feedback from students had been gathered informally through the use of VisionQuest software. A formal questionnaire, QUIS (Questionnaire for User Interaction Satisfaction), has also been used to gather studentsí feedback on the electronic classrooms. Results from the QUIS showed that in general, students have rated their learning experiences in the electronic classrooms consistently higher than their learning experiences in traditional classrooms. Students reported that they learned more, the class was more interesting, and that they were more motivated. They also felt they had a greater opportunity to be heard by the instructor and that they heard more from their classmates (Norman, 1992).
Written comments on the QUIS from students also provided insights into the advantages and disadvantages of the electronic classrooms. Students liked having hands-on experience with the software, and they liked the multimedia capabilities of the electronic classrooms. As noted above, students reported they had better class discussions/participation. Many commented that the rooms were comfortable. They felt that the big projection screens in the room were helpful. Written comments that the class was more interesting correlated with the results from the QUIS rating section.
Disadvantages that students cited were centered more on the comfort of the room than on educational or process issues. Many said there wasnít enough desk space. Inexperienced users needed a better introduction to the technology. Some students said that the computers were not used fully enough and some did not like the fact that they had to share a computer.
Starting the Fall semester of 1995, the student questionnaires were changed from the Likert-type scale questions contained in the QUIS to several open-ended questions that explored successful and unsuccessful activities.
The themes that emerged from these results were very similar to the
results of the QUIS in relation to successful activities. Activities such
as "hands-on" learning and group discussions were common themes from classes.
In addition students liked having access to information (i.e., instructor
notes, images, or course information). When examining the "unsuccessful"
activities mentioned by students, the focus has changed from the comfort
of the room as was mentioned in the QUIS results to the process of teaching
and learning. Students listed "plain" lectures as being unsuccessful and
complained that the instructor was "capturing" their screens for too long
(the LINK box allows instructors to stop student activities by sending
the instructor display to every student machine). Students also felt that
the technology was "taking away time" from the class. Students who found
it was harder to learn the subject matter cited the struggle between spending
time learning the technology versus spending time learning the course content.
There were many lessons learned about the electronic classrooms, from physical setup to technical support, to teaching and learning. The creation and use of these electronic classrooms have gained adminstration and faculty support so that additional electronic classrooms are being built. Instructional technology has become a major campus focus partially due to the success of these electronic classrooms.
Our key insight is the identification of the emerging patterns of teaching/learning:
- active individual learning and reporting
- small-group collaborative learning and reporting, and
- entire-class collaborative learning.
Having a high level of support appears to be the key to the successful use of the Teaching/Learning Theaters (Yu & Gilbert, 1992), but this support has a high operating cost. The costs of building these types of rooms has decreased -- it cost approximately $466,000 to build the AT&T Teaching/Learning Theater in 1991; $350,000 to build the IBM-TQ Teaching/Learning Theater in 1993; $150,000 to build the AITS Teaching/Learning Theater in 1996. The current staffing for support includes 2 full-time staff plus 14 part-time undergraduate and graduate students. Distributing total costs over the weekly usage of 60 hours per classroom, yields approximately $75/hr to support activities in these three electronic classrooms.
As was mentioned previously, the design of the instructorís desk was too large in the AT&T Teaching/Learning Theater. The subsequent instructorís desks were designed to be smaller. Lighting has usually been problematic in rooms that combine computers and projection capability. In the AT&T Teaching/Learning Theater, the staff went "overboard" in designing the lighting zones; there are eight separately controllable zones. Although presets were provided, the touch-screen controls were complex. The IBM-TQ Teaching/Learning Theater design has three lighting zones and a clearer interface to the controls. The setup of the desks in a U-shaped configuration is recommended over rows since it fosters better face-to-face discussions.
Electronic classrooms shift the role of the instructor from "sage on the stage" to "guide on the side." Faculty members can not only present information to the students; they can also devote more time to working with students on interpreting, integrating, and structuring masses of information generated or accessed in digital form. Sense-making and asking the "right" questions in information-intensive, technology-mediated environments are extremely important to promote comprehension, critical thinking, and learning. In addition, using technology in the classroom seems to empower and engage the students in their own learning (Alavi, Wheeler & Valacich, 1995).
Technology can play the role of equalizer by providing alternative and parallel channels for students to participate in the classroom discussions and provide feedback to each other and to the faculty. This equalizer role is particularly welcomed by students who shy away from speaking up in the classroom but who eagerly share their comments in electronic -- and anonymous -- form.
Faculty must be willing to change the way they teach to take advantage of the interactive environment available in the electronic classrooms. The few faculty who tried to come and do their usual lecture with overheads and the whiteboard and then used the room as a computer lab for exercises were not successful.
At this time, students still arrive on campus with varying levels of computer expertise. Faculty need to consider dedicating some time during the initial classes for providing computer orientation for many students. We anticipate that this will become less of a problem.
Credit is due to the University administration for
supporting the creation of these electronic classrooms before any proof
of success was available. Over the past seven years, the successes of these
facilities have argued strongly for the value of such resources for the
faculty. Future plans include an electronic classroom in a newly completed
building for the College of Agriculture and an electronic classroom for
the College of Behavioral and Social Sciences.
Our electronic classrooms and the new patterns of teaching/learning have reshaped educational processes for many faculty and students at the University of Maryland. Most early-adopting faculty are eager repeat users and there is a steadily growing community of faculty users. Other electronic technologies such as email and Web usage are spreading rapidly within electronic classroom courses and beyond.
Electronic classrooms seem likely to become more
common across our campus and at other universities, even though the cost
is high. The low cost and flexibility of email and web usage means these
technologies are likely to become universal, but many faculty and students
will probably seek out the special opportunities and intense learning experiences
that are possible in electronic classrooms.
Alavi, M., Wheeler, B. C., and Valacich, J. S. (1995). Using IT to reengineer business education: An exploratory investigation of collaborative telelearning, MIS Quarterly 19, 3, 293-312.
Alavi, M. and Yoo, Y. (1995). Education in the Information Age: The case of an Electronic Classroom, Supplemental Proceedings of the 28th Annual Hawaii International Conference on Systems Science, Advanced Tutorial on Electronic Support for Learning.
Alavi, M., Yoo, Y., and Vogel, D.R. (1997). Using Information Technology to Add Value to management Education, Academy of management Journal 40,5,1310-1333.
Allen, R. L., Bowen, J. T., Clabaugh, S., DeWitt, B. B., Francis, J., Kerstetter, J. P., Rieck, D. A. (1996). Classroom Design Manual: 3rd Edition, Available from Sue Clabaugh, Academic Information Technology Services, Univ. of Maryland, College Park, MD 20742.
Alonso, D. (1995). Forms of control and interaction as determinants of lecture effectiveness in the electronic classroom. Unpublished masterís thesis, Dept. of Psychology, University of Maryland, College Park, MD. http://lap.umd.edu/LAPFolder/Papers/dianesthesis/content.html
Becker, H. J. (1994). How exemplary computer-using teachers differ from other teachers: Implications for realizing the potential of computers in schools. Journal of Research on Computing in Education, 26, 3 (Spring 1994), 291-321.
Borkowski, E. Y. (1997). Classroom interactions in a technology-rich classroom: a descriptive study of a course taught in a teaching theater. Unpublished masterís seminar paper, Dept. of Education Policy, Planning, and Administration, University of Maryland, College Park, MD.
Bruce, B., Peyton, J. K., and Batson, T. (Eds.) (1993). Network-Based Classrooms: Promises and Realities. New York: Cambridge University Press.
Davidson, N. and Worsham, T. (1992). Enhancing Thinking Through Cooperative Learning, New York: Teachers College Press.
Edelson, D. C., Pea, R. D., and Gomez, L. M. (1996). The collaboratory notebook, Communications of the ACM 39, 4, 32-33.
Gilbert, S. (1996). Making the most of a slow revolution, Change: The Magazine of Higher Learning, 28, 2 (March/April 1996), 10-23.
Guo, G. (1995) NU Smart Classrooms at the Evanston Campus. http://www.nwu.edu/itg/smartclass/. Online. World Wide Web. 24 July 1995.
Harasim, L., Hiltz, S. R., Teles, L., and Turoff, M. (1995). Learning Networks: A Field Guide to Teaching and Learning Online, Cambridge, MA: MIT Press.
Hiltz, S. R. (1994). The Virtual Classroom: Learning without Limits via Computer Networks. Norwood, NJ: Ablex Publishing Corp.
Jacoby, B. and Associates (1996). Service-Learning in Higher Education, San Francisco, CA: Jossey-Bass Publishers.
Jonassen, D. H., Campbell, J. P., and Davidson, M. E. (1994). Learning with media: Restructuring the debate, Educational Technology Research and Development 42, 2, 31-39.
Kern, R. G. (1995). Restructuring classroom interaction with networked computers: Effects on quantity and characteristics of language production, The Modern Language Journal 79, 457-476.
Hofstetter, F. T. (1995). Multimedia Literacy. New York: McGraw-Hill.
Laurillard, D. (1993). Rethinking University Teaching: A Framework for the Effective Use of Educational Technology, London and New York: Routledge.
National Academy of Sciences/National Research Council (1996). Report on National Science Education Standards, Washington, DC.
Neal, L. (1997). Using multiple technologies for distance learning, In Smith, M. J., Salvendy, G., and Koubek, R. J., Design of Computing Systems: Social and Ergonomic Considerations, Amsterdam: Elsevier.
Noble, D. (1998). Technology in education: The fight for the future, EDUCOM Review 33, 3 (May/June), 22-34.
Norman, K. L. (1992). Defining and assessing usability in emerging systems: A Case Study of the Electronic Classroom. Proceedings of Usability Concepts and Procedures: Third Conference on Quality in Documentation, pp. 82-95, University of Waterloo.
Norman, K. L. (1994a). HyperCourseware for interactive instruction in the electronic classroom. Behavior Research Methods, Instruments & Computers, 26, 255-259.
Norman, K. L. (1994b). Navigating the educational space with HyperCourseware. Hypermedia., 6, 35-60.
Norman, K. L. (1997). Teaching in the Switched On Classroom: An Introduction to Electronic Education and HyperCourseware (http://lap.umd.edu/SOC/sochome.html), University of Maryland, Dept of Psychology.
Norman, K. L. and Carter, L. (1994). An evaluation of the electronic classroom: The AT&T Teaching Theater at the University of Maryland. Interpersonal Computing and Technology: An Electronic Journal of the 21st Century, 2, 22-39. http://www.helsinki.fi/science/optek/1994/n1/norman.txt
President's Committee of Advisors on Science and Technology, Panel on Educational Technology (1997). Report to the President on the Use of Technology to Strengthen K-12 Education in the United States, Washington, DC.
Ruberg, L. F., Moore, D. M., and Taylor, C. D. (1996). Student participation, interaction, and regulation in a computer-mediated communication environment: A qualitative study, Journal of Educational Computing Research 14, 3, 243-268.
Sebrechts, M. M., Silverman, B. G., Boehm-Davis, D. A., and Norman, K. L. (1995). Establishing an electronic collaborative learning environment in a university consortium: The CIRCLE project. Computers in Education 25, 215-225.
Shneiderman, B. (1993a). Education by Engagement and Construction: Experiences in the AT&T Teaching Theater. Keynote for ED-MEDIA 93, Orlando, FL.
Shneiderman, B. (1993b). Engagement and construction: Educational strategies for the post-TV era, Journal of Computers in Higher Education, 2 (4), 106-116.
Shneiderman, B. (1998a). Relate-Create-Donate: An educational philosophy for the cyber-generation, Computers & Education 31, 1.
Shneiderman, B. (1998b). Designing the User Interface: Strategies for Effective Human-Computer Interaction: Third Edition, Reading, MA: Addison-Wesley Publ. Co.
Shneiderman, B., Alavi, M., Norman, K., and Borkowski, E. Y. (1995). Windows of opportunity in electronic classrooms, Communications of the ACM 38, 11 (November 1995), 19-24.
Slavin, R. (1990). Cooperative Learning: Theory, Research, and Practice, Englewood Cliffs, NJ: Prentice-Hall.
Soloway, E., Jackson, S. L., Klein, J., Quintana, C., Reed, J., Spitulnik, J., Stratford, S. J., Studer, S., Jul, S., Eng, J. and Scala, N. (1996). Learning theory in practice: Case studies of learner-centered design, Proc. ACM CHI'96 Conference, ACM, New York, 189-196.
Stuebing, S. (1994). Campus Classroom Connections. Newark, NJ: New Jersey Institute of Technology.
Yu, E. and Gilbert, W. (1992). Supporting Faculty
in a High-Tech Teaching Environment. Proceedings of ACM SIGUCCS User
Services Conference XX: Learning from the Past, Stepping into the Future,
ACM, New York, 271-276.
Figure 2 Layout of the IBM-TQ Teaching/Learning Theater
Cultural Themes in America: American Cyberculture**
Diversity in American Culture*
Electronic Communication and American Culture*
Growing up in America**
Material Aspects of American Life*
Seminar in American Studies**
Quantitative Approach to Applied Anthropology**
Art History and the New Technologies: Research, Teaching, and Communication**
Byzantine Art and Archaeology**
Arts, Humanities, and Literatures in Nineteenth and Twentieth Century Europe: The Idea of the Modern**
Arts, Humanities and Literatures in Seventeenth- and Eighteenth Century Europe: The Identity of the Artist**
Cross-Cultural Perspectives on Quality**
Cross Cultural Differences**
The Role of the Media in the American Political Process**
Accounting Information Systems**
Business Computer Application Programming**
Business Problem Solving Using Computers***
Decision Modeling and Analysis***
Human Resource Management***
Information Systems Analysis and Design II***
Information Technology and Corporate Transformation***
Introduction to Business Information Systems**
Introduction to Design and Quality*
Management Information Systems***
Marketing Research Methods***
Methods of Measuring Quality*
Problems and Applications in Human Resource Management***
Seminar in Decision Support Systems**
Special Topics: Accounting Information Systems***
Special Topics in Business and Management: Electronic Commerce***
Strategic Information Systems***
Survey of Business Information Systems and Technology**
Systems Analysis and Design**
Operations Analysis for Construction**
Human Factors in Computer and Information Systems**
Introduction to Programming*
Teaching with Computers**
Selected Topics in English and American Literature**
Selected Topics: The Computer and the Text: Multimedia as Critical Expression**
The Computer & The Text: Hypermedia as Critical Expression**
Written Text, Visual Text**
Introduction to International Negotiations*
The Evolution of American Business, 1825-present**
Quantitative Approaches to 19th Century U.S. Social History***
Honors Seminar: Technology and Decision Making in the 21st Century*
Honors Seminar: The Information Age**
Switched on Minds: Cognitive Issues in HyperSpace*
Synthesis of Human Development Concepts***
Computers and Archival Administration***
Instructional Development Roles for Library Media Specialists***
Euclidean and Non-Euclidean Geometries**
Selected Topics in Analysis: Computer Experiments in Chaotic Dynamics***
Advanced English as a Foreign Language***
Advanced Oral Communications Skills***
Advanced Writing for International Students***
English as a Foreign Language***
English as a Foreign Language: Intermediate III***
Reading and Writing for International Students***
Saving the Bay***
Writing for International Graduate Students***
Introduction to Dynamics***
Experimental Psychology: Cognitive Processes**
Information Technology and Instruction***
Seminar in Human Performance: The Human/Computer Interface***
Statistical Methods in Psychology*
Theory and Decision of Choice***
Thinking and Problem Solving**
Commercial Spanish II**
Planning Housing Environments for an Aging Society**
Table 1 Courses Taught in the Teaching/Learning Theaters
(*Lower Division Undergraduate **Upper Division Undergradutae ***Graduate)
Perceived Skill Develop-ment
Group Case Evaluation
|Perceived Skill Development||
|Group Case Evaluation||
Table 2. Means, Standard Deviations, Correlations, and Reliabilities
for Study Variables, Separated by Experimental Condition (1=worst, 4=best)
Note: Correlations above the diagonal are for students in the Teaching/Learning Theater exposed to the computer-mediated collaborative learning; those below the diagonal are for students in the traditional classroom exposed to verbal collaborative learning. Alpha reliabilties are presented on the diagonal.