next up previous contents
Up: Experiences and Change History Previous: Experiment with NASA SEL

Two replications of the experiment at the University of Kaiserslautern

A summary of two runs of the two replications can be found in the summary report.

The Participants

This experiment was replicated twice. The subjects of the replications were graduate students of the Computer Science Department at the University of Kaiserslautern, Germany. They were enrolled in the basic Software Engineering course lasting a semester. This course teaches the basic Software Engineering principles. The course is supplemented by practical exercises. In the first replication 25 students, in the second replication 26 students participated. All were volunteers.

The subjects were assigned to the PBR roles randomly.

The Techniques

PBR was used with the three perspectives described in this lab package: designer, tester, user.

The comparison technique was an Ad-hoc reading approach. i.e., no specific guidelines were given to students how to read the document. It is a non-systematic technique in which reviewers have general and identical responsibilities.

The Documents Used

Only the generic documents were used in both replications.

The Setting

The experiment was conducted in a classroom environment at the University of Kaiserslautern. Participants reviewed the documents during one and a half hour review sessions, during which they were isolated from distractions and interruptions.

Other Changes


Comments from Participants

After the replications, we solicited comments from the participants. Most students suggest to increase the time available for individual defect detection

Summary of Results

The replications corroborate the results of the previous experimens in Maryland. PBR teams were seen to have an improved (statistically significant at the 0.05-level) coverage of defects, for the generic documents. It was also shown that individuals using PBR performed better on the generic documents than Ad-hod reviewers. As this result was only statistically significant for the second replication, we perfomed a meta-analysis of our replications and the original experiments. It turned out that the effect of PBR on individual defect detection effectiveness is statistically significant.

Web Accessibility