According to the current design of the experiment, we end up with four groups of people for each document; design, test-, and use-based, and the group applying the usual NASA technique. Let us denote the number of subjects in each group as , , and respectively. Now, since these subjects did the tests on an individual basis, they need to be combined into ``artificial'' teams in front of the analysis.

When combining individual results into team results, the union of the defect coverage of the various team members is considered the team defect coverage, and the defect detection rate is computed accordingly. The problem is the assignment of subjects to teams. Initially, we considered two approaches:

- Combine the individuals in all possible ways forming one team
where all members applied the ususal NASA techniqe, and another team
where all members applied PBR with one member from each of the three
perspectives. Both teams had three members. This will give us:
* *

- The other solution is to make sure we have statistical
independence between the teams we form. Thus, each individual data
point can occur only once, and this will give us only
( + + )/3

()/3

In order to find a solution to this problem, we contacted Chip Denman and Paul Smith from the Statistics Lab at the University of Maryland. They suggested that we applied a permutation test as explained below.

Mon Jun 24 13:58:35 EDT 1996