Next: Experiment at
NASA SEL
Up: Experiences and Change History
Previous: Experiences and Change History
A summary of this run of the experiment can be found in our
summary report.
The Participants
This experiment was conducted with 12 software developers from the
Software Engineering Laboratory (SEL) at the NASA/Goddard Space Flight
Center. All were volunteers.
The subjects were assigned to the PBR roles based on their experiences
as testers and developers. This was done in order to avoid the
situation where one group consisted only of testers and the other only
of developers. However, this could not be done perfectly since there
was not an equitable breakdown of designers and testers.
The Techniques
PBR was used with the three perspectives described in this lab
package: designer, tester, user.
The comparison technique was the usual reading technique used at
NASA/Goddard. This technique had evolved over time, and was based
upon recognizing certain types of concerns which were identified and
accumulated as a set of issues requiring clarification by the
document's authors. It is a non-systematic technique in which
reviewers have general and identical responsibilities.
The Documents Used
The generic documents were used along with two actual flight dynamics
documents from the NASA SEL domain.
The Setting
The experiment was conducted in the normal work environment at the
NASA SEL. The participants were subject to all the distractions and
interruptions of a normal work day, and asked to keep a log of the
time spent on reviewing the documents.
Other Changes
Because this was a pilot study, the experimental design was somewhat
different from the one presented in this lab package. In particular,
- The time limit on each review was set at three hours, rather than
two. (Since only one subject took more than two hours, the time limit
was subsequently shortened.)
- Training sessions were provided only for the generic documents,
and not for the domain-specific.
Comments from Participants
Although we had assumed that the subjects were already familiar enough
with the domain-specific documents that no training sessions would be
necessary, subjects felt that training was warranted. Therefore
training sessions before the domain-specific documents were added to
the experimental design for future runs.
Summary of Results
PBR teams showed a significant improvement over usual technique teams
for the generic documents (at the 0.1-level of significance).
However, PBR teams performed about the same as the usual technique
teams on the NASA documents.
There was no significant change due to reading technique in scores for
individuals on either the generic or NASA documents.
An important result of this pilot study was the chance to improve the
experimental design. A list of these change is included as part of
the change history of the 1995 experiment.
Next: The Experiment: 1995
Up: Experiences and Change History
Previous: Experiences and Change History
Generated by latex2html-95.1
Mon Jun 24 13:58:35 EDT 1996
Web Accessibility