Title: Is the PSP useful as a vehicle for experimentation?

Proposer: Claes Wohlin, Lund University, Lund, Sweden

The PSP (Personal Software Process) is interesting in two aspects from
an empirical point of view.

First, it is necessary to find ways to evaluate if the PSP is effective.
Several success stories of how different quality aspects have improved
during a PSP course are available. Most of them is from universities,
but some successes are also reported from industry. Empirically, it is
necessary to evaluate if the PSP provides long-term improvements and to
show that it is not just due to a change in general. The problem of
evaluating the PSP as such will be addressed briefly. Some ideas will be
presented to stimulate a discussion concerning the subject.

Second, which is the main objective of the presentation, it is necessary
to evaluate if the PSP can be used to evaluate other methods and
techniques,i.e. to use the PSP to understand other issues in software
engineering. In the presentation, we will argue that the PSP is indeed
worth applying as a vehicle for performing experiments. The potential in
using the PSP as a vehicle for experimentation will be discussed based
on our experience.

We have taught the PSP to 100 students, and it provides a basis for
evaluating a number of issues, for example, background vs performance in
the PSP and different languages. Most of the students have used C, but a
significant number of people have used C++ and a hand full has used
Java.The langauge investigation which is currently conducted suffers
from one major drawback, namely non-random sampling; the students were
allowed to choose which programming language to use. Furthermore, the
external validity must be taken into consideration, what does certain
results using the PSP mean when trying to generalize the results, i.e.
to software development in general?

We believe that the PSP has a potential as a vehicle for
experimentation, if the experiments are designed properly. We would
like, through a presentation at ISERN, to stimulate the discussion
concerning using the PSP for experimental purposes. In the presentation,
we will highlight the disadvantages and advantages. Moreover, we will
discuss some of the issues which can be studied using the PSP, for
example, the use of different design methods, programming langauges and
reading techniques. Furthermore, we will provide some preliminary
results and lessons learned from our study concerning comparison of
diffrent programming languages using the PSP.

The major issue to discuss is the ability to use the PSP for small scale
experiments and how this can be used to draw more general conclusions. A
major problem in experimentation is the toy problem syndrome, i.e. we
conduct minor studies often in a university environment, and then we try
to project the results to industrial software development. Is the PSP a
step in the right direction to cope with this problem? The PSP is
designed to teach lage-scale software development on an individual
level. Can we use the PSP to do experiments and then scale back up
again?