Introduction

Research in computer science often involves the development of new systems, methods and techniques in order to solve problems. However, once developed there exists little experimental data to demonstrate the usefulness of the new approach. In many cases no comparison is made between a new development and the state of the practice. Although the new development could be very useful, it may not be transferred to industry since companies find it risky, time-consuming and expensive to introduce new technologies. There are always problems in the transfer of knowledge from science to industry but it becomes even more difficult without any experimental data.

The question is how to get valid experimental data. Even for scientists this doesn't seem to be a non-trivial task. There are several reasons for this:

1. There is no generally accepted method or process for conducting experiments in software engineering in order to get valid data although a lot of work has been done in recent years. Proposals for how to conduct experiments are given in [BSH86] and [Sel92].

2. There are a lot of aspects which must be kept in mind when conducting an experiment. If one important parameter is not taken into account, the results can be completely wrong.

3. At the moment computer scientists develop new techniques and methods proposing that they are useful and better than others. They don't provide any (qualitative or quantitative) data that support whether their new technique or method is effective. From a scientist' point of view there seems to be no reason to conduct experiments with students or in industry to gather data. The result is that the new development is either not used in industry or that it is used without knowing the risks and problems (Then a project is the experiment/case study).

4. Conducting an experiment to get valid data is expensive. It takes a lot of time and effort for both the participant and the experimenter.

5. Students don't learn how to assess new techniques because they don't know how to do experiments. That is directly linked to the problem that many software engineers themselves are not familiar with how to establish or evaluate a proper experimental design [FPG94].

6. The lack of metrics in the area of software development makes it difficult to conduct experiments. If you want to perform an experiment you will have to think carefully about the things you really want to measure and the metrics you want to use.

To sum up, much more experimental work must be done to move software engineering from a craft to an engineering discipline. To bridge the gap between software development and software engineering, scientists have to check the usefulness of their developments via experiments or case studies.

This lab package has three major objectives:

1. Description of a particular technique to study, perspective-based reading.
2. Documentation of the way a particular controlled experiment was prepared, conducted and analyzed in order to discuss various issues in experimentation.
3. Explanation of the experiment to make it understandable and repeatable for other researchers.

The process starts with the development of a new technique or method. This section describes the Perspective-Based Reading technique as well as the experimental context within which it is situated.

The next step is a controlled experiment or case study to validate the new technique or method. We include our experimental design in this lab package, as well as allow our experimental materials to be downloaded, in the hope that other researchers will be motivated to undertake replications of our experiment in other environments.

Finally, the data is analyzed and interpreted. We provide our analysis techniques and tools for other researchers. We also hope to maintain here a history of replications of this experiment, so that general conclusions about PBR may be drawn and the impact of environmental factors studied.

Web Accessibility