Empirical Software Engineering

Model Driven Architecture (MDA) is a relatively new software design approach proposed by OMG. I was curious how much empirical research has been done on MDA or related topics, so I have done some survey. I have identified only a few so far, but I will update the list as I find more references.

B. Anda, K. Hansen, "A Case Study on the Application of UML in Legacy Development", ISESE'06, 2006.

"those who applied UML in modelling and enhancing legacy software experienced more challenges and fewer benefits from the use of UML than did the developers who modelled and developed new software"

Several experience reports have been presented at the MoDELS 2005 Conference.

The OOIS-MDSD Workshop mentioned the following topics in the call for participation:
* Empirical validation of the value of modeling, especially using the OMG's UML and MDA
* Empirical support for the effectiveness of model-driven and agile development approaches
* Empirical validation of the benefits of Agile development (XP, Scrum, etc.)
However, the actual program didn't seem to cover empirical results.

One of the most important insights gained from more than 30 years of history in software engineering history is that one solution never fits all: no single "silver-bullet" model, practice or technology works in all contexts. Appropriate solutions depend on the project goals, requirements, application domain and other factors, and it is common that a solution that worked wonderfully in one project results in a catastrophic failure in others.

Therefore, an important question is "what works when, and why." A factor that makes it difficult to answer this question is that software development is a human process. Due to the complexity and uncertainty of human activities, reasoning from theory is often insufficient to understand true implications.

The above discussion leads to a need of empirical research in software engineering. An empirical paradigm involves the following steps:
* Building models
* Experimenting
* Learning, encupsulating knowledge

Note that this involves an iterative improvement.