Creating and deploying DCQA processes presented challenges for which we have created the following novel solutions:
Some recent articles:
- Modeling the QA subtask configuration space:
We formally model aspects of the QA subtasks and underlying software
that will be varied under control of the distributed process. This
includes not only process and software configuration parameters, but
also constraints among them. To do this, we developed a general
representation with configuration options, option settings, and
inter-option constraints. We also developed the notion of temporary
inter-option constraints to help us reduce configuration space size
artificially in certain situations.
- Exploring the configuration space:
The configuration space of a QA process for a performance-intensive
infrastructure system can be quite large. Even with a large pool of
user-supplied resources, brute-force approaches may be infeasible or
simply undesirable. Consequently, we developed techniques to
explore/search the configuration space. We developed a general search
strategy based on uniform sampling of the configuration space and
supplemented it with customized adaptation strategies to allow
goal-driven process adaptation.
- Feedback: As
subtasks are scheduled and executed in parallel at several sites,
feedback (subtask results) is collected. QA processes can analyze this
feedback and modify their behavior based on it. We have developed
techniques for automatically characterizing such feedback, visualizing
it, and adapting the QA process.
- Unpredictable resource availability:
Since QA subtasks are assigned to remote machines, volunteered by end
users, we cannot know when resources will be available. Moreover, some
volunteers may wish to maintain some control of how their resources
will be used; for example limiting which version of a system can
undergo QA on their resources. In such cases, it is impossible to
pre-compute QA subtask schedules. Therefore, we have developed
scheduling techniques that adapt based on a variety of factors
including resource availability.
- Process management:
We developed a new process, called the Skoll process, that provides a
flexible framework to integrate the above mentioned QA techniques and
- Cemal Yilmaz, Atif Memon, Adam Porter, Arvind Krishna, Douglas C. Schmidt and Aniruddha Gokhale, Techniques and Processes for Improving the Quality and Performance of Open-Source Software. The Software Practice and Improvement Journal. 11(2), pp. 163-176, May 2006 .
- Adam Porter, Atif Memon, Cemal Yilmaz, Douglas C. Schmidt, Bala
Natarajan, Skoll: A Process and Infrastructure for Distributed
Continuous Quality Assurance. (submitted to IEEE Transactions on
Software Engineering, August 2005).
- Arvind S. Krishna, Cemal Yilmaz, Atif Memon, Adam Porter, Douglas C. Schmidt, Aniruddha Gokhale, and Balachandran Natarajan, Applying Model-based Distributed Continuous Quality Assurance Processes to Enhance Persistent Software Attributes, IEEE Software, Nov/Dec 2004.