CMSC838M: Advanced Topics in Software Testing
This course will examine advanced software testing techniques. In particular, the important phases of testing will be reviewed, emphasizing on the significance of each phase when testing different types of software. Students will learn the state of the art in testing technology for object-oriented, component-based, concurrent, distributed, graphical-user interface, and web software. In addition, closely related concepts such as model checking and program analysis will also be studied. Emerging concepts such as test-case prioritization and their impact on testing will be examined. Students will gain hands-on testing/analysis experience by proposing new solutions to open research problems in the field of software testing and experimentally demonstrating the strengths/weaknesses of their solutions.
By the end of this course, students should be familiar with the state-of-the-art in software testing. Students should also be aware of the major open problems in testing.
|
Is the course valid for PhD qualifying coursework? |
Yes (Software Engineering/Programming Languages) |
|
Is the course valid for MS qualifying coursework? |
Yes (Software Engineering/Programming Languages) |
|
Is the course valid for MS comps? |
Yes. (Both Midterm and Final exams count towards the MS comps.) |
Tue.Thu -
Tue.Thu -
CMSC 435 or equivalent.
(Subject to change; additional readings may be assigned)
Lecture
#
|
Date
|
Topic |
|
1, 2, 3, 4
|
Aug. 31, Sep. 2, 7, 9
|
Course
Overview |
Mary
Jean Harrold Proceedings
of the conference on The future of Software engineering May 2000 Introduction to special section on software testing R.
Hamlet Communications
of the ACM
June 1988 Testing: principles and practice Stephen
R. Schach ACM
Computing Surveys (CSUR) March 1996 Software safety: why, what, and how Nancy G.
Leveson ACM
Computing Surveys (CSUR) June 1986 Validation, Verification, and Testing of Computer Software
W.
Richards Adrion , Martha A. Branstad , John C. Cherniavsky ACM
Computing Surveys (CSUR) June 1982 |
5
|
Sep. 14
|
Tools Overview I |
A tutorial &
hands-on training by Shradha Upadhyay
|
6
|
Sep. 16
|
Finishing up the Overview |
|
7
|
Sep. 21
|
Tools Overview II |
A tutorial &
hands-on training by Shradha Upadhyay
|
8
|
Sep. 23
|
Tools Overview III(slides) |
A tutorial &
hands-on training by Kunal Harsha Shanishchara
|
9
|
Sep. 28, Sep. 30, Oct.
5, Oct. 7
|
Taxonomies of techniques,Test Case Generation(Slides)
(Slides) (Slides) (Slides) (Slides) (Slides) |
Rethinking the taxonomy of fault detection techniques Michael
Young, Richard N. Taylor Proceedings
of the 11th international conference on Software engineering May 1989 A test generation strategy for pairwise testing The category-partition method for specifying and generating
fuctional tests T. J.
Ostrand , M. J. Balcer Communications
of the ACM June
1988 Hierarchical GUI test case generation using
automated planning Using a goal-driven approach to generate test cases for GUIs
Atif M.
Memon, Martha E. Pollack, Mary Lou Soffa Proceedings
of the 21st international conference on Software engineering May 1999 Predicate-based test generation for computer
programs |
10
|
Oct. 12
|
Test Adequacy and Coverage |
Software unit test coverage and adequacy Hong Zhu,
Patrick A. V. Hall, John H. R. May ACM
Computing Surveys (CSUR) December 1997 The evaluation of program-based software test data adequacy
criteria E. J.
Weyuker Communications
of the ACM
June 1988 Volume 31 Issue 6 Coverage criteria for GUI testing Atif M.
Memon, Mary Lou Soffa, Martha E. Pollack ACM
SIGSOFT Software Engineering Notes, Proceedings of the 8th European software
engineering conference held jointly with 9th ACM SIGSOFT symposium on
Foundations of software engineering September 2001 |
11
|
Oct. 14
|
Data-flow Analysis, Data-flow Testing (Slides) |
Interprocedual data flow testing M.
Harrold, M. Soffa ACM
SIGSOFT Software Engineering Notes, Proceedings of the ACM SIGSOFT '89 third
symposium on Software testing, analysis, and verification November 1989 An applicable family of data flow testing criteria |
12
|
Oct. 19
|
Test Oracles (Slides) |
Automated test oracles for GUIs Atif M.
Memon, Martha E. Pollack, Mary Lou Soffa ACM
SIGSOFT Software Engineering Notes, Proceedings of the eighth international
symposium on Foundations of software engineering for twenty-first century
applications
November 2000 Specification-based test oracles for reactive systems Debra J.
Richardson, Stephanie Leif Aha, T. Owen O'Malley Proceedings
of the 14th international conference on Software engineering
June 1992 |
13
|
Oct. 21
|
Test Coverage II (Slides) |
Experiments on the effectiveness of dataflow- and
control-flow-based test adequacy criteria |
14
|
Oct. 26
|
Regression Testing & Software Engineering and
Testing (Slides) |
An empirical study of regression test selection techniques
Todd L.
Graves, Mary Jean Harrold, Jung-Min Kim, Adam Porter, Gregg Rothermel ACM
Transactions on Software Engineering and Methodology (TOSEM) April 2001 Atif M.
Memon, Mary Lou Soffa September
2003 Proceedings
of the 9th European software engineering conference held jointly with 10th
ACM SIGSOFT international symposium on Foundations of software engineering |
15
|
Nov. 2
|
Midterm
Exam |
|
16
|
Nov. 4
|
Student
Topic Presentation |
Test Control NotationKunal
Shanishchara (Slides) |
17
|
Nov. 9
|
Guest
Speaker |
Testing Object-oriented ProgramsBin Gan (Guest Speaker) |
18
|
Nov. 11
|
Student
Topic Presentation |
Employing User Models for TestingPunam
Agrawal (Slides) |
19
|
Nov. 16
|
Student
Topic Presentation |
Test Case PrioritizationXun Yuan (Slides) |
20
|
Nov. 18
|
Student
Topic Presentation |
Web Testing(Slides) |
21
|
Nov. 23
|
Student
Topic Presentation |
Test-suite Composition and Regression TestingPing Chen (Slides) |
22
|
Nov. 30
|
Student
Topic Presentation |
Penetration TestingGleneesha
Johnson (Slides) |
23
|
Dec. 2
|
Students’
Project Presentations |
Kunal,
Punam, Xun |
24
|
Dec. 7
|
Students’
Project Presentations |
|
25
|
Dec. 9
|
Open
Discussion |
|
Students should choose a project with the consent of the instructor. The student(s) may develop a new testing technique for one of the following software:
· Object-oriented
· Component-based
· Concurrent
· Distributed
· Graphical-user Interface
· Web
·
OR suggest your own project.
A one-page project proposal, (due on Oct. XX), summarizing your project and your new technique will be presented (10 minutes) in class.
The student must experimentally demonstrate that the new technique can be successfully used to test the software. A project report must be submitted at the end of the semester. The report should contain the following sections:
1. Abstract
2. Introduction (motivation, importance of testing the particular type of software, one paragraph on why existing techniques are insufficient, your technique and its key contributions)
3. Detailed description of your idea (several sections, if necessary)
4. Algorithms (pseudo-code) and Experiments (graphs/tables)
5. Related Work (weaknesses of each related approach)
6. Conclusions (weaknesses of your technique and possible future directions)
Remember, suggestions for new projects are welcome.
Students must present a set of related papers (perhaps from the list of suggested papers) on one of the following topics:
o Combining and Comparing Techniques
o Defect and Failure Estimation and Analysis
o Model Checking
o Test Case Prioritization
o Testing Concurrent Programs
o Testing Object oriented Software
o Testing Spreadsheets
o Web Testing
Each presentation will be 75 minutes long, including 10 minutes for questions/discussion. The presentation will be divided into the following parts:
· Problem definition/motivation
· What are the challenges?
· Background literature surveyed by the authors
· Specific technique developed in these papers
· Weaknesses of the techniques, future work
Students will need to develop a power-point presentation, which will be made available on the course web page at least 2 days before the presentation. The final exam will be based on the contents of these presentations.
· 25% Mid-term Exam.
· 25% Final Exam.
· 20% Topic Presentation (40 minutes).
· 5% Project Presentation (10 minutes).
·
25% Term Project (chosen by the student and
approved by the instructor. May be a team (2 students) project, depending on
its scope).
It is expected that all students understand University policies on academic honesty. Cheating on assignments or exams is very serious and will not be tolerated in this class. It is permissible to talk to other students about assignments and to discuss particular solutions. However, you are not to share code with anyone except your partner.
**Classes Start 8/30/04 (Monday)
**Labor Day Holiday 9/6/04 (Monday)
**Thanksgiving Holiday 11/25/04 (Thursday) through 11/28/04 (Sunday)
**Last Class 12/10/04 (Friday)
Study Day 12/11/04 or 12/12/04 (Saturday or Sunday)
Final Exams Start 12/13/04 (Monday)
Final Exams End 12/18/04 (Saturday)
Main Winter Commencement Ceremony 12/18/04 (Saturday Evening)
College Commencement Ceremonies 12/19/04 (Sunday)