Home Contact

CMSC737: Fundamentals of Software Testing (Fall 2011)

Dept. of Computer Science, University of Maryland, College Park, MD, USA.


Date Topic Readings Comments
Sep 1 Introduction to the course Class webpage Sign up for the class forum
Sep 6 The need for testing; testing as an integral part of software engineering; software engineering processes and testing.
  • Testing: a roadmap, Mary Jean Harrold, Proceedings of the conference on the future of Software engineering May 2000.
  • Testing: principles and practice, Stephen R. Schach, ACM Computing Surveys, (CSUR) March 1996, Volume 28 Issue 1.
Sep 8 Understanding and Partitioning Input spaces Open discussion in class. Notes on whiteboard.
Sep 13 Test harnesses. Phase 2 status reports by teams. GUITAR Homepage Also attending: Bryan Robbins and Bao Nguyen
Sep 15 GUITAR 1.0 and its design, architecture, and API GUITAR Homepage Discussion led by: Bryan Robbins and Bao Nguyen
Sep 20 The relationship between sampling and testing – an Introduction A test generation strategy for pair-wise testing, Kuo-Chung Tai; Yu Lei, Software Engineering, IEEE Transactions on, Volume: 28 Issue: 1, Jan. 2002, Page(s): 109 -111.
Sep 22 The GUITAR API and the first step towards implementing it.
Sep 27 The role of experimentation in testing – an Introduction Xun Yuan, Atif M. Memon, "Generating Event Sequence-Based Test Cases Using GUI Runtime State Feedback," IEEE Transactions on Software Engineering, vol. 36, no. 1, pp. 81-95, Jan./Feb. 2010
Sep 29 Designing an experiment to compare test adequacy criteria Scott McMaster and Atif Memon. 2008. Call-Stack Coverage for GUI Test Suite Reduction. IEEE Trans. Softw. Eng. 34, 1 (January 2008), 99-115.
Oct 4 On experimentation and evaluation of testing techniques. And issues of abstration in model-based testing.
Oct 6 Progress in implementing the GUITAR API.
Oct 11 Testing: Summarizing general concepts Slides Slides
Oct 13 Fundamental concepts: bottom-up, top-down, mocks, fakes, stubs, drivers. Testing for performance, robustness, usability, correctness
Oct 18 Regression Testing An empirical study of regression test selection techniques, Todd L. Graves, Mary Jean Harrold, Jung-Min Kim, Adam Porter, Gregg Rothermel, ACM Transactions on Software Engineering and Methodology (TOSEM) April 2001, Volume 10 Issue 2. Slides
Oct 20 Predicate-based test generation Predicate-based test generation for computer programs, Kuo-Chung Tai, Proceedings of the 15th International Conference on Software Engineering, 1993. Page(s): 267-276.
Oct 25 Build testing. Testing component hierarchies. Multi-configuration systems. Shared component-based systems. Continuous testing. “Effective and Scalable Software Compatibility Testing” by Il-Chul Yoon, Alan Sussman. Atif M. Memon, and Adam Porter, in ISSTA '08: Proceedings of the International Symposium on Software Testing and Analysis
Oct 27 Testing Rich Internet Applications. Testing Mobile applications.
  • Domenico Amalfitano, Anna Rita Fasolino, Porfirio Tramontana: Reverse Engineering Finite State Machines from Rich Internet Applications. WCRE 2008: 69-73.
  • Domenico Amalfitano, Anna Rita Fasolino, Porfirio Tramontana: Experimenting a reverse engineering technique for modelling the behaviour of rich internet applications. ICSM 2009: 571-574.
  • Domenico Amalfitano, Anna Rita Fasolino, Porfirio Tramontana, "A GUI Crawling-Based Technique for Android Mobile Application Testing," icstw, pp.252-261, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, 2011.
Nov 1 Test Adequacy and Coverage Criteria
  • Software unit test coverage and adequacy Hong Zhu, Patrick A. V. Hall, John H. R. May ACM Computing Surveys (CSUR) December 1997 Volume 29 Issue 4.
  • Coverage criteria for GUI testing Atif M. Memon, Mary Lou Soffa, Martha E. Pollack ACM SIGSOFT Software Engineering Notes, Proceedings of the 8th European software engineering conference held jointly with 9th ACM SIGSOFT symposium on Foundations of software engineering September 2001 Volume 26 Issue 5.
  • Xun Yuan, Myra B. Cohen, Atif M. Memon, "GUI Interaction Testing: Incorporating Event Context," IEEE Transactions on Software Engineering, vol. 37, no. 4, pp. 559-574, July/Aug. 2011.
  • The evaluation of program-based software test data adequacy criteria E. J. Weyuker Communications of the ACM June 1988 Volume 31 Issue 6.
Nov 3 MIDTERM EXAM Everything discussed in class including lecture on Nov. 1. Open notes/book. No computers.
Nov 8 Issues in Regression testing.
  • Jung-Min Kim, Adam Porter, and Gregg Rothermel. 2000. An empirical study of regression test application frequency. In Proceedings of the 22nd international conference on Software engineering (ICSE '00). ACM, New York, NY, USA, 126-135.
  • Atif M. Memon. 2008. Automatically repairing event sequence-based GUI test suites for regression testing. ACM Trans. Softw. Eng. Methodol. 18, 2, Article 4 (November 2008), 36 pages.
Nov 10 Test prioritization Gregg Rothermel, Roland J. Untch, and Chengyun Chu. 2001. Prioritizing Test Cases For Regression Testing. IEEE Trans. Softw. Eng. 27, 10 (October 2001), 929-948.
Nov 15 On Fault seeding and test coverage for project
Nov 17 Software Metrics for Quality Assurance Calculation and optimization of thresholds for sets of software metrics, by Steffen Herbold, Jens Grabowski and Stephan Waack. Empirical Software Engineering, Volume 16, Number 6, 812-841.
Nov 22 Directed random testing
  • Eclat: Automatic generation and classification of test inputs, Carlos Pacheco and Michael D. Ernst, in ECCOP '05.
  • Feedback-directed random test generation, Carlos Pacheco, et. al., in ICSE '07.
Forum Discussion
Nov 29 Web service testing
  • Automated Testing and Response Analysis of Web Services ICWS07 Evan Martin, Suranjana Basu, Tao Xie.
  • Generating test cases for web services using data perturbation ACM SIGSOFT Software Engineering, Volume 29 Issue 5, September 2004 Jeff Offutt, Wuzhi Xu.
Forum Discussion
Dec 1 Security testing
  • Patrice Godefroid, Michael Y. Levin, David A Molnar. "Automated Whitebox Fuzz Testing". Network Distributed Security Symposium (NDSS), Internet Society, 2008.
  • Vijay Ganesh , Tim Leek , Martin Rinard, Taint-based directed whitebox fuzzing, Proceedings of the 2009 IEEE 31st International Conference on Software Engineering, p.474-484, May 16-24, 2009.
Forum Discussion
Dec 6 Web testing
  • Filippo Ricca and Paolo Tonella. 2001. Analysis and testing of Web applications. In Proceedings of the 23rd International Conference on Software Engineering (ICSE '01). IEEE Computer Society, Washington, DC, USA, 25-34.
  • Giuseppe A. Di Lucca and Anna Rita Fasolino. 2006. Testing Web-based applications: The state of the art and future trends. Inf. Softw. Technol. 48, 12 (December 2006), 1172-1186. DOI=10.1016/j.infsof.2006.06.006 http://dx.doi.org/10.1016/j.infsof.2006.06.006.
  • Ali Mesbah, Engin Bozdag, and Arie van Deursen. 2008. Crawling AJAX by Inferring User Interface State Changes. In Proceedings of the 2008 Eighth International Conference on Web Engineering (ICWE '08). IEEE Computer Society, Washington, DC, USA, 122-134. DOI=10.1109/ICWE.2008.24 http://dx.doi.org/10.1109/ICWE.2008.24.
Forum Discussion
Dec 8 Database testing
  • Hardik Bati, Leo Giakoumakis, Steve Herbert, and Aleksandras Surna. 2007. A genetic approach for random testing of database systems. In Proceedings of the 33rd international conference on Very large data bases (VLDB '07). VLDB Endowment 1243-1251.
  • Yuetang Deng, Phyllis Frankl, and David Chays. 2005. Testing database transactions with AGENDA. In Proceedings of the 27th international conference on Software engineering (ICSE '05). ACM, New York, NY, USA, 78-87. DOI=10.1145/1062455.1062486 http://doi.acm.org/10.1145/1062455.1062486.
Forum Discussion
Dec 13