© 2001 by Charles Lin. All rights reserved. You must receive explicit written permission to copy information on this webpage.

Background

For example, most beginning programmers believe "I must spend all my time coding, because time spent planning and testing code is a waste of time". Yet, in many software companies, you may spend over half your time planning how to write code. This involves sitting down and determing what to write, how to write, developing classes, determining who codes what, having a regimented way to test the code, and make changes.

At this point, you aren't ready to work in teams, so some of that kind of testing is postponed to a course in software engineering.

Nevertheless, it's a good idea to test your code. The question is: how?

Primary Outputs: Are They a Crutch?

In other words, students are so fearful of writing their own test files. They literally do not trust themselves to write their own test files successfully, and want to see an "official version". However, you must eventually learn to decide what is a "correct" test file for yourself. You may not have the luxury of always having some sample output which you can depend on.

The other problem with sample outputs is that some people will not even know why a sample output is produced. They simply try to mimic it the best they can, and when the output isn't correct, they complain to the TA that the output isn't correct.

Such students, were they given the sample input, and told that, as a quiz, they must produce the sample output, would be unable to do so. Fortunately, most of those students did not pass CMSC 114 and CMSC 106. As students of CMSC 214, you are expected to be able to know how to write your own test files.

When and What to Test

However, for most people, that's simply going to lead to too many bugs and rewriting of code. It's better to test on a smaller scale. Typically, you want to test one or two methods at a time. To do so, you will create a test driver program. The goal is to test each method.

The question is which methods to test first. Here's how I would do it:

1. Write the default constructor and print() method first (which can be used to implement the output operator).

   You want to start simple, and the simplest place to start is the default constructor. You also need a way of determing whether the object is "correct" and one way to do this is to be able to print it out.

   Here's how I suggest you start out:

   #include 
   #include "Fract.h"
   
   using namespace std;
   
   int main()
   {
      Fract f1;  // default fraction
   
      cout << "Testing default constructor" << endl;
      cout << "---------------------------" << endl;
      cout << f1 << endl;
      cout << "[ANSWER] 0 / 0" << endl << endl;
   }
   

   It's easy, it's simple. But there are important features. First, there are some output lines to indicate what kind of test is being run. In this case, it's the default constructor being run.

   Then, the object is printed. Then, the "expected answer" is printed. It's useful to print the expected answer, so you can anticipate what you think the output is. It's often a good idea to know what you think the output is and see if it is indeed the output.

2. You may wish to test other constructors (except the copy constructor, which is tested later)

   #include 
   #include "Fract.h"
   
   using namespace std;
   
   int main()
   {
      Fract f1( 2, 6 );  // fraction 2/6 
   
      cout << "Testing next constructor" << endl;
      cout << "------------------------" << endl;
      cout << f1 << endl;
      cout << "[ANSWER] 1 / 3" << endl << endl;
   }
   

3. If you've written the copy constructor and assignment operator, you may want to test that next.

   If you haven't written it because you intend to use the implicit version created by the compiler, then testing the copy constructor and assignment operator may be silly. Still, it doesn't hurt. When you test these methods, create an "original" object, make a copy (one copy using the copy construtor and another using assignment operator), modify the original, and see if the copy is modified. If it's a deep copy, then the copy should be the copy of the original PRIOR to the original being modified. Here's a sample test driver.

   #include 
   #include "Fract.h"
   
   using namespace std;
   
   int main()
   {
      Fract orig( 2, 5 ); 
      Fract copyOfOrig = orig; // copy constructor 
      Fract secondCopy;
   
      secondCopy = orig; // assignment operator 
   
      orig.add( 1, 5 ); // modify orig 
   
      cout << "Testing copy constructor" << endl;
      cout << "------------------------" << endl;
      cout << orig << endl;
      cout << "[ANSWER] 3 / 5" << endl;
      cout << copyOfOrig << endl;
      cout << "[ANSWER] 2 / 5" << endl << endl;
   
      cout << "Testing operator=" << endl;
      cout << "-----------------" << endl;
      cout << orig << endl;
      cout << "[ANSWER] 3 / 5" << endl;
      cout << secondCopy << endl;
      cout << "[ANSWER] 2 / 5" << endl << endl;
   }
   

   As you may notice, it may not be a good idea to test the copy constructor and assignment operator right away. The reason? You need methods to modify the original. Without it, you may not be able to test the full extent of the copy constructor/assignment operator.

   This is especially true if you are testing linked lists. You need a way to create something besides an empty linked list to properly test the copy constructor/assignment operator.

4. Test the other methods.

   There should be several other methods, and you should write tests for them.

One Large Test Driver

Drawbacks of Printing

The advantage of printing is that it's easy to write the code. However, printing output has drawbacks. It's hard to automate. You, the programmer, must sit down and determine whether your code passed the test or not by visually inspecting line after line of output.

Wouldn't it be nice if you could somehow automate the steps so that it only prints when something has failed? That would be convenient.

Alas, it takes more time to write automated tests, than simply printing everything. But that time may be well worth it (if you have lots of time). You can often run test after test, with very little effort. You don't have to recompile code.

To write such tests, you have to be reasonably clever to come up with tests to fit whatever class you are testing. Let's look at an example. Suppose you wish to test whether a sorted linked list is sorting correctly or not. You can create an input file that looks like:

add 3
add 10
add 8
add 5
answer 4 3 5 8 10

For example, here's a way to to write a test driver that can handle the above input file.

#include 
#include "List.h"

using namespace std;

int main()
{
   List list;
   string cmmd;

   while ( cin >> cmmd )
    {
      if ( cmmd == "add" )
        {
            int num;
            cin >> num;
            list.add( num );
        }
      else if ( cmmd == "answer" )
        {
            Iterator iter = list.iterator();
            int size;
            cin >> size;
            if ( size != list.size() )
               cout << "Error: sizes do not match" << endl;

            int *arr = new int[ size ];
            int index = 0;
            for ( iter.goFirst(); iter.inList(); iter.goNext() )
               {
                   if ( iter.getCurrent() != arr[ index ] )
                     {
                         cout << "Error at index " << index << endl;
                         cout << "Expected " << arr[ index ] 
                              << " but see " << iter.getCurrent() << endl;
                     }
                   index++;
               }
        }
      else
         cout << "Invalid command: " << cmmd << endl;
    }

}

Admittedly, the input file requires that you write the solution down, but once you do, it's easy to create many variations of the input, without having to recompile the code.

Writing Debugging Methods

This method might have a prototype that looks like:

    bool verify( int arr[], int size );

While this method is not useful to the "user" of the class, it is useful to you, the developer of the class.

Figuring out Test Cases

The key is to make simple tests first, which test all possible situations, then work your way up to more complicated tests. Some programmers pick very difficult cases first (often relying on the sample input or output provided), but refuse to try simple cases (complaining it takes too much time to write their own test cases). Start with simple input cases, debug that first, then work your way to more complicated cases. You'll discover that fixing problems in small test cases often means avoiding problems in large test cases.

Code Coverage

if-else statements

if ( <cond 1> )
  {
    cout << "In body 1" << endl;
    // body 1 
  } 
else if ( <cond 2> )
  {
    cout << "In body 2" << endl;
    // body 2 
  } 
else if ( <cond 3> )
  {
    cout << "In body 3" << endl;
    // body 3 
  } 

Loops

while( <cond> )
  {
    cout << "In while body" << endl;
    // body  
  } 

Combining Loops and if-else

There are some software tools that check for test coverage. Basically, you use the tool to "instrument" your executable (which modifies the executable with special testing code). Then, you run a series of test files. The tool checks to see which lines of code are run and which are not run.

If a certain line of code is not run, then there are two possibilities. First, you did not create a test case that would make the code run (and thus, that piece of code was untested), or second, it's impossible to get to that piece of code (because of various conditions that are impossible to satisfy), in which case, the code is not important and shouldn't be there.

Unfortunately, it's hard to tell which of the two cases has occurred. All you know is that some code wasn't run, so it wasn't tested. Based on that information, you need to figure out if it's possible to write a test case to make the code that didn't run, run.

We won't be using such tools in this course, but it's useful to know that they exist.

Web Accessibility