Updates

• Added information about StudentTests in the Testing section, below.
• The project CVS has moved.. You should now check out Project3B rather than Project3, which moves various classes into the cmsc433.p3.helpers package to make test assessment straightforward, and works around a bug in submit server to do with the the JavaSources directory. The writeup below has been reflected to address these changes. In particular, the The Project and the Testing sections.

Introduction

In this project, you will develop a tool to perform automatic refactoring of Java source code. A refactoring is a program transformation that has no effect on how the program behaves. For example, renaming a local variable consistently throughout a method is a kind of refactoring. In your tool, you will use the visitor pattern to traverse an abstract syntax tree (AST) of Java source code and rewrite it. Your tool will work on a subset (detailed later) of the Java programming language.

We will talk in more detail about refactoring in a little while. Everything you need to know about refactoring for this project is described in this document. Do not wait until we get to it in lecture to start the project.

All the code you will need for this project is available in your CVS repository as an Eclipse project. This includes a basic parser for a subset of Java. Ensure that the basic parser working right away; scroll down to "Setting Up the Project" for instructions.

Here is the abstract syntax tree specification for this project.

Background: Refactoring

A refactoring is a program transformation that is semantics-preserving, meaning that the program before and after refactoring has the same behavior. (In practice, refactorings often do modify the behavior of a program very slightly.) Canonical examples of refactorings are renaming local variables or fields consistently, removing an unused parameter from a method, or adding an extra parameter to a method with a default value.

Refactorings are useful in a number of circumstances. They can often be applied to make code clearer and easier to understand. Refactoring is also often a prelude to a more complicated transformation. For example, we might perform a refactoring that adds another parameter to a method with a default value. Once the (behavior-preserving) refactoring is done, then we begin changing the behavior of the program by using the parameter and changing calls to the method so they do not always use the default value. In this case, the idea is that the initial refactoring (adding a method parameter consistently) paves the way for the behavior change.

Each refactoring has one or more inputs that describe how things are going to be refactored. Sometimes refactorings are invalid -- either because they would cause the program to change its behavior, because they would result in code that does not compile, or because of complexity. For example, renaming a local variable so that it has the same name as another local variable is invalid.

In this project, you will implement a series of refactorings, each of which transforms a single Java class. In the description of each case below, we list at least one valid refactoring and some invalid refactoring(s) of the original example. A comprehensive discussion of valid and invalid refactorings can be found in a subsequent section of this project description.

• Remove Unused Parameter: The input is a method method and a parameter index n. In the refactored class, method is rewritten with the nth parameter removed, and all callers of method are rewritten not to pass in what was the nth parameter. The first parameter is numbered 0.
  Example: Before refactoring

  public class Adder {
      int count;
      public void caller() {
          addCount(3, "Adding three");
      }
      public int addCount(int x, String y) {
          count += x;
          return count;
      }
  }
  

  Example: After refactoring method=addCount, n=1

  public class Adder {
      int count;
      public void caller() {
          addCount(3);
      }
      public int addCount(int x) {
          count += x;
          return count;
      }
  }
  

  Example: Invalid refactoring: method=addCount, n=0
  The parameter at position 0 -- namely x -- is in use and therefore can not be removed.

• Replace Magic Number with Symbolic Constant: The input is a magic number magicNum, a type type (which is the Java type of magicNum), and a symbolic constant name name. In the refactored class, all instances of magicNum are replaced by name. Additionally, a new private static final field -- the symbolic constant -- is added to the class.
  Example: Before refactoring

  public class Physics {
      public int coef = 0;
      public double formula(int offset) {
          double answer = 9.81 * (coef + offset);
          return answer;
      }
  }
  

  Example: After refactoring magicNum=9.81, type=double, name=G

  public class Physics {
      public int coef = 0;
      private static final double G = 9.81;
      public double formula(int offset) {
          double answer = G * (coef + offset);
          return answer;
      }
  }
  

  Example: Invalid refactoring: magicNum=9.81, type=double, name=coef
  A field named coef already exists in the class.

• Remove Assignments to Parameters: The input is a method method. In the refactored class, method is rewritten such that any parameter of method that is assigned to is "replaced" with a new local variable. If a parameter is only read from or never used, then it is left alone. The new local variables follow a specific naming scheme (detailed later).
  Example: Before refactoring

  public class Sales {
      public int discount(int quantity, int inputVal, int yearToDate) {
          if (inputVal > 100) inputVal -= 2;
          if (inputVal > 50) inputVal -= 2;
          if (quantity > 4) inputVal -= (inputVal *0.5);
          return inputVal;
      }
      public int specialDiscount(int inputVal) {
          int param0 = 10;
          inputVal -= param0;
          return inputVal;
      }
  }
  

  Example: After refactoring method=discount

  public class Sales {
      public int discount(int quantity, int inputVal, int yearToDate) {
          int param0 = inputVal;
          if (param0 > 100) param0 -= 2;
          if (param0 > 50) param0 -= 2;
          if (quantity > 4) param0 -= (param0 *0.5);
          return param0;
      }
      public int specialDiscount(int inputVal) {
          int param0 = 10;
          inputVal -= param0;
          return inputVal;
      }
  }
  

  Example: Invalid refactoring: method=discount2
  There is no method named discount2.
  Example: Invalid refactoring: method=specialDiscount
  A local variable named param0 already exists in specialDiscount, conflicting with the naming scheme for removing assignments to parameters (see details below).

• Reduce Scope of (Local) Variable: The input is a method method and a local variable name local. In the refactored class, the scope of local is reduced to the smallest scope possible. local must already be in the top-most scope of method, and must have either no initializer or a constant initializer (a ConstantExpression in our AST). The declaration of local is moved to the top of its new scope.
  Example: Before refactoring

  public class Foo {
      public void foo(int a) {
          int i = 7;
          int j,k;
          int l = 8;
          j = 40 - a;
          k = a * 3;
          if (k < j) {
              k += i;
          }
          if (k > j) {
              int b = 5;
              j += 2;
              k -= (l * b);
          }
          if (k > j) {
              k -= l;
          }
      }
  }
  

  Example: After refactoring method=foo, local=i
  (This causes the local variable i's definition to move inside the first if statement)

  public class Foo {
      public void foo(int a) {
          int j,k;
          int l = 8;
          j = 40 - a;
          k = a * 3;
          if (k < j) {
              int i = 7;
              k += i;
          }
          if (k > j) {
              int b = 5;
              j += 2;
              k -= (l * b);
          }
          if (k > j) {
              k -= l;
          }
      }
  }
  

  Example: After refactoring method=foo, local=l
  (This causes the local variable l's definition to move to the first local variable position; we explain why a little later.)

  public class Foo {
      public void foo(int a) {
          int l = 8;
          int i = 7;
          int j,k;
          j = 40 - a;
          k = a * 3;
          if (k < j) {
              k += i;
          }
          if (k > j) {
              int b = 5;
              j += 2;
              k -= (l * b);
          }
          if (k > j) {
              k -= l;
          }
      }
  }
  

  Example: Invalid refactoring: method=bar, local=i
  There is no method named bar.
  Example: Invalid refactoring: method=foo, local=b
  The local variable b is not in the top-most scope of foo.

Background: Java Abstract Syntax Trees and Visitors

You will perform refactorings by using the visitor pattern to traverse an abstract syntax tree of Java source code. In case you have forgotten, an abstract syntax tree is a structured representation of program text that elides unimportant (at least to a compiler) details and is unambiguous. For example, here is the abstract syntax tree corresponding to the example program in the first refactoring:

The classes implementing the AST are stored in the package javaparse; here is the specification. Each node in the abstract syntax tree represents an entity in the source program, but details like semicolons, commas, and parentheses are not explicitly represented. Each node in the figure with a capitalized name represents an object whose class extends Node, where each child of that object in the tree is one of its fields. For example, class SourceFile has a single field theClassDef of type ClassDef; in the figure, the topmost node labeled SourceFile represents an object whose class is SourceFile, and therefore has a single child corresponding to SourceFile's field theClassDef, which has class ClassDef. Each leaf in the tree, in italics, is a String literal. For example, the MethodDef object has a field name whose String value is "addCount". In the figure, we show the contents of a string only, eliding quotation marks (unless they actually appear in the string itself). All fields are public to facilitate the use of the visitor pattern.

The interface NodeVisitor describes a visitor to the AST nodes:

public interface NodeVisitor {

    public void visit(SourceFile n);

    public void visit(Modifier n);
    public void visit(Type n);

    public void visit(ClassDef n);
    public void visit(VariableDef n);
    ...
}

public interface Node {
    /** Standard Visitor Pattern accept method.  Always
        invokes v.visit(this). */
    public void accept(NodeVisitor v);
}

• PrintVisitor traverses an abstract syntax tree and displays the corresponding source code to System.out. This is sometimes called unparsing. Note that our PrintVisitor is not very smart about indentation and such, so it does not produce the most legible output.
• ASTVisitor prints to System.out a textual representation of an abstract syntax tree similar to what is shown above.
• ChangeClassName replaces all occurrences of a given class name with a given value. It is implemented by extending XFormInPlaceVisitor, which is a basic visitor for modifying a syntax tree in place. On its own, XFormInPlaceVisitor does nothing to the syntax tree other than traverse and effectively assign fields of the tree back to themselves; this can be seen by running the DummyVisitor class which extends XFormInPlaceVisitor but does not override any of its visit methods. You may find it useful to base some of your visitors on XFormInPlaceVisitor.

The Project

You must write the four refactorings listed above as NodeVisitors. Each of your refactoring classes should have a exactly one static method makeVisitor that produces a NodeVisitor. We have provided you with a file Main.java that can be used to parse a Java program, apply a refactoring, and output the result. It is invoked as follows:

The first argument is the name of the refactoring visitor class you wish to invoke. arg1 through argn are the arguments to the refactoring, and file.java1 is the Java source file to be refactored. The Main method allows this final file to end with either .java or with extension .java1 instead. All of the public tests, and any that you write for student tests, should end in extension .java1 and not .java. This is because of a limitation in the submit server.

When Main is invoked, using reflection it invokes

RefactoringVisitorName.makeVisitor(arg1, ..., argn)

static public NodeVisitor makeVisitor(String oldName, String newName) {
  return new ChangeClassName(oldName, newName); 
}

helpers.ChangeClassName
ChangeClassNameTestInput
NewName
ChangeClassNameTestInput.java1

Here is a list of things you don't need to worry about, meaning we won't test your refactorings with any input programs containing these features.

• You can assume the input Java source files are well-formed and will compile with a standard Java compiler.
• You can assume that each .java1 file contains exactly one class with the same name as the file.
• Any bugs in Main.java are not your problem.
• Method invocation will never be of the form e.foo(). It will always be of the form foo(). (You'll notice there's no other option in our AST.)
• Field accesses will never be of the form e.field. It will always be of the form field. (Again, there's no other option in our AST.)
• You can assume the sets of local variable names, parameter names, and field names do not overlap in the input program. For example, if there exists a field x, then there won't exist any parameters or local variables named x.
• The last two bullets imply that assignments of the form this.x = x won't appear in the input programs we test with.
• You can ignore overriding (including any use of extends), overloading, and implementing interfaces.
• You can ignore constructors and new.
• You can ignore try/catch/finally/throws.
• You can ignore any place our parser fails.

In some cases, it may be impossible to apply a refactoring with the given parameters. For example, you can't remove assignments to parameters for a method that doesn't exist, and it's not a good idea to add a new method with the same name as an already existing method. In all such cases, your tool should raise an IllegalRefactoringException.

Important: The output of your refactorings will be either a refactored AST or an IllegalRefactoringException. Once you raise IllegalRefactoringException, it doesn't matter what you have or have not done to the AST (i.e., you don't need to write any code to restore it to its original state).

Below we give you details about what bad cases you must catch. We may have forgotten about some potentially bad refactorings. Although your tool only needs to handle the bad cases we describe below, the first person to report a new bad refactoring will get 5 extra credit points per refactoring, up to a total of 15 points. Note that you will only get credit for telling us about refactorings that are bad under our assumptions above; we already know there are many other things to worry about for the full Java source language.

You need to write four refactoring visitor classes:

• RemoveUnusedParam
  • Factory method: static public NodeVisitor makeVisitor(String method, String n);
  • Raise IllegalRefactoringException if any of the following occur:
    • method does not exist
    • n is not an integer or is negative
    • parameter n does not exist
    • parameter n is in use
• ReplaceMagicNumber
  • Factory method: static public NodeVisitor makeVisitor(String magicNum, String type, String name);
  • If type is one of {short, int, long, float, double} and magicNum is numeric, you may assume that magicNum is consistent with type. For example, we won't test your implementation with the following arguments: makeVisitor("3.14","int","PI")
  • If magicNum is not found in the class, do not throw IllegalRefactoringException. Instead, just return the AST with the symbolic constant (field) added to the class.
  • The symbolic constant (field) you add must have the private, static, and final modifiers and no other modifiers.
  • magicNum will always be a nonnegative number. That is, you may assume magicNum will not be preceded by a + or - sign. This is because our AST represents signed constants as a UnaryPreOpExpression with a ConstantExpression child, instead of as just a ConstantExpression.
  • magicNum should be replaced even if it appears in a field initializer
  • Raise IllegalRefactoringException if any of the following occur:
    • name matches the name of an existing local variable, parameter, or field
    • type is not one of {short, int, long, float, double}
    • magicNum is non-numeric
• RemoveParamAssignments
  • Factory method: static public NodeVisitor makeVisitor(String method);
  • New local variables names always begin with param with an increasing integer suffix starting with 0 -- e.g., param0, param1, param2, etc.
  • The integer suffix is increased only after the previous suffix is used. I.e., it's not always the case that the first parameter is param0, the second parameter is param1, and so on. If the first parameter is never assigned to, and hence a new local variable is never created, the name param0 should be used for the second parameter if applicable. If the second parameter is never assigned to, param0 should be use for the third parameter, and so on. The example shown earlier demonstrated this point.
  • A parameter appearing later in method's parameter list may not be represented with a suffix less than the suffix used to represent an earlier parameter. I.e., integer suffixes are set based on the order of parameters in the parameter list and not based on when parameters are assigned to in the body of method. For example, if method is discount and discount has the signature

    public int discount(int quantity, int inputVal, int yearToDate)

    and all three parameters of discount are assigned to, then param0 must be used for quantity, param1 for inputVal, and param2 for yearToDate. Even if, yearToDate is assigned to before quantity and inputVal in the body of discount, param2 would still be used for yearToDate.
  • The type of a new local variable should be the same as the type of the parameter it "replaces".
  • New local variable declarations are to be added to the very top of method's body and in increasing order of suffix. So the declaration for param0 comes first, then the declaration for param1, and so on.
  • A new local variable should be initialized in its declaration to the parameter it replaces.
  • If all parameters are left alone -- i.e, no parameters are ever assigned to -- do not throw IllegalRefactoringException. Instead, just return the AST with no modifications.
  • Note that i++, i--, ++i, and --i are all assignments to i while +i and -i are not assignments to i.
  • Raise IllegalRefactoringException if any of the following occur:
    • method does not exist
    • an existing field, local variable, or parameter is named param0, param1, etc. -- but only if param with that integer suffix actually ends up being used to remove assignments to a parameter
• ReduceScope
  • Factory method: static public NodeVisitor makeVisitor(String method, String local);
  • local's declaration should be moved to the smallest scope and be the first statement in that scope.
  • If local is already in the smallest scope (i.e., local's scope can not be reduced), you should still move local's declaration to be the first statement in that scope. (This sounds weird, but it makes the implementation a bit easier.) In such a case, do not throw IllegalRefactoringException. An example shown earlier demonstrated this point.
  • Note that while

    int i = 7;

    does not count as a use of i, the following does count as a use of i and as such would limit i's scope

    int i;
    i = 7;

  • You may assume that you will not have to move local's declaration into the initialization portion of a for statement. E.g, you will never have to reduce the scope for a local variable -- say i -- such that the following will happen

    public void foo () { int i = 0; int j; j = 45; for (; i < 10; i++) { j += i; } }

    -----> WRONG!

    public void foo () { int j; j = 45; for (int i = 0; i < 10; i++) { j += i; } }

    In this case, the refactored code should actually be the same as the original. Please read the ForStatement description in the abstract syntax tree specification for further details on for statements.
  • You may assume that if local's declaration exists in the top-most scope of method, then local appears at least once after its declaration in method. That is, you will not be asked to reduce the scope of a local variable that is never used.
  • Raise IllegalRefactoringException if any of the following occur:
    • method does not exist
    • local does not exist in method
    • local does not exist in method's top-most scope
    • local's declaration has an initializer and the initializer is not a ConstantExpression (even if the expression would evaluate to a constant. E.g., 8+9 is not a ConstantExpression in the AST, so your refactoring should pretend it is not a constant).
    • local's smallest scope is inside a while loop or for loop

The tool will not accept all Java programs. In particular, the parser only handles Java 1.3, and our AST cannot handle all the features of Java. The tool will reject programs using either extends or implements. You do not need to worry about refactorings that interact with the methods of Object; you can pretend for purposes of this project that Object contains no fields or methods. That is, the input programs we give you won't contain references to fields or methods "normally" inherited from Object (e.g., equals() or toString()) because we will pretend Object is empty and, therefore, nothing is inherited from Object.

What to Submit

• RemoveUnusedParam.java
• ReplaceMagicNumber.java
• RemoveParamAssignments.java
• ReduceScope.java

Testing

  NodeVisitor v = ReplaceMagicNumber.makeVisitor("...", "...");
  SourceFile sf = ....;
  sf.accept(v);

For your student tests, you should have statement coverage of at least 70\% for the four classes you have to implement; i.e., ReduceScope.java, RemoveParamAssignments.java, RemoveUnusedParam.java, and ReplaceMagicNumber.java (along with any other classes you implement in support of these). Your student tests do not need to test PrintVisitor, ASTVisitor, ChangeClassName, etc.; all of these are in the cmsc433.p3.helpers package for this reason.

Hints and Notes

• Your refactorings should modify the AST "in place." That is, you should not build a completely new AST as there would be no way to return that AST to the the caller. In your case, the caller will most likely be Main.java. Specifically, you can not return a different SourceFile node to the caller. You could build new subsets of the AST and replace parts of the AST with those subsets all the way up to and including the ClassDef, but not the SourceFile. Look at how XFormInPlaceVisitor works to see this. Inside your refactoring class, you cannot modify the caller's SourceFile reference.
• Start off looking at the output of the visitor ASTVisitor. This visitor will "pretty print" the AST structure of the input source file in tree fashion.
• Pay close attention to the filenames you will submit. A sure fire way to get a zero for the "Remove Unused Parameter" refactoring is to submit a file named, say, RemoveUnusedParameter.java instead of RemoveUnusedParam.java.
• We stated above: "Once you raise IllegalRefactoringException, it doesn't matter what you have or have not done to the AST." This makes your life a whole lot easier. Understanding this point can save you a lot of headache when designing your refactorings.
• IllegalRefactoringException extends RuntimeException and not Exception. I.e., you don't have to declare that a method throws IllegalRefactoringException -- you can just throw it.
• Come up with your own input that covers many different "good" cases and definitely covers all the bad cases. This will help you understand the details of each refactoring before you begin designing your implementation.
• Java allows parameter and local variable declarations to shadow field declarations. You don't have to worry about this, however, since we stated above: "You can assume the sets of local variable names, parameter names, and field names do not overlap in the input program."
• Java does not allow local variable declarations, even inside nested scopes, to shadow other local variable or parameter declarations. We stated above that only code that will compile with a standard Java compiler will be used to test your refactorings.
• This project is hard.

Bad Refactorings You May Ignore

Here is a list of bad refactorings you may ignore. You don't need to worry about them (i.e., you don't need to write code to catch these cases):

• Setting the name of a new field to be an invalid Java field name, such as a reserved keyword or just an ill-formed identifier.
• Replacing a signed magic number with a symbolic constant. That is, you may assume magicNum will not be preceded by a + or - sign. This is because our AST represents signed constants as a UnaryPreOpExpression with a ConstantExpression child, instead of as just a ConstantExpression.
• Reducing the scope of a local variable such that the local variable's declaration is moved into the initialization portion of a for statement. See the description for ReduceScope for more information.
• Reducing the scope of a local variable that is never used.

This list may be updated later so please check the project description frequently.