Tuning program performance is an important step in the software development cycle. A program that profiles program performance is often used to determine which portions of the code the program spends most of its time in. Although the problem of instrumenting code in order to measure performance information has been extensively studied, very few tools exist for visualizing profiling data.
I will briefly discuss two profiling tools that include visualization support. Microsoft Visual C++, a widely used C++ development environment for Windows, includes a profiling tool in some versions. The tool generates spreadsheet-like reports that may include the execution time of each method called in the program as well as other metrics. A report can be saved as a text file and imported into Microsoft Excel. Excel macros provided with the profiling tool can be used to generate simple graphs of the execution time of each method and the number of times each method is called (hits).
Visual Quantify is a performance profiling tool produced by Rational Software. Like the Visual C++ profiler, it generates spreadsheet-like reports on program execution (although more sophisticated). Below is an example report generated by Visual Quantify:
Visual Quantify visualizes the report in the form of a graph -- nodes represent functions and edges specify calling relationships between functions (e.g., function A calls function B). The thickness of the edges between methods encode the weight of the edge (how much it contributes to the programs overall execution time). For many programs, a path with thick edges stands out as the "critical path" in the program. It is along this path that optimizations may significantly improve performance.
In Comparing the profiling visualization tools available in Visual C++ and Visual Quantify, I will first show the results for a trivial program (yes, Hello World) in order to point out the essential features of the two tools without the clutter of a realistic visualization. I will then show the results for a real application.
Hello World!
The Code for the sample program, which consists of a single method, sayHello and a main, is provided below:
#include <stdio.h>
void sayHello(int numberOfTimes)
{
for ( int i = 0; i < numberOfTimes; i++ )
printf("Hello World!\n");
}
int main(int argc, char *argv[])
{
sayHello(1000);
return 0;
}
The Microsoft Excel Graph produced from the Visual C++ report is show below. It clearly shows that the function sayHello is responsible for almost all of the program's execution time.
The graph Produced by Visual Quantity clearly shows the critical path traversed by the application through sayHello, although in somewhat more detail.
Quantify is much more effective in visually presenting the profiling data. The critical path is clearly visible in green. Quantify automatically adjusts the amount of data shown (in this case, fewer than 10 percent of the methods) to reasonably fit on the screen, based on importance. The program contains six threads of execution, which Quantify also clearly shows.
A front-end for the GNU profiler gprof produces call-graphs similar to those in Quantify, although it has fewer features.
There has also been a large amount of work on visualization of parallel system performance. However, profiling parallel systems is generally much more difficult and the issues are very different. A nice overview of parallel performance tools can be found here.
Although Quantify provides visualizations that are clearly superior to a text-based report, Quantify still cannot reasonable show all functions and there relations at once.