Code Coverage Analysis with tcov

It is a good practice to do code coverage analysis in the development environment, to find the cold and hot spots of the application. Cold spots are those parts of code that are rarely used (or executed); and the Hot spots are those parts of the code, that are very frequently used. Doing this analysis in the development stage, helps: (i) improving the performance of the application by tuning the hot spots, and (ii) removing the dead code and/or rewriting or moving the cold spots, in such a way that the code will be executed for decent number of times in the life time of the program. The tricky part is creating extensive test cases to cover all the branches of the source code. Since we may not be able to generate test cases for all scenarios that can exercise all parts of the code, this analysis also helps detecting new test cases that we could invent to cover certain features. For example, code coverage analysis on regular builds certainly helps us discover the blocks of code that was never executed; so it gives us an opportunity to come up with better test cases to run against profile feedback collect builds during training run.

Sun Microsystems bundled a tool called tcov (short for: test coverage), with Sun Studio compiler suite, for code coverage analysis. tcov gives line-by-line information on how a program executes. Most of this information is annotated in a copy of the source file {with .toc extension}, when the {tcov} tool is run against the source file, along with profile data from the execution of test cases.

Steps to do the Code Coverage Analysis with tcov

1. Compile the program with additional option -xprofile=tcov
   
   
2. Run the program and all the test cases against the application. Since it was compiled with -xprofile option, {by default} the run-time will create a directory called <executable>.profile in the same directory from where the executable is run. This behavior can be overridden by setting SUN_PROFDATA_DIR or SUN_PROFDATA environment variables.
   
   <executable>.profile directory contains a file called tcovd. tcovd holds the information about the line numbers, and the execution count. This is a plain text file.
   
   
3. Run tcov with -x option, over each source file to generate the annotated source file

Here's an example. Example was taken from the article on profile feedback optimization.

eg.,

% cat tcovex.c
#include <stdio.h>
#include <stdlib.h>

static unsigned _sum (unsigned *a0, unsigned *a1, unsigned *a2) {

        unsigned result = 0;

        if (a0 == NULL) {
                printf("a0 == NULL");
        } else {
                result += (*a0);
        }

        if (a1 == NULL) {
                printf("a1 == NULL");
        } else {
                result += (*a1);
        }

        if (a2 == NULL) {
                printf("a2 == NULL");
        } else {
                result += (*a2);
        }

        return (result);
}

int main(int argc, const char *argv[]) {
        int i, j, niters = 1, n=3;
        unsigned sum, answer = 0, a[3];

        niters = 1000000000;

        if (argc == 2) {
                niters = atoi(argv[1]);
        }

        for (j = 0; j < n; j++) {
                a[j] = rand();
                answer += a[j];
        }

        for (i = 0; i < niters; i++) {
                sum = _sum (a+0, a+1, a+2);
        }

        if (sum == answer) {
                printf("answer = %u\n", answer);
        } else {
                printf("error sum=%u, answer=%u", sum, answer);
        }

        return (0);
}

% cc -xO2 -xprofile=tcov -o tcovex tcovex.c

% setenv SUN_PROFDATA_DIR /tmp

% ls -ld /tmp/*.profile
No match

% ./tcovex 10000000
answer = 32709

% ls -ld /tmp/*.profile
drwxrwxrwx   2 build    engr         179 Sep 22 19:25 /tmp/tcovex.profile/

% ls -lR /tmp/tcovex.profile/
/tmp/tcovex.profile/:
total 16
-rw-rw-rw-   1 build    engr         318 Sep 22 19:25 tcovd

% tcov -x $SUN_PROFDATA_DIR/tcovex.profile tcovex.c

% ls -l $SUN_PROFDATA_DIR/*.tcov
-rw-rw-rw-   1 build    engr        1857 Sep 22 19:27 /tmp/tcovex.c.tcov

% cat /tmp/tcovex.c.tcov
           #include 
           #include 

           static unsigned _sum (unsigned *a0, unsigned *a1, unsigned *a2) {

10000000 ->     unsigned result = 0;

                if (a0 == NULL) {
     ##### ->           printf("a0 == NULL");
                } else {
10000000 ->             result += (*a0);
                }

10000000 ->     if (a1 == NULL) {
     ##### ->           printf("a1 == NULL");
                } else {
10000000 ->             result += (*a1);
                }

10000000 ->     if (a2 == NULL) {
     ##### ->           printf("a2 == NULL");
                } else {
10000000 ->             result += (*a2);
                }

                return (result);
           }

           int main(int argc, const char *argv[]) {
      1 ->      int i, j, niters = 1, n=3;
                unsigned sum, answer = 0, a[3];

                niters = 1000000000;

                if (argc == 2) {
      1 ->              niters = atoi(argv[1]);
                }

      1 ->      for (j = 0; j < n; j++) {
      3 ->              a[j] = rand();
                        answer += a[j];
                }

      1 ->      for (i = 0; i < niters; i++) {
10000000 ->             sum = _sum (a+0, a+1, a+2);
                }

      1 ->      if (sum == answer) {
      1 ->              printf("answer = %u\n", answer);
                } else {
     ##### ->           printf("error sum=%u, answer=%u", sum, answer);
                }

                return (0);
           }


                 Top 10 Blocks

                 Line      Count

                    6   10000000
                   11   10000000
                   14   10000000
                   17   10000000
                   20   10000000
                   23   10000000
                   45   10000000
                   40          3
                   30          1
                   36          1


           18   Basic blocks in this file
           14   Basic blocks executed
        77.78   Percent of the file executed

     70000009   Total basic block executions
     3888889.25 Average executions per basic block

Note:
Lines with prefix "#####" were never executed.

Reference and suggested reading:
Sun Studio 10 man page of tcov
__________________
Technorati tags: sun studio
C
C++