JCover Code Coverage: important to understand!

Date Added: May 02, 2010 05:11:11 PM

Author:

What are some of the coverages typically found useful?

There are several types of coverages discussed in the research literature. Among these, the following are popular: Statement coverage, Condition coverage, Decision coverage, Condition /Decision coverage, Multiple condition coverage, Loop coverage. Even among these, Statement and Branch coverage are the most common. Some other coverages are Call-pair coverage, Path coverage and Data-flow coverage.

Is there a difference between "Statement Coverage" and "Line Coverage"?

It is generally believed that statement coverage is the same as line coverage. This might be true for programming languages such as Fortran and COBOL, but is debatable in the context of languages such as Java. In Java, we could define multiple statements on the same line, or split a statement across multiple lines. If a coverage analysis tool gathers line coverage for Java, how can it handle the following case correctly?

class TestClass {

public static void main(String[] args) {

f(); g();

}

void f() { /* body */ }

void g() { /* body */ }

}

The main method contains two statements - one call to f() and another to g(). What happens if call to f() succeeds, but call to g() fails at runtime? Will the line-based coverage tool report this correctly?

JCover uses the definition of statement as given in the Java Language Specification, so when it says for example, 100 statements were covered, there is no ambiguity.

Is "Branch coverage" important?

Most practitioners strongly recommend that branch coverage be taken into account in addition to statement coverage. The reason is easy to understand. Consider the following code:

class TestClass2 {

public static void main(String[] args) {

if(args.length > 0) {

System.out.println("Args available");

}

System.out.println("Completed");

}

}

If a tool only supports statement coverage, it cannot determine whether the code was tested for the case where command line argument was not supplied (i.e., args.length == 0). This is the classic null else problem.

In that case, why should we restrict only to Statement and Branch coverage?

When we perform testing, we usually set some coverage objectives, and declare testing to be adequate if the objectives are met satisfactorily. What coverage measures we use depends on various factors such as their acceptance in the industry and tool support. If you believe that your testing strategies call for the more exotic measures such as data-flow and path, you should go ahead and use them. After all, testing is a serious activity, and anything that improves the quality of the software is quite welcome.

Will there be a performance hit when I run the coverage-enabled application?

When an application is run with coverage analysis enabled, extra code gets executed at runtime to gather the critical parameters required for accurate coverage reporting. Depending on the coverage implementation technology as well as the nature of the application, performance deterioration is to be expected.

Does 100% coverage indicate that my application is bug free?

Certainly not. Coverage measure is just an indicator of how well various parts of your program were exercised during testing. If some parts of the code were not covered during testing, it might indicate a poor testing strategy, among other things. For example, the following simple program easily achieves 100% statement coverage, but is buggy if the program specification says it must print the larger of two positive integers specified as commandline arguments.

class Buggy {

public static void main(String args[]) {

System.out.println(args[0] + args[1]);

}

}

So a test team cannot release a piece of software to the client just because 100% statement coverage was achieved.

Should I always try to achieve 100% coverage?

Getting 100% coverage is a lofty goal, but rarely achieved in practice. Remember also that when we talk about a coverage percentage, it is with respect to some program element such as statement, branch, etc. Achieving 100% statement coverage may be easier than achieving 100% branch coverage. Achieving 100% path coverage is impossible except in trivial cases. If 100% coverage of some element is not achieved during testing, it may not be a panicky situation in itself, but may warrant a closer scrutiny of test cases and the code to rule out dangerous pathology.

User:		Password:
Keep me logged in.
Register \| I forgot my password