Result Aggregation using JUnit- Rule (Testwatcher)

Hi All,

I was asked to create a custom JUnit test result file which only has test case name and pass/failure message.

result.txt should contain like mentioned below.

testmethod1 : success
testmethod2 : failed
testmethod3 : success
testmethod4 : success
etc..

1. Result Aggregation:

The challenge is , we can not log the test results within the test method itself because at this time we will not be able to know whether the test will pass or fail. You may think ,we can write a log statement at the end of the test method but that will log only the success test methods, what about theFailed/Ignored/Errors test methods? As soon as assert failed , AssertionError will be thrown by the Junit and we will not be having any control to log this anywhere.. Still you may think, you can catch this exception and write a log statement and then fail the test methods. Yeah , this would work but it has some drawbacks.

  1. we will not be able to capture the ignored tests.
  2. we will not be able to clearly classify Errors vs Failures (these are completely different)
  3. we need to write a try/catch block in all of our tests. More customization required.

The above functionlity can be achieved easily using TestWatchmen Rule provided by JUnit-Framework.

JUnit-4.9 introduced new features called "Rule" which can be used to keep a log of each passing, ignored, failing tests. Click here for more information

With this approach we need to do no modification to the existing code still we can do our result aggregation.

Step 1: Create a class which implements TestRule Interface.

 
package com.vel.qa.tests.detailedtests;

import org.junit.rules.TestWatcher;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;

import com.vel.qa.helpers.LoggerUtility;

public class ResultAggregator extends TestWatcher {

	private static String resultLog = "";
	private LoggerUtility loggerUtil = null;

	ResultAggregator() {
		// TODO need to refactor this constructor
		loggerUtil = LoggerUtility.getInstance();
	}

	@Override
	public Statement apply(Statement base, Description description) {
		return super.apply(base, description);
	}

	@Override
	protected void succeeded(Description description) {
		resultLog = description.getMethodName() + ": " + "SUCCESS \n";
		loggerUtil.getLogger("result").info(resultLog);
	}

	@Override
	protected void failed(Throwable e, Description description) {
		resultLog = description.getMethodName() + ":  " + "FAILED " + "\n";
		loggerUtil.getLogger("result").info(resultLog);
	}

	@Override
	protected void starting(Description description) {
		super.starting(description);
	}

	@Override
	protected void finished(Description description) {
		super.finished(description);

	}

}

This class will keep a log of each passing and failing test:

Step 2: Create test class and add @Rule annotations.

  
package com.numerify.qa.tests.detailedtests;

import static org.junit.Assert.fail;

import org.junit.Rule;
import org.junit.Test;

public class ResultAggregatorTest {

	@Rule
	public ResultAggregator log = new ResultAggregator();

	@Test
	public void testmethod1() {

	}

	@Test
	public void testmethod2() {
		fail();
	}

}

If you execute the above test class you will get an output as shown below.

testmethod1 : SUCCESS

testmethod1 : FAILED

Note: LoggerUtil is the class which is an custom defined logger for logging different files using log4j.

2. Repeat test with different input files.

We could achieve this by extending Junit-Core Runners [Junit-4X runners , Junit3X runners will be compatible with annotations] and iterate it with our files. I understood that this runner [Parameterized.java] are provided by JUnit itself in the recent releases. Click here to know more about Parameterized-tests

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s