Archive for June, 2009

MultiThreaded TestNG, Surefire Craziness.

June 30th, 2009 No comments

The Setup

Maven 2.0.9
Surefire 2.4.3
TestNG version 5.8

The Craziness

If you annotate a method to have an invocationCount of 2, and a threadPoolSize of 2, and your test fails in an @BeforeMethod or an @AfterMethod when run by surefire, the test run count reported by surefire will be inaccurate. Significantly innacurate at times.

For example:

package com.infusion.crm.application;

import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.AssertJUnit;
import org.testng.annotations.Test;

* Created by IntelliJ IDEA.
* User: joey
* Date: Jun 29, 2009
* Time: 3:19:13 PM
* To change this template use File | Settings | File Templates.
public class TestThreadPoolSize {
public void before(){

public void test1(){

Returns the following from maven test:

Running TestSuite
Tests run: 8, Failures: 1, Errors: 0, Skipped: 7, Time elapsed: 10.298 sec <<< FAILURE! Results : Failed tests: before(com.infusion.crm.application.TestThreadPoolSize) Tests run: 8, Failures: 1, Errors: 0, Skipped: 7

This problem is also manifested if you create an AnnotationTransformer that forces the invocationCount and threadPoolSize of a test to both be greater then 1.

This bug is only manifested when multiple threads are used. If you set the invocation count to 20 and the thread pool size to 1, the bug does not appear.

Sometimes it is unpredictable and worse then just 2x the right number of tests. I ran a test, with 9 methods, and 4 threads, and a very exhaustive @BeforeMethod that takes several seconds to run. It failed repeatedly and said: 142 test run, 3 failed, 139 skipped.

Categories: java, maven, testng Tags:

Converting jUnit test to TestNG tests.

June 30th, 2009 No comments


After writing the below script, and submitting my change for a code review, the reviewer pointed me to a TestNG provided tool that will convert them. So here it is:

java org.testng.JUnitConverter -overwrite -annotation -srcdir src

Be sure to include the testng jar, along with the $JAVA_HOME/lib/tools.jar in your classpath when you run that.

Original SED way of converting

Because of the exceptional design of TestNG, this is a pretty simple text replace exercise. Here is a shell script using sed that will do most of the work for you. After it runs, just remove jUnit from your dependency list / classpath and re-compile, hand edit any errors.

There are two steps to running this, first, create a list of all the java files that contain junit references in your src.

find . -name "*.java" | xargs grep -L junit > toconvert

Then, use xargs to run the script on all the files.

cat toconvert | xargs testngify

Here is the contents of the testngify shell script:


function replace {
#echo executing pattern: $1
sed "$1" < $IN_FILE > $OUT_FILE

replace "s/import org\.junit\.After;/import org\.testng\.annotations\.AfterMethod;/g"
replace "s/import org\.junit\.Assert;/import org\.testng\.AssertJUnit;/g"
replace "s/import org\.junit\.Before;/import org\.testng\.annotations\.BeforeMethod;/g"
replace "s/import org\.junit\.Test;/import org\.testng\.annotations\.Test;/g"
replace "s/import junit\.framework\.Assert;/import org\.testng\.AssertJUnit;/g"
replace "s/import\s*static\s*org\.junit\.Assert\.\*;/import static org\.testng\.AssertJUnit\.\*;/g"
replace "s/import\s*static\s*org\.junit\.Assert\.assertEquals;/import static org\.testng\.AssertJUnit\.assertEquals;/g"
replace "s/\(\s\)Assert\./\1AssertJUnit\./g"
replace "s/import org\.junit\.Test;/import org\.testng\.annotations\.Test;/g"
replace "s/\(@Test\s*\)(expected\s*=/\1(expectedExceptions =/g"
replace "s/\(@Test\.*\)ComparisonFailure\.class/\1AssertionError\.class/g"
replace "s/@After\s*$/@AfterMethod/g"
replace "s/@Before\s*$/@BeforeMethod/g"
replace "s/@Before()\s*$/@BeforeMethod/g"
replace "s/org\.junit\.Assert\./org.testng.AssertJUnit\./g"
replace "s/.*@Ignore(\"\(.*\)\")/\t@Test(enabled=false) \/\/\1/g"
replace "s/\(.*\)@Ignore\(.*\)/\1\2/g"
replace "s/^.*org\.junit.*$//g"

Categories: java Tags: , , , ,