A custom JUnit Runner

Sometimes, JUnit does not exactly fit the bill. In a lot of cases, I can work around it by extending JUnit using the rules feature or creatively implementing a parameterized unit test. However, in the last months I had several occasions, where the result of this approach was clumsy or the problem not solvable at all. This resulted in some things not being as well tested as I would have liked them to be. So I set out to implement custom JUnit runners. And it turned out to be simple!

Extending or customizing JUnit with a custom(ized) runner is relatively straight forward. JUnit will choose a custom runner whenever your test is annotated with the @RunWith annotation. The argument of the annotation specifies the desired runner. This runner can be any class extending Runner and providing a constructor that takes Class<?> descriptor  of the annotated test as an argument.
Consider that you whish to execute your tests for a specific set of different locales. Your system allows manipulating the locale to be used via a global class LocaleHolder:

class LocaleHolder {
    private static Locale locale;

    public static Locale set(Locale locale) {
        Locale old = LocaleHolder.locale;
        LocaleHolder.locale = locale;
        return old;
    }
    public static Locale get() {
        return locale;
    }
}

Now you want to implement your tests just once and have them executed for each possible Locale:

import com.example.junit.Locales;
@RunWith(Locales.class)
public class LocalesTest{
    @Test
    public void xyzIsCool(){
        System.out.println("Locale in use: " + LocaleHolder.get());
    }
}

Please note, that in the real word, you can achieve the same by simply using a parameterized JUnit test, but the task has a perfect scope for an example.
In the first step, we have to implement the custom “Locales” runner. As this runner will execute the same tests multiple times and therefore will be a node in the test runners gui, it should inherit from ParentRunner. Suite, which extends ParentRunner, is even better suited, as it implements a complete runner that can be used out of the box:

public class Locales extends Suite {
    private static final Iterable<Locale> localesToUse =
            asList(Locale.FRENCH, Locale.GERMAN, Locale.ENGLISH);
   
     public Locales(Class<?> klass) throws InitializationError {
        super(klass, extracAndCreateRunners(klass));
    }

    private static List<Runner>
            extracAndCreateRunners(Class<?> klass) throws InitializationError {
        List<Runner> runners = new ArrayList<Runner>();
        for(Locale locale : localesToUse){
            runners.add(new LocalesRunner(locale, klass));
        }
        return runners;
    }
}

Basically, the LocalesRunner itself should set the correct locale before each test and execute the test in the same fashion as JUnit would do. The simplest way to get new functionality of the ground, is to derive from BlockJUnit4ClassRunner, which is the default test runner in JUnit . The best way to extend or augment standard JUnit behavior programmatically, is to inject your functionality in the Statement chain that the base class runner builds. For more information about extending statements, refer to the various JUnit rules examples, as they use this mechanism, too.

class LocalesRunner extends BlockJUnit4ClassRunner { 
    private final Locale locale;
    LocalesRunner(Locale locale, Class<?> klass) throws InitializationError { 
        super(klass); 
        this.locale = locale; 
    }
    @Override 
    protected Statement methodBlock(final FrameworkMethod method) { 
        return new Statement() { 
            @Override 
            public void evaluate() throws Throwable { 
                Locale oldLocale = LocaleHolder.set(locale); 
                try { 
                    LocalesRunner.super.methodBlock(method).evaluate(); 
                } finally { 
                    LocaleHolder.set(oldLocale); 
                } 
            }         }; 
    } 
    @Override// The name of the test class 
    protected String getName() { 
        return String.format("%s [%s]", super.getName(), locale); 
    }
    @Override// The name of the test method 
    protected String testName(final FrameworkMethod method) { 
        return String.format("%s [%s]", method.getName(), locale); 
    } 
}

Now we are almost done (heard that one before?). The example should compile and the Test at the beginning of this post should get executed for each locale in the list. We also extended the test name output, so that one can see what test was executed for which locale.

A last interesting challenge was to think about how to write unit test for a custom test runner. In a lot of cases, it was sufficient to provide a simple test case like in this example. In other cases I needed to develop the runners incrementally using TDD, so I wanted smaller, specific unit tests. It turned out, that it is pretty easy to run JUnit for my test runner test cases programmatically:

private Result runTest(Class<?> testClass) {
    JUnitCore core = new JUnitCore();
    return core.run(testClass);
}

Writing such custom runners is obviously not difficult and can be done for specific problems in any project. It is done fast an helps you to provide more convenient ways to write test according to your projects needs.