The debate whether to specify configuration information in the form of annotations on Java source code or in external configuration files, is still ongoing. The current consensus seems to be that configuration that impacts the source code—such as marking a method as @Test—should use annotations, whereas an external file is more suitable for configuration info that impacts the behavior of an application.

While an external configuration file can be changed—and subsequently reloaded by the application—without altering the source code, configuration specified as an annotation can only change by recompiling the source.

In a recent blog post, Annotation Transformers in Java, TestNG author Cedric Beust discusses various approaches to overriding annotations via XML, including:

[An] approach that blends these two techniques has also been under scrutiny recently: being able to override Java annotations with XML files. I haven't come across an implementation of this idea that can claim it's the defacto standard, and at this point, I suspect that this approach is unlikely to take off because it suffers from the "worst of both worlds" syndrome. Here is why. Consider the simple following code:

public class Mytest {  

@Test(invocationCount = 10)    
public void verify() {    // ...  }}

This instructs TestNG to invoke the verify() test method ten times. If I want to override this value at runtime, I find myself having to write a fairly complex piece of XML code in order to pinpoint the exact annotation I'm trying to override. It would probably look like this:

<override-annotation>
  <package name="org.foo.tests">
    <class name="MyClass">
      <method name="verify">
        <annotation name="org.testng.Test">
          <attribute name="invocationCount" value="15" />
        </annotation>
      </method>
    </class>
  </package>
</override-annotation>

I have been working on a slightly different approach with TestNG that lets you specify this runtime override in Java. Again, it's not a silver bullet and suffers from some of the compromises expressed above, but if you can live with the idea that you have to recompile your override if you modify it (but not the annotated code you are trying to override, so it's already a progress), it's actually fairly simple to achieve.

Beust introduces the notion of an annotation transformer interface:

public interface IAnnotationTransformer {  

    public void transform(ITest annotation, 
        Class testClass,
        Constructor testConstructor, 
        Method testMethod);
}

An implementation of this interface serves as the mechanism to transform the meaning of a specified annotation:

public class MyTransformer implements IAnnotationTransformer {  
    public void transform(ITest annotation, 
                          Class testClass,                        
                          Constructor testConstructor,         
                          Method testMethod)  {    
        if ("verify".equals(testMethod.getName())) {          
            annotation.setInvocationCount(15);    
        }  
    }
}

Overriding an annotation then does not require recompiling source code of the class that contains the specified annotation. Beust implemented annotation transformers in the upcoming 5.3 release of TestNG. Of this approach, he observes that:

The advantage of annotation transformers is that they can be very powerful: it's very easy to apply them to several methods, several classes, all methods of a class or of a package.

However, he also notes that implementation can be tricky, since, for instance, the current version of the JDK does not provide write access to annotations.

What guidelines do you follow in your projects to decide between annotations and an external configuration file? What do you think of Beust's approach to overriding annotation values?