I recently attended an excellent talk at JavaPolis, Elliotte Rusty Harold's XOM Design Principles. Although the talk is nominally about XOM (an API for XML documentation manipulation), in fact more than half of it is about API design principles in general. This is a curiously neglected subject. There are tons of books and articles about how to design and write good Java code, but surprisingly little about the specific topic of API design. Yet with the proliferation of new Java APIs, whether through JSRs or through Open Source projects, this is an increasingly important subject.

I've been closely involved with the evolution of the JMX API for over five years and have learnt a great deal about what works and what doesn't during that time. During the talk, I had the odd experience of continually wanting to cheer as Elliotte made point after point that I hugely agreed with.

I'm going to try to summarize here what I see as being the key points from this talk, from my own experience, and from a couple of other sources:

• An excellent tutorial on netbeans.org, How to Design a (module) API.
• A related NetBeans BOF at JavaOne 2005 by Tim Boudreau and Jaroslav Tulach, entitled How to write APIs that will stand the test of time.
• Of course, Josh Bloch's Effective Java book.

[Update: Although I was unaware of it when writing this blog entry, the slides referenced by Josh Bloch in a comment here cover some of the same ground and add much of interest.]

Design to evolve

If your API is worth anything, it will evolve over time. You should plan for this from the outset. A key part of the planning is to decide what sort of compatibility you will guarantee between revisions.

The best approach is to say that once something is in the API it will stay there and it will continue to work. Tweaking the API incompatibly between revisions will result in user reactions ranging from annoyance to murderous rage. The problem is particularly severe if your API ends up being used by different modules that are part of the same application. If Module 1 uses Commons Banana 1.0 and Module 2 uses Commons Banana 2.0 then life will be a whole lot easier if 2.0 is completely compatible with 1.0. Otherwise your users risk wasting huge amounts of time tweaking classpaths in a futile endeavour to make things work. They might end up having to play mind-destroying games with class-loaders, which is a clear signal that you have failed.

For APIs that are part of Java SE, we have an extreme form of compatibility. The aim is that no code whatsoever should break when you update from one version to the next. This means that classes and methods are never removed. It also means that we try to avoid changes that might break code that was depending on certain implementation details, even if the code shouldn't have been doing that.

The no-code-breakage rule applies to already-compiled code (binary compatibility). In some rare circumstances we might make changes that mean some existing code no longer compiles (source compatibility). For example, adding an overloaded method or constructor can sometimes produce ambiguity errors from the compiler when a parameter is null. We do try to find a way to avoid changes that break source compatibility in this way, but sometimes the best approach does imply that some source code might stop compiling. As an example, in Java SE 6 the constructors for javax.management.StandardMBean have been generified. Some existing source code might conceivably stop compiling because it does not respect the constraints that are expressed using generics here, but that code is easily fixed by adding a cast, and the rare cases where that happens are outweighed by cases where the constraints will catch programming errors at compile time.

In general, you can't know what users of your API will do with it. When contemplating a change that might break existing code, you have to reason conservatively. Only if you can honestly say that it is next to impossible that a change will break code can you reasonably make it. You should certainly rule out completely a signature change, which basically means removing or renaming a visible method or class or changing the parameters of a visible method. (But you can remove a method if it overrides a method in a parent class without changing the parent method's semantics.)

Since the very earliest versions of your API are sure to have many mistakes in them, and you don't want to freeze those mistakes for all time, it's a good idea to bring out one or more 0.x versions before the 1.0 version. Users of these versions know that the API is unstable and won't curse you if it changes. Once you've brought out 1.0 you're committing to compatibility. For APIs that are developed through the JCP, these 0.x versions correspond to the phases before the final release (Early Draft Review, Public Review, Proposed Final Draft). If possible, it's a good idea to make an implementation of the API available at the same time as these intermediate specifications.

If at some stage you decide that there's really too much accumulated cruft from previous versions and you want to start over, then create a new API with different package names. Then code that uses the old version and code that uses the new version can co-exist easily.

API design goals

What should the design goals of your API be? Apart from compatibility, the following goals from Elliotte's presentation seem like an excellent set:

• It must be absolutely correct. In the case of XOM, this meant that the API could never produce malformed XML documents no matter what the caller did. For the JMX API, for example, it means that you can never get the MBean Server into an inconsistent state by registering strange MBeans in it or using funny ObjectNames or performing several operations concurrently.
• It must be easy to use. This is hard to quantify. A good way to get an idea is to write lots of example code. Are there groups of operations that you keep having to repeat? Do you have to keep looking up your own API because you forget what things are called? Are there cases where the API doesn't do what you might expect?
• It must be easy to learn. This overlaps considerably with ease of use. But there are some obvious principles to make learning easier. The smaller the API, the less there is to learn. Documentation should include examples. Where appropriate, the API should look like familiar APIs.
• It must be fast enough. Elliotte was careful to put this in the list after the above items. Make sure the API is simple and correct. Then think about performance. You might be inclined to make API changes because the original API could only be implemented in an inefficient way. By all means change it to allow a more efficient implementation, provided you don't compromise correctness or simplicity. Don't rely on your intuition to know what performs well. Measure. Then tweak the API if you've determined that it really matters.
• It must be small enough. This covers the size of the compiled code and especially the amount of memory it needs as it runs. The same principles as for speed apply. Make it simple and correct first; measure; and only then think about tweaking the API.

Be minimalist

Because of the compatibility requirement, it's much easier to put things in than to take them out. So don't add anything to the API that you're not sure you need.

There's an approach to API design which you see depressingly often. Think of everything a user could possibly want to do with the API and add a method for it. Toss in protected methods so users can subclass to tweak every aspect of your implementation. Why is this bad?

• The more stuff there is in the API, the harder it is to learn. Which classes and methods are the important ones? Which of the five different ways to do what I need is the best?

  The situation is exacerbated by the Javadoc tool, which dumps all the classes in a package, and all the methods in a class, in an undifferentiated lump. We can expect that JSR 260 will update the Javadoc tool to allow you to produce "views" of the API, and in that case fatter APIs will not be so overwhelming.

• The bigger the API, the more things can go wrong. The implementation isn't going to be perfect, but the same investment in coding and testing will yield better results for a smaller API.

• If your API has more methods than it needs, then it's taking up more space than it needs.

The right approach is to base the API on example code. Think of problems a user might want to solve with the API. Add just enough classes and methods to solve those problems. Code the solutions. Remove anything from the API that your examples don't need. This allows you to check that the API is useful. As a happy side-effect, it gives you some basic tests. And you can (and should) share the examples with your users.

Interfaces are overvalued

There's a certain style of API design that's very popular in the Java world, where everything is expressed in terms of Java interfaces (as opposed to classes). Interfaces have their place, but it is basically never a good idea for an entire API to be expressed in terms of them. A type should only be an interface if you have a good reason for it to be. Here's why:

• Interfaces can be implemented by anybody. Suppose String were an interface. Then you could never be sure that a String you got from somewhere obeyed the semantics you expect: it is immutable; its hashCode() is computed in a certain way; its length is never negative; and so on. Code that used String, whether user code or code from the rest of the J2SE platform, would have to go to enormous lengths to ensure it was robust in the face of String implementations that were accidentally incorrect. And to even further lengths to ensure that its security could not be compromised by deliberately evil String implementations.

  In practice, implementations of APIs that are defined entirely in terms of interfaces often end up cheating and casting objects to the non-public implementation class. DOM typically does this for example. So you can't give your own implementation of the DocumentType interface as a parameter to DOMImplementation.createDocument and expect it to work. Then what's the point in having interfaces?

• Interfaces cannot have constructors or static methods. If you need an instance of an interface, you either have to implement it yourself, or you have to ask some other object for it. If Integer were an interface, then to get the Integer for a given int you could no longer use the obvious new Integer(n) (or, less obvious but still documented inside Integer, Integer.valueOf(n)). You would have to use IntegerFactory.newInteger(n) or whatever. This makes your API harder to understand and use.

• Interfaces cannot evolve. Suppose you add a new method to an interface in version 2 of your API. Then user code that implemented the interface in version 1 will no longer compile because it doesn't implement the new method. You can still preserve binary compatibility by catching AbstractMethodError around calls to the new method but that is clunky. If you use an abstract class instead of an interface you don't have this problem. If you tell users not to implement the interface then you don't have this problem either, but then why is it an interface?

• Interfaces cannot be serialized. Java serialization has its problems, but you can't always get away from it. The JMX API relies heavily on serialization, for example. For better or worse, the way serialization works is that the name of the actual implementation class is serialized, and an instance of that exact same class is reconstructed at deserialization. If the implementation class is not a public class in your API, then you won't interoperate with other implementations of your API, and it will be very hard for you to ensure that you even interoperate between different versions of your own implementation. If the implementation class is a public class in your API, then do you really need the interface as well?

Of course, there are sometimes good reasons for a type to be interface. Here are some common ones:

• Callbacks. If the interface is intended to be implemented by user code, then it is often more appropriate than an abstract class. See Runnable for example. This is mostly true of interfaces with just one method. Once there start being several methods you often find that an implementation class only needs to do something in one of them, and it's annoying to have to implement all the others. Furthermore if an interface has three methods today then you might want it to have four tomorrow, which is not usually possible as we saw. An abstract class can avoid these problems.

• Multiple inheritance. It is occasionally useful to be able to implement an interface deep in the inheritance hierarchy. A good example is Comparable, where for example Integer is Comparable but its parent class Number is not. However, there aren't many other good examples of this in the core Java classes. It is usually bad practice to implement some random interface in a class whose primary purpose is something else. Implementing the interface in a private inner class is usually cleaner, and then of course it could just as well be an abstract class.

• Dynamic proxies. The invaluable java.lang.reflect.Proxy class allows you to make an implementation of any interface at runtime, where calling any of the interface's methods results in a call to a single invoke method. There's no way to construct a dynamic proxy for an abstract class, so if you think it will be useful for users to make dynamic proxies that is one reason to favour an interface. (cglib can sometimes be used to achieve the same effect for abstract classes, but with several limitations, plus the documentation is really poor.)

Be careful with packages

The Java language has fairly limited ways of controlling the visibility of classes and methods. In particular, if a class or method is visible outside its package, then it is visible to all code in all packages. This means that if you define your API in several packages, you have to be careful to avoid being forced to make things public just so that code in other packages in the API can access them.

The simplest solution to avoid this is to put your whole API in one package. For an API with fewer than about 30 public classes this is usually the best approach.

If your API is too big for a single package to be appropriate, then you should plan to have private implementation packages. That is, some packages in your implementation are excluded from the Javadoc output and are not part of the public API, even though their contents are accessible. If you look at the JDK, for example, there are many sun.* and com.sun.* packages of this sort. Users who rely on the Javadoc output will not know of their existence. Users who browse the source code can see them, and can access the public classes and methods, but they are discouraged from doing so and warned that there is no guarantee that these classes will remain unchanged across revisions.

A good convention for private packages is to put internal in the name. So the Banana API might have public packages com.example.banana and com.example.banana.peel plus private packages com.example.banana.internal and com.example.banana.internal.peel.

Don't forget that the private packages are accessible. There may be security implications if arbitrary code can access these internals. Various techniques exist to address these. The NetBeans API tutorial describes one. In the JMX API, we use another. There is a class javax.management.JMX which contains only static methods and has no public constructor. This means that user code can never have an instance of this class. So in the private com.sun.jmx packages, we sometimes add a parameter of type JMX to sensitive public methods. If a caller can supply a non-null instance of this class, it must be coming from the javax.management package.

Other random tips

Here are some other random tips based on our experience with the JMX API and on the sources I mentioned.

Immutable classes are good. If a class can be immutable, then it should be. Rather than spelling out the reasons, I'll refer you to Item 13 in Effective Java. You wouldn't think of designing an API without having this book, right?

The only visible fields should be static and final. Again this one is pretty banal and I mention it only because certain early APIs in the core platform violated it. Not an example to follow.

Avoid eccentricity. There are many well-established conventions for Java code, with regard to identifier case, getters and setters, standard exception classes, and so on. Even if you think these conventions could have been better, don't replace them in your API. By doing so you force users to throw away what they already know and learn a new way of doing an old thing.

For instance, don't follow the bad example of java.nio and java.lang.ProcessBuilder where the time-honoured T getThing() and void setThing(T) methods are replaced by T thing() and ThisClass thing(T). Some people think this is neato-keen and others that it is an abomination, but either way it's not a well-known idiom so don't force your users to learn it.

Don't implement Cloneable. It is usually less useful than you might think to create a copy of an object. If you do need this functionality, rather than having a clone() method it's generally a better idea to define a "copy constructor" or static factory method. So for example class Banana might have a constructor or factory method like this:

      public Banana(Banana b) {      // copy constructor
    	  this(b.colour, b.length);
      }
      // ...or...
      public static Banana newInstance(Banana b) {
    	  return new Banana(b.colour, b.length);
      }
    

The advantage of the constructor is that it can be called from a subclass's constructor. The advantage of the static method is that it can return an instance of a subclass or an already-existent instance.

Item 10 of Effective Java covers clone() in excruciating detail.

Exceptions should usually be unchecked. Item 41 of Effective Java gives an excellent summary here. Use a checked exception "if the exceptional condition cannot be prevented by proper use of the API and the programmer using the API can take some useful action once confronted with the exception." In practice this usually means that a checked exception reflects a problem in interaction with the outside world, such as the network, filesystem, or windowing system. If the exception signals that parameters are incorrect or than an object is in the wrong state for the operation you're trying to do, then an unchecked exception (subclass of RuntimeException) is appropriate.

Design for inheritance or don't allow it. Item 15 of Effective Java tells you all you might want to know about this. The summary is that every method should be final by default (perhaps by virtue of being in a final class). Only if you can clearly document what happens if you override the method should it be possible to do so. And you should only do that if you have coded useful examples that do override the method.

Summary

• Design to evolve.
• Correctness, then simplicity, then efficiency.
• Interfaces are overvalued.
• Be careful with packages.
• Read Effective Java.

  interface String {
    signature {
      int size() {
        post { result >= 0; }
      }
      String concat(String s) {
        post { result.size() == size() + s.size(); }
      }
      ...
    } 
  }