The process of continuous integration has been around since at least Martin Flowler's article on the topic in 2000. Since then, a handful of continuous integration tools emerged, both closed- and open-source. In a recent JavaWorld article, Which open source CI tool is best suited for your application's environment?, John Ferguson Smart compares four open-source ones: CruiseControl, Continuum, Luntbuild, and Hudson.

The gist of his comparison is that while those tools differ in the richness of features as well in their ease of setup, they all provide the same core benefits of giving an instant snapshot of a project's health: They pull source files from a source control system, automatically run a set of builds and tests, and then publish the test results on Web pages, or via RSS and email.

CruiseControl is the most mature tool, according to Smart:

It supports virtually any type of project, be it Ant, Maven, Maven 2, make, or just a plain-old command line script, as well as a wide range of SCM (source configuration management) tools. Its notification techniques are second to none. And it also benefits from a large number of third-party plug-ins... CruiseControl is a highly configurable tool, and all CruiseControl configuration is done in a central XML configuration file... Setting up a project using CruiseControl involves a bit more work than with the other tools...

One of CruiseControl's strengths is its sophisticated and reactive build cycle. CruiseControl continuously monitors the SCM servers for any updates and triggers a build whenever it detects any new changes to the source code repository... Developers don't need to wait for scheduled builds to learn about integration issues. Builds can also be scheduled to be run at certain times or intervals.

A more recent entrant in the field is Continuum. It is an outcome of the Maven project management toolkit and, not surprisingly, supports Maven configurations well:

Continuum is still a relatively young product, with a lot of room for new and improved features. A potential issue for big projects or programs is that, like CruiseControl, Continuum provides no simple way of handling dependencies between projects... Nevertheless, Continuum is easy to install and easy to use, and will suffice in many situations. Its simplicity, and its intuitive Web administration console, make it a good candidate for introducing continuous integration into organizations that have had no previous experience with the subject.

A less-known open-source alternative is Luntbuild. Perhaps because it has its roots in a commercial product, Luntbuild offers a large feature set, including the ability to interface with CVS, Subversion, Borland StarTeam, AccuRev, IBM Rational ClearCase, and Microsoft's Visual Source Safe:

Luntbuild is produced and maintained by a company called PMEase. Luntbuild also has a commercial cousin, called QuickBuild, which is marketed by the same company... [It] comes with a real installer, which walks you through the installation process and proposes a number of configuration options. At the simplest level, you can use a fully self-contained embedded database and Jetty Web server. Or you can configure Luntbuild to work with an external database, and/or on an external Web server. You can even set up your user accounts from an external LDAP (lightweight directory access protocol) directory.

Hudson is a newcomer to the field, but it already features a powerful Web UI:

Hudson boasts a particularly clear presentation of build results... In addition to the build status and a summary of the changes made in the source code repository since the last build, the build results page can also display downloadable artifacts generated by the build. Hudson also provides excellent built-in support for JUnit test reports: test results, as well as a graph of historical test data can be viewed directly from within the build results page.

Hudson also comes with several innovative features not found in other continuous integration tools, such as the ability to checksum project artifacts:

Hudson records a unique "fingerprint" (actually, just the MD5 checksum) for each artifact generated during a build. So if you find yourself with an artifact, and you aren't sure which build generated it, Hudson can work it out for you. This functionality can be useful when tracking down incompatibilities between library versions.

Some continuous integration systems, such as CruiseControl, provide fun features as well, such as an X10 interface that allows developers to connect a lava lamp to CruiseControl to indicate project status (green for passed tests, red or flashing red for failed tests).

With those many choices, it may be hard to choose a continuous integration solution. What continuous integration do you use for your projects? What features were most important in making your choice?