Continuous Integration Engine Comparison

In lieu of my task of setting up a testing workbench at our development office I've been checking out various integration servers. Focusing on Bamboo, Continuum and Hudson. What follows is an informal look on various continuous integration servers.

Bamboo

I started with rolling out Bamboo. Mainly because it is part of the Atlassian portfolio. We have been impressed with Confluence in the past, and JIRA as well has been making a big foothold in the market. JIRA being something we are seriously considering implementing as our issue tracker and project planning tool. So it was a fair bet that these guys were up to something impressive. Or atleast, we'll qualify for a discount if we buy more than one licensed product :p

The install went flawlessly. Just download the tar.gz and run the bamboo.sh script. Create a user and go on to log in to configure it. There was really nothing much to configure. I just had to point it to my JIRA instance (hey, its integrated with JIRA) and create a Maven2 builder.

Bamboo has the ability to run several agents to distribute the build load. This isn't something I am going to dig into for now, as I am just going to do local builds. But its nice to have the opportunity the day our builds are taking way too long.

Ok, I'm ready to create my first build plan. The wizard-like interface took me through it without much pain. I hooked it up to my Subversion repository and told it to build with Maven. Clicked that the build was going to produce test results. And away it went - pulling down my sources and starting the build - as expected.

It then displays some graphs about the rate of success of your builds, with a summary with information about tests which took a long time to fix. In essence, it gives you a overview about which parts of your code is harder to maintain.

Bamboo build duration chart

Test summary

Ok, this was more or less what I was expecting from a build system. No fuzz, and it works. And then handing over an overview of whats going on between my commits in the codebase. As well as telling me off via IM if my build breaks.

As a bonus point there seems to be a well-documented plugin developer's guide. And with several extensions already written, this project looks promising.

Hudson

Hudson is an open source continous integration server, and written in Java as well. As with Bamboo, you can either run it standalone, or deploy it as a web-application in your JEE container (such as Tomcat).

It also has support for Maven2, and of course Subversion. So no problems there either. I got to configure my project easily and was up and flying with my first project build within 5 minutes.

However, what is lacking is a nice overview of my project status. No graphs, just a list of the build history with a red or blue light depending on wether the build broke or not.

Hudson also has a plugin framework. And I found some plugins written for it that seemed to be of interest - like JIRA integration and Jabber messaging. But no fancy visuals to hang on my wall.

Continuum

Next on my list was Continuum. This is a standalone jetty app. No installation fuzz necessary here either. It has a more tight Maven2 integration, so it relies on you having configured the scm plugin in your pom.xml:

<project>
...
<scm>
<connection>scm:svn:http://svn/trunk/project</connection>
</scm>
...
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-scm-plugin</artifactId>
<version>1.0</version>
<configuration>
<connectionType>connection</connectionType>
</configuration>
</plugin>
</plugins>

...

After adding the relevant sections into my pom and checking it into my repository, I just pointed Continuum to my pom.xml in via http. It pulled down the file, and used it to configure the rest of my project from that. Smart. And built it successfully.

But also here is missing the features of Bamboo that I fell in love with. Graphically showing me how healthy my projects are. Its just lists. And no information about the tests that were run - only the maven console output. You have to dig yourself into several screens to get a feel of the history of your project.

Summary

For now it looks like Bamboo will be what we will evaluate in more detail. Its not only a build system, but actually gives you a feel about how your project is doing over time - and that seems quite useful in a Q&A system. We need heads up on where to focus our attention, and tools are a way to help us do that. Hudson and Continuum also has this ability, but its buried under layers of engineering specifics, instead of letting it float to the top and give us the goodies.

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • E-mail this story to a friend!
  • TwitThis
  • bodytext
  • del.icio.us
  • Facebook
  • Google
  • Reddit
  • Slashdot

You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.

AddThis Social Bookmark Button

4 Responses to “Continuous Integration Engine Comparison”

  1. Hi Alex,

    Thanks for all the kind words about Bamboo :). Just in case you’re wondering if all the ‘pretty graphs’ are actually useful, have a read of this:
    http://blogs.atlassian.com/developer/2007/05/full_speed_ahead.html

    Cheers,
    Andreas

  2. Edwin Wong (Atlassian) Says:

    Hi Alex,

    Thanks for checking out Bamboo!

    With regards to Bamboo plugins, you can find them at our repository here: http://confluence.atlassian.com/display/BAMEXT/Home.

    Also, you’ll be more than welcome to contribute as well!

    Cheers,
    Edwin

  3. I surprised you saw no graphs with Hudson… bit late now I guess but I have lots of graphs for tests failures/passing, disk usage, findbugs results, check style results and so on. There is also the nice weather icon on the main page to give you an idea of how well your jobs are doing over all.

  4. Adam,

    Maybe me not finding all the graphs and other snacks Hudson offers me is a lack of design? Good software should IMHO be self explanatory, and offer a view that doesn’t hide all the needed functions :)