Freelance Developer

I wanted a tool to write acceptance tests with. Something easy to set up, and of course, flexible. What gets churned out here at my company are webapps, so I had this in mind when I went ahead looking for the right tool. After browsing around I came to the conclusion that the tools with most drive behind them was FIT and Selenium. Here follows my encounter with them.

FIT

This looked like a neat idea. You have your customers come up with a html table through any WYSIWYG application - like MS Word or whatever. The customer then creates columns with input values and an expected result column.

As a developer you then write a fixture that takes this html file as input. In essence - data driven testing.

You also have fitnesse (which I have been unable to download because of their site problems) that has a wiki style frontend for FIT tables. Your customers can enter the wiki, and create their own tests, change them, run them and view the results.

However, I'm not convinved that this is going to supplement our jUnit testing on a web framework. You should already have tests that check if you can handle the login() procedure with both a valid and invalid username in your standard unit-tests. I'm not really seeing the usecase here I guess. And this won't really show us any errors we get because of some missing html or http component. And since web is our bread and butter I gave this the thumbs down for now.

Selenium

In comes Selenium. And for webapps this looks like a killer testing platform. With the firefox plugin I can record an interactive session I have with a webpage. For example, I can login and have it search for some text to use as a success-criteria. I can for example check that all menupoints are on certain pages, or do form validator checking. And its all done in firefox. I open the Firefox Selenium IDE plugin. Hit the record-button and start using the site I'm developing for as a normal user. I then save the recorded session for later playback so I can check if the next version I am about to deploy breaks something.

But I want to automate my selenium tests. I don't want to open up firefox every time my webapp is due for acceptance testing - which should be every time I check in my code. This is something I want handled by my integration server.

It seems the firefox plugin can export my selenium tests (originally written in selenese) to Java. Hmm, nice. Maybe I can use this for jUnit then.

And there is a Java API for selenium!

I set up a Selenium Remote Control server. I am developing on a Linux box, and therefore don't have access to Internet Explorer and Safari locally. But we have a test-server in a corner which runs Windows. So I downloaded Selenium RC on it. I can now connect to this server instance and have it execute selenium commands on a wide range of browsers running on that machine.

To do this through java code I added a dependencies in my pom.xml to the selenium API:

<dependency>
<groupId>org.openqa.selenium.client-drivers</groupId>
<artifactId>selenium-java-client-driver</artifactId>
<version>1.0-beta-1</version>
<scope>test</scope>
</dependency>

And in my $HOME/.m2/settings.xml file I had to add the selenium repository server:

<profile>
<id>selenium</id>
<repositories>
<repository>
<id>openqa.org</id>
<name>Openqa Release Repository</name>
<url>http://archiva.openqa.org/repository/releases</url>
<layout>default</layout>
<snapshots><enabled>false</enabled></snapshots>
<releases><enabled>true</enabled></releases>
</repository>
</repositories>
</profile>

Now we are ready to create a jUnit test file based on the output the firefox plugin recorded earlier. The export function created a skeleton for me to base my test on:

package com.example.tests;
 
import com.thoughtworks.selenium.*;
import java.util.regex.Pattern;
 
public class NewTest extends SeleneseTestCase {
        public void setUp() throws Exception {
                setUp("http://change-this-to-the-site-you-are-testing/",
                      "*chrome");
        }
        public void testNew() throws Exception {
                selenium.open("/");
                selenium.click("link=Børs/marked");
                selenium.waitForPageToLoad("30000");
                selenium.click("link=Aksjekurser");
                selenium.waitForPageToLoad("30000");
                verifyTrue(selenium.isTextPresent("DN Finans"));
                selenium.click("link=markedsoversikt");
                selenium.waitForPageToLoad("30000");
                verifyTrue(selenium.isTextPresent("Børskommentar Oslo"));
                selenium.click("link=DN.no");
                selenium.waitForPageToLoad("30000");
                verifyTrue(selenium.isTextPresent("SØK"));
        }
}

The SeleneseTestCase class is a class coming with the Selenium API. It extends the TestCase method already in jUnit and adds verifyXXX() methods instead of using assertXXX(). These verifiers don't throw exceptions like assert does, but rather collects all the errors and waits until the tearDown is called to show any errors that occured.

I opted out of this because I want my tests to fail the minute they do. I'd rather have several selenium test suites breaking up the tests in easier to manage chunks. I also wish to check on several browsers at once, so I just set up a standard jUnit test to see if all works as expected:

public class DnTest {
 
        private static DefaultSelenium firefox;
        private static DefaultSelenium explorer;
 
        /**
         * @throws java.lang.Exception
         */
        @BeforeClass
        public static void setUpBeforeClass() throws Exception {
                firefox = new DefaultSelenium("selenium.nhst.as", 4444,
                                              "*firefox", "http://www.dn.no");
                explorer = new DefaultSelenium("selenium.nhst.as", 4444,
                                               "*iexplore", "http://www.dn.no");
        }
 
        /**
         * @throws java.lang.Exception
         */
        @AfterClass
        public static void tearDownAfterClass() throws Exception {
                firefox = null;
                explorer = null;
        }
        @Test
        public void testMenu() throws Exception {
                firefox.start();
                firefox.open("/");
                firefox.click("link=Børs/marked");
                firefox.waitForPageToLoad("30000");
                firefox.click("link=Aksjekurser");
                firefox.waitForPageToLoad("30000");
                assertTrue(firefox.isTextPresent("DN Finans"));
                firefox.click("link=markedsoversikt");
                firefox.waitForPageToLoad("30000");
                assertTrue(firefox.isTextPresent("Børskommentar Oslo"));
                firefox.click("link=DN.no");
                firefox.waitForPageToLoad("30000");
                assertTrue(firefox.isTextPresent("SØK"));
                firefox.stop();
        }
}

The above snippet connects to my selenium rc host. Requests an firefox and iexplore session and feeds the firefox session with commands that are to be executed. And if any error occurs, the asserts will fail and give me a standard assert error I can address in Eclipse by clicking on the line that failed.

However, you would probably want to run firefox.start() and firefox.stop() in your setup() and tearDown(), so that firefox will be closed even though a test fails.

But this takes a looooong time to run. It can take several minutes to run. It starts firefox, sends queues commands and closes firefox. And then you do the same with other browsers. So I don't want these tests to be in my standard test suite executed by the maven test goal. I wanted to only execute this during the integration-test phase.

Lets configure the maven-surefire-plugin:

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/selenium/*Test.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<id>surefire-it</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/selenium/*Test.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
This code will exclude every test in the selenium-package during normal test run, but include them under the integration-test phase.

And now we have automated our Selenium acceptance tests :)

You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.