Feeds:
Posts
Comments

Posts Tagged ‘webdriver’

Want to learn more about WebDriver? What do you want to know?

If you’re testing a web application, you can’t go far wrong with Selenium WebDriver. But in this web 2.0 world of ajax-y goodness, it can be a pain dealing with the asynchronous nature of modern sites. Back when all we had was web 1.0 you clicked a button and eventually you got a new page, or if you were unlucky: an error message. But now when you click links all sorts of funky things happen – some of which happen faster than others. From the user’s perspective this creates a great UI. But if you’re trying to automate testing this you can get all sorts of horrible race conditions.

Thread.sleep

The naive approach is to write your tests the same way you did before: you click buttons and assert that what you expected to happen actually happened. For the most part, this works. Sites are normally fast enough, even in a continuous integration environment, that by the time the test harness looks for a change it’s already happened.

But then… things slow down a little and you start getting flickers - tests that sometimes pass and sometimes fail. So you add a little delay. Just 500 milliseconds should do it, while you wait for the server to respond and update the page. Then a month later it’s flickering again, so you make it 1 second. Then two… then twenty.

The trouble is, each test runs at the pace that it runs at its slowest. If login normally takes 0.1 seconds, but sometimes takes 10 seconds when the environment’s overloaded – the test has to wait for 10 seconds so as not to flicker. This means even though the app often runs faster,  the test has to wait just in case.

Before you know it, your tests are crawling and take hours to run – you’ve lost your fast feedback loop and developers no longer trust the tests.

An Example

Thankfully WebDriver has a solution to this. It allows you to wait for some condition to pass, so you can use it to control the pace of your tests. To demonstrate this, I’ve created a simple web application with a login form – the source is available on github. The login takes a stupid amount of time, so the tests need to react to this so as not to introduce arbitrary waits.

The application is very simple – a username and password field with an authenticate button that makes an ajax request to log the user in. If the login is successful, we update the screen to let the user know.

The first thing is to write our test (obviously in the real world we’d have written the test before our production code, but its the test that’s interesting here not what we’re testing – so we’ll do it in the wrong order just this once):

@Test
public void authenticatesUser()
{
    driver.get("http://localhost:8080/");

    LoginPage loginPage = LoginPage.open(driver);
    loginPage.setUsername("admin");
    loginPage.setPassword("password");
    loginPage.clickAuthenticate();
    Assert.assertEquals("Logged in as admin", loginPage.welcomeMessage());
}

We have a page object that encapsulates the login functionality. We provide the username & password then click authenticate. Finally we check that the page has updated with the user message. But how have we dealt with the asynchronous nature of this application?

WebDriverWait

Through the magic of WebDriverWait we can wait for a function to return true before we continue:

public void clickAuthenticate() {
    this.authenticateButton.click();
    new WebDriverWait(driver, 30).until(accountPanelIsVisible());
}

private Predicate<WebDriver> accountPanelIsVisible() {
    return new Predicate<WebDriver>() {
        @Override public boolean apply(WebDriver driver) {
            return isAccountPanelVisible();
        }
    };
}
private boolean isAccountPanelVisible() {
    return accountPanel.isDisplayed();
}

Our clickAuthenticate method clicks the button then instructs WebDriver to wait for our condition to pass. The condition is defined via a predicate (c’mon Java where’s the closures?). The predicate is simply a method that will run to determine whether or not the condition is true yet. In this case, we delegate to the isAccountPanelVisible method on the page object. This does exactly what it says on the tin, it uses the page element to check whether it’s visible yet. Simple, no?

In this way we can define a condition we want to be true before we continue. In this case, the exit condition of the clickAuthenticate method is that the asynchronous authentication process has completed. This means that tests don’t need to worry about the internal mechanics of the page – about whether the operation is asynchronous or not. The test merely specifies what to test, the page object encapsulates how to do it.

Javascript

It’s all well and good waiting for elements to be visible or certain text to be present, but sometimes we might want more subtle control. A good approach is to update Javascript state when an action has finished. This means that tests can inspect javascript variables to determine whether something has completed or not – allowing very clear and simple coordination between production code and test.

Continuing with our login example, instead of relying on a <div> becoming visible, we could instead have set a Javascript variable. The code in fact does both, so we can have two tests. The second looks as follows:

public void authenticate() {
    this.authenticateButton.click();
    new WebDriverWait(driver, 30).until(authenticated());
}

private Predicate<WebDriver> authenticated() {
    return new Predicate<WebDriver>() {
        @Override public boolean apply(WebDriver driver) {
            return isAuthenticated();
        }
    };
}

private boolean isAuthenticated() {
    return (Boolean) executor().executeScript("return authenticated;");
}
private JavascriptExecutor executor() {
    return (JavascriptExecutor) driver;
}

This example follows the same basic pattern as the test before, but we use a different predicate. Instead of checking whether an element is visible or not, we instead get the status of a Javascript variable. We can do this because each WebDriver also implements the JavascriptExecutor allowing us to run Javascript inside the browser within the context of the test. I.e. the script “return authenticated” runs within the browser, but the result is returned to our test. We simply inspect the state of a variable, which is false initially and set to true once the authentication process has finished.

This allows us to closely coordinate our production and test code without the risk of flickering tests because of race conditions.

Read Full Post »

Want to learn more about WebDriver? What do you want to know?

So you’ve written your first Selenium 2.0 test, but is that really the right way to build tests? In the second article in this series we’ll look at the difference between test specification and test implementation and how Selenium achieves this with page objects.

Specification vs Implementation

There’s an important difference in tests between the specification of what to test versus the implementation of how to test.

For example, my test specification might be “When the user enters their username and password and clicks the login button, then they are logged in and can see their recommendations”. This describes a scenario – it’s a specification of what the test should do. However, the test implementation has to deal with things like:

  • The username field is named “f_username”
  • The password field is named “f_password”
  • The login button is found via the CSS “#loginButton input”

If I change the layout of my login page, does the specification change? Hell no – I still need to provide credentials and click the login button. But has my implementation changed? Almost certainly!

Separating test specification from test implementation makes tests more robust. If I change how login works, I don’t want to have to change every single test that needs a logged in user – there’s probably a few of them! Instead, I only want to change the implementation – which fields to use, which buttons to press – in a single, common location.

Using Page Objects

In Selenium, you can separate specification from implementation by using page objects. That is, unlike in our previous example where the test code is littered with implementation details (how to find the keyword field or how to find the submit button), instead we move this logic into a page object.

If we built a page object to represent the Amazon home page, what would it look like?

public class AmazonHomePage {
    public static AmazonHomePage navigateTo(WebDriver driver);
    public AmazonSearchResultsPage searchFor(String searchTerm);
}

The homepage has two responsibilities that our test cares about:

  1. Given a browser, navigate to the page
  2. Once on the page, search for a specific search term

Notice that the searchFor operation returns a different type of page object – because after entering the search we will be on a new page: the search results page. What does that page look like?

public class AmazonSearchResultsPage {
    public String getTopResultTitle();
}

We only need one thing from our search results page – the title of the top result.

Tests with Page Objects

The page object encapsulates all the logic about how to perform certain actions. Our test now only needs to deal with what to actually do – so what does our test look like now?

AmazonHomePage homePage = AmazonHomePage.navigateTo(driver);
AmazonSearchResultsPage resultsPage =
    homePage.searchFor("iain banks");
assertThat(resultsPage.getTopResultTitle(), is("Transition"));

This is a pretty neat specification for our test: given a user on the amazon home page, when the user searches for ‘iain banks’, then the title of the top result is ‘Transition’.

There’s no implementation details in our test now. If any of the implementation details around searching change – we only need to change our page object. This is particularly important when the page objects are reused by multiple tests. Instead of having to manually change dozens of tests, we make our change in a single location – the page object.

Implementing Page Objects

By removing the implementation details from the test, they now sit within the page object. So how could we implement the page object interfaces we defined above? We could simply reuse the logic we had in our tests previously:

public AmazonSearchResultsPage searchFor(String searchTerm) {
    // Enter a search term
    WebElement keywordsField =
        driver.findElement(By.name("field-keywords"));
    keywordsField.sendKeys(searchTerm);

    // Click go
    WebElement goButton =
        driver.findElement(By.cssSelector("#navGoButton input"));
    goButton.click();
    ...

However, Selenium gives us another way to do this – we can also define the web elements declaratively, by using annotations. Although either approach is valid, using annotations tends to be neater. So let’s have a look at the top of the homepage class now:

public class AmazonHomePage {

	@FindBy(name="field-keywords")
	private WebElement keywordsField;

	@FindBy(css="#navGoButton input")
	private WebElement goButton;

Here we define the same two WebElements we had in our test before. But rather than call the web driver to find the elements for us explicitly, they’re now pre-populated based on the annotation.

To implement the interface we described above, we simply need to invoke the sendKeys and click methods on these elements as we did previously:

public AmazonSearchResultsPage searchFor(String searchTerm) {
    keywordsField.sendKeys(searchTerm);
    goButton.click();
    return PageFactory.initElements(driver,
        AmazonSearchResultsPage.class);
}

On lines two and three we type in our search term to the field we declared above and then click the go button. But what on earth is happening on lines four and five? This piece of magic – the PageFactory – is what allows us to use annotations to declare our elements.

Page Factory

The page factory instantiates our page object (AmazonSearchResultsPage) and finds the annotated fields. The fields are then initialised to the associated elements on the page. This is how we’re able to simply use the fields without having to initialise them. The returned search results page is then a fully populated page object, ready to be used by the test as before.

There’s one more method to see on the homepage – the navigateTo method. This also uses the page factory – this time to initialise our AmazonHomePage.

	public static AmazonHomePage navigateTo(WebDriver driver) {
		driver.get("http://www.amazon.co.uk/");
		return PageFactory.initElements(driver,
             AmazonHomePage.class);
	}

And there we have it – a better amazon search example – now using page objects and annotations. This is a much better way to structure tests than having implementation and specification intermingled within the test.

As before, all the examples are available on github. This article was part 2 in a series, if there’s a specific aspect of using Selenium you’d like to see covered in a future article let us know in the comments below.

Read Full Post »

On a pet project recently I set out to build automated UI (integration) tests as well as the normal unit tests. I wanted to get all of this integrated into my maven build, with code coverage reports so I could get an idea of areas with insufficient test coverage. Rather than just publish the source code for the project, I’ve put together a simple example to demonstrate how I got all this setup; so if you’re looking to integrate maven, junit, webdriver (now selenium) and emma - read on to find out how I went about it.

First off, all the source code for this is available on github: https://github.com/activelylazy/coverage-example. I’ll show key snippets, but obviously there’s lots of detail omitted that (hopefully) isn’t relevant.

The Example App

Rather than break with tradition, the example application is a simple, if slightly contrived, hello world:

How It Works

The start page is a simple link to the hello world page:

<h1>Example app</h1>
<p>See the <a id="messageLink" href="helloWorld.html">message</a></p>

The hello world page just displays the message:

<h1>Example app</h1>
<p id="message"><c:out value="${message}"/></p>

The hello world controller renders the view, passing in the message:

public class HelloWorldController extends ParameterizableViewController {
    // Our message factory
    private MessageFactory messageFactory;
    @Override
    protected ModelAndView handleRequestInternal(HttpServletRequest request,
        HttpServletResponse response) throws Exception {
        // Get the success view
        ModelAndView mav = super.handleRequestInternal(request, response);
        // Add our message
        mav.addObject("message",messageFactory.createMessage());
        return mav;
    }
    @Autowired
    public void setMessageFactory(MessageFactory messageFactory) {
        this.messageFactory = messageFactory;
    }
}

Finally the MessageFactory simply returns the hard-coded message:

public String createMessage() {
    return "Hello world";
}

The unit test

We define a simple unit test to verify that the MessageFactory behaves as expected:

public class MessageFactoryTest {
    // The message factory
    private MessageFactory messageFactory;
    @Test
    public void testCreateMessage() {
        assertEquals("Hello world",messageFactory.createMessage());
    }
    @Autowired
    public void setMessageFactory(MessageFactory messageFactory) {
        this.messageFactory = messageFactory;
    }
}

Build

A basic maven pom file is sufficient to build this and run the unit test. At this point we have a working app, with a unit test for the core functionality (such as it is) that we can build and run.

<project>
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.example</groupId>
    <artifactId>helloworld</artifactId>
    <packaging>war</packaging>
    <version>1.0-SNAPSHOT</version>
    <name>helloworld Maven Webapp</name>
    <build>
        <finalName>helloworld</finalName>
    </build>
    <dependencies>
        ...omitted...
    </dependencies>
</project>

Code Coverage

Now let’s integrate Emma so we can get some code coverage reports. First, we define a new Maven profile, this allows us to control whether or not we use emma on any given build.

<profile>
    <id>with-emma</id>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>emma-maven-plugin</artifactId>
                <inherited>true</inherited>
                <executions>
                    <execution>
                        <id>instrument</id>
                        <phase>process-test-classes</phase>
                        <goals>
                            <goal>instrument</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</profile>

This simply invokes the “instrument” goal during the Maven “process-test-classes” phase; i.e. once we’ve compiled our class files, use emma to instrument them. We can run this by invoking maven with the new profile:

mvn clean install -Pwith-emma

Once the build has completed, we can run Emma to generate code coverage reports:

On Windows:

java -cp %USERPROFILE%/.m2/repository/emma/emma/2.0.5312/emma-2.0.5312.jar emma report -r xml,html -in coverage.ec -in target/coverage.em

On Linux:

java -cp ~/.m2/repository/emma/emma/2.0.5312/emma-2.0.5312.jar emma report -r xml,html -in coverage.ec -in target/coverage.em

We can now view the HTML coverage report in coverage/index.html. At this point, it shows we have 50% test coverage (by classes). MessageFactory is fully covered, but the HelloWorldController doesn’t have any tests at all.

Integration Test

To test our controller and JSP, we’ll use WebDriver to create a simple integration test; this is a JUnit test that happens to launch a browser.

public class HelloWorldIntegrationTest {
    // The webdriver
    private static WebDriver driver;
    @BeforeClass
    public static void initWebDriver() {
        driver = new FirefoxDriver();
    }
    @AfterClass
    public static void stopSeleniumClent() {
        try {
            driver.close();
            driver.quit();
        } catch( Throwable t ) {
            // Catch error & log, not critical for tests
            System.err.println("Error stopping driver: "+t.getMessage());
            t.printStackTrace(System.err);
        }
    }
    @Test
    public void testHelloWorld() {
        // Start from the homepage
        driver.get("http://localhost:9080/helloworld/");
        HomePage homePage = new HomePage(driver);
        HelloWorldPage helloWorldPage = homePage.clickMessageLink();
        assertEquals("Hello world",helloWorldPage.getMessage());
    }
}

Lines 4-18 simply start Web Driver before the test and shut it down (closing the browser window) once the test is finished.

On line 22 we navigate to the homepage with a hard-coded URL.

On line 23 we initialise our Web Driver page object for the homepage. This encapsulates all the details of how the page works, allowing the test to interact with the page functionally, without worrying about the mechanics (which elements to use etc).

On line 24 we use the homepage object to click the “message” link; this navigates to the hello world page.

On line 25 we confirm that the message shown on the hello world page is what we expect.

Note: I’m using page objects to separate test specification (what to do) from test implementation (how to do it). For more on why this is important see keeping tests from being brittle.

Homepage

The homepage object is pretty simple:

public HelloWorldPage clickMessageLink() {
    driver.findElement(By.id("messageLink")).click();
    return new HelloWorldPage(driver);
}

HelloWorldPage

The hello world page is equally simple:

public String getMessage() {
    return driver.findElement(By.id("message")).getText();
}

Running the Integration Test

To run the integration test during our Maven build we need to make a few changes. First, we need to exclude integration tests from the unit test phase:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    ...
    <configuration>
        ...
        <excludes>
            <exclude>**/*IntegrationTest.java</exclude>
            <exclude>**/common/*</exclude>
        </excludes>
    </configuration>
</plugin>

Then we define a new profile, so we can optionally run integration tests:

<profile>
    <id>with-integration-tests</id>
    <build>
        <plugins>
            <plugin>
                <groupId>org.mortbay.jetty</groupId>
                <artifactId>maven-jetty-plugin</artifactId>
                <version>6.1.22</version>
                <configuration>
                    <scanIntervalSeconds>5</scanIntervalSeconds>
                    <stopPort>9966</stopPort>
                    <stopKey>foo</stopKey>
                    <connectors>
                        <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
                            <port>9080</port>
                            <maxIdleTime>60000</maxIdleTime>
                        </connector>
                    </connectors>
                </configuration>
                <executions>
                    <execution>
                        <id>start-jetty</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>run</goal>
                        </goals>
                        <configuration>
                            <daemon>true</daemon>
                        </configuration>
                    </execution>
                    <execution>
                        <id>stop-jetty</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>stop</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.5</version>
                <inherited>true</inherited>
                <executions>
                    <execution>
                        <id>integration-tests</id>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>test</goal>
                        </goals>
                        <configuration>
                            <excludes>
                                <exclude>**/common/*</exclude>
                            </excludes>
                            <includes>
                                <include>**/*IntegrationTest.java</include>
                            </includes>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</profile>
<profile>
<id>with-integration-tests</id>
<build>
<plugins>
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>maven-jetty-plugin</artifactId>
<version>6.1.22</version>
<configuration>
<scanIntervalSeconds>5</scanIntervalSeconds>
<stopPort>9966</stopPort>
<stopKey>foo</stopKey>
<connectors>
<connector implementation=”org.mortbay.jetty.nio.SelectChannelConnector”>
<port>${test.server.port}</port>
<maxIdleTime>60000</maxIdleTime>
</connector>
</connectors>
</configuration>
<executions>
<execution>
<id>start-jetty</id>
<phase>pre-integration-test</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<daemon>true</daemon>
</configuration>
</execution>
<execution>
<id>stop-jetty</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.5</version>
<inherited>true</inherited>
<executions>
<execution>
<id>integration-tests</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<excludes>
<exclude>**/common/*</exclude>
</excludes>
<includes>
<include>**/*IntegrationTest.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>

This may look complex, but really we’re just configuring jetty to run while we run our integration tests; then configuring how to run the integration tests themselves.

In lines 9-19 configure jetty – the port to run on and how to stop it.

Lines 21-30 configure jetty to run during the “pre-integration-test” phase of the maven build.

Lines 31-37 configure jetty to be stopped during the “post-integration-test” phase of the maven build.

In lines 40-62 we use the maven-surefire-plugin again, this time to run during the “integration-test” phase of the build, only running our integration test classes.

We can run this build with:

mvn clean install -Pwith-emma -Pwith-integration-tests

This will build everything, run the unit tests, build the war, fire up jetty to host the war, run our integration tests (you’ll see a firefox window popup while the rest runs) then shut down jetty. Because the war is built with instrumented classes, Emma also tracks code coverage while we run our integration tests.

We can now build our application, running unit tests and integration tests, gathering combined code coverage reports. If we re-run the emma report and check code coverage we now see we have 100% test coverage – since the controller is also being covered through tests.

Issues

What are the outstanding issues with this, what further extensions can be made?

  • The build produces an instrumented WAR – this means you need to run a second build, without emma, to get a production-ready build.
  • The integration test hard-codes the port that Jetty is configured to start on; meaning the tests can’t be run directly within Eclipse. It is possible to pass this port in, defaulting to say, 8080, so that integration tests can be run seemlessly within Eclipse as well via the maven build
  • When running on your build server you probably don’t want Firefox popping up at random (if X is even installed); so running xvfb is a good idea. It is possible to setup maven to start & stop xvfb before & after the integration tests.

Read Full Post »

Note: this article is now out of date, please see the more recent version covering the same topic: Testing asynchronous applications with WebDriverWait.

WebDriver is a great framework for automated testing of web applications. It learns the lessons of frameworks like selenium and provides a clean, clear API to test applications. However, testing ajax applications presents challenges to any test framework. How does WebDriver help us? First, some background…

Drivers

WebDriver comes with a number of drivers. Each of these is tuned to drive a specific browser (IE, Firefox, Chrome) using different technology depending on the browser. This allows the driver to operate in a manner that suits the browser, while keeping a consistent API so that test code doesn’t need to know which type of driver/browser is being used.

Page Pattern

The page pattern provides a great way to separate test implementation (how to drive the page) from test specification (the logical actions we want to complete – e.g. enter data in a form, navigate to another page etc). This makes the intent of tests clear:

homePage.setUsername("mytestuser");
homePage.setPassword("password");
homePage.clickLoginButton();

Blocking Calls

WebDriver’s calls are also blocking – calls to submit(), for example, wait for the form to be submitted and a response returned to the browser. This means we don’t need to do anything special to wait for the next page to load:

WebDriver driver = new FirefoxDriver();
driver.get("http://www.google.com/");
WebElement e = driver.findElement(By.name("q"));
e.sendKeys("webdriver");
e.submit();
assertEquals("webdriver - Google Search",driver.getTitle());

Asynchronous Calls

The trouble arises if you have an asynchronous application. Because WebDriver doesn’t block when you make asynchronous calls via javascript (why would it?), how do you know when something is “ready” to test? So if you have a link that, when clicked, does some ajax magic in the background – how do you know when the magic has stopped and you can start verifying that the right thing happened?

Naive Solution

The simplest solution is by using Thread.sleep(…). Normally if you sleep for a bit, by the time the thread wakes up, the javascript will have completed. This tends to work, for the most part.

The problem becomes that as you end up with hundreds of these tests, you suddenly start to find that they’re failing, at random, because the delay isn’t quite enough. When the build runs on your build server, sometimes the load is higher, the phase of the moon wrong, whatever – the end result is that your delay isn’t quite enough and you start assert()ing before your ajax call has completed and your test fails.

So, you start increasing the timeout. From 1 second. To 2 seconds. To 5 seconds. You’re now on a slippery slope of trying to tune how long the tests take to run against how successful they are. This is a crap tradeoff. You want very fast tests that always pass. Not semi fast tests that sometimes pass.

What’s to be done? Here are two techniques that make testing asynchronous applications easier.

RenderedWebElement

All the drivers (except HtmlUnit, which isn’t really driving a browser) actually generate RenderedWebElement instances, not just WebElement instances. RenderedWebElement has a few interesting methods on it that can make testing your application easier. For example, the isDisplayed() method saves you having to query the CSS style to work out whether an element is actually shown.

If you have some ajax magic that, in its final step, makes a <DIV> visible then you can use isDisplayed() to check whether the asynchronous call has completed yet.

Note: my examples here use dojo but the same technique can be used whether you’re using jquery or any other asynchronous toolkit.

First the HTML page – this simply has a link and a (initially hidden) <div>. When the link is clicked it triggers an asynchronous call to load a new HTML page; the content of this HTML page is inserted as the content of the div and the div made visible.

<html>
<head>
 <title>test</title>

 <script type="text/javascript" src="js/dojo/dojo.js" djConfig=" isDebug:false, parseOnLoad:true"></script>

 <script src="js/asyncError.js" type="text/javascript"></script>

 <script>
   /*
    * Load a HTML page asynchronously and
    * display contents in hidden div
    */
   function loadAsyncContent() {
     var xhrArgs = {
       url: "asyncContent.htm",
       handleAs: "text",
       load: function(data) {
         // Update DIV with content we loaded
         dojo.byId("asyncContent").innerHTML = data;

         // Make our DIV visible
         dojo.byId("asyncContent").style.display = 'block';
       }
     };

     // Call the asynchronous xhrGet
     var deferred = dojo.xhrGet(xhrArgs);
   }
 </script>
</head>
<body>
 <div id="asyncContent" style="display: none;"></div>
 <a href="#" id="loadAsyncContent" onClick="loadAsyncContent();">Click to load async content</a>
 <br/>
</body>
</html>

Now the integration test. Our test simply loads the page, clicks the link and waits for the asynchronous call to complete (highlighted). Once the call is complete, we check that the contents of the <div> is what we expect.

@RunWith(SpringJUnit4ClassRunner.class)
public class TestIntegrationTest {
 @Test
 public void testLoadAsyncContent() {
   // Create the page
   TestPage page = new TestPage( new FirefoxDriver() );

   // Click the link
   page.clickLoadAsyncContent();
   page.waitForAsyncContent();

   // Confirm content is loaded
   assertEquals("This content is loaded asynchronously.",page.getAsyncContent());
 }
}

Now the page class, used by the integration test. This interacts with the HTML elements exposed by the driver; the waitForAsyncContent method regularly polls the <div> to check whether its been made visible yet (highlighted)

public class TestPage  {

 private WebDriver driver;

 public TestPage( WebDriver driver ) {
   this.driver = driver;

   // Load our page
   driver.get("http://localhost:8080/test-app/test.htm");
 }

 public void clickLoadAsyncContent() {
   driver.findElement(By.id("loadAsyncContent")).click();
 }

 public void waitForAsyncContent() {
   // Get a RenderedWebElement corresponding to our div
   RenderedWebElement e = (RenderedWebElement) driver.findElement(By.id("asyncContent"));

   // Up to 10 times
   for( int i=0; i<10; i++ ) {
     // Check whether our element is visible yet
     if( e.isDisplayed() ) {
       return;
     }

     try {
       Thread.sleep(1000);
     } catch( InterruptedException ex ) {
       // Try again
     }
   }
 ]

 public String getAsyncContent() {
   return driver.findElement(By.id("asyncContent")).getText();
 }
}

By doing this, we don’t need to code arbitrary delays into our test; we can cope with server calls that potentially take a little while to execute; and can ensure that our test will always pass (at least, tests won’t fail because of timing problems!)

JavascriptExecutor

Another trick is to realise that the WebDriver instances implement JavascriptExecutor. This interface can be used to execute arbitrary Javascript within the context of the browser (a trick selenium finds much easier). This allows us to use the state of javascript variables to control the test. For example – we can have a variable populated once some asynchronous action has completed; this can be the trigger for the test to continue.

First, we add some Javascript to our example page above. Much as before, it simply loads a new page and updates the <div> with this content. This time, rather than show the div, it simply sets a variable – “working” – so the div can be visible already.

 var working = false;

 function updateAsyncContent() {
   working = true;

   var xhrArgs = {
     url: "asyncContentUpdated.htm",
     handleAs: "text",
     load: function(data) {
       // Update our div with our new content
       dojo.byId("asyncContent").innerHTML = data;

       // Set the variable to indicate we're done
       working = false;
     }
   };

   // Call the asynchronous xhrGet
   var deferred = dojo.xhrGet(xhrArgs);
 }

Now the extra test case we add. The key line here is where we wait for the “working” variable to be false (highlighted):

 @Test
 public void testUpdateAsyncContent() {
   // Create the page
   TestPage page = new TestPage( getDriver() );

   // Click the link
   page.clickLoadAsyncContent();
   page.waitForAsyncContent();

   // Now update the content
   page.clickUpdateAsyncContent();
   page.waitForNotWorking();

   // Confirm content is loaded
   assertEquals("This is the updated asynchronously loaded content.",page.getAsyncContent());
 }

Finally our updated page class, the key line here is where we execute some Javascript to determine the value of the working variable (highlighted):

 public void clickUpdateAsyncContent() {
   driver.findElement(By.id("updateAsyncContent")).click();
 }

 public void waitForNotWorking() {
   // Get a JavascriptExecutor
   JavascriptExecutor exec = (JavascriptExecutor) driver;

   // 10 times, or until element is visible
   for( int i=0; i<10; i++ ) {

     if( ! (Boolean) exec.executeScript("return working") ) {
       return;
     }

     try {
       Thread.sleep(1000);
     } catch( InterruptedException ex ) {
       // Try again
     }
   }
 }

Note how WebDriver handles type conversions for us here. In this example the Javascript is relatively trivial, but we could execute any arbitrarily complex Javascript here.

By doing this, we can now flag in Javascript, when some state has been reached that allows the test to progress; we have made our code more testable, and we have eliminated arbitrary delays and our test code runs faster by being able to be more responsive to asynchronous calls completing.

Have you ever had problems testing asynchronous applications? What approaches have you used?

Read Full Post »

%d bloggers like this: