[tweetmeme source=”activelylazy” only_single=false]
Note: this article is now out of date, please see the more recent version covering the same topic: Testing asynchronous applications with WebDriverWait.
WebDriver is a great framework for automated testing of web applications. It learns the lessons of frameworks like selenium and provides a clean, clear API to test applications. However, testing ajax applications presents challenges to any test framework. How does WebDriver help us? First, some background…
Drivers
WebDriver comes with a number of drivers. Each of these is tuned to drive a specific browser (IE, Firefox, Chrome) using different technology depending on the browser. This allows the driver to operate in a manner that suits the browser, while keeping a consistent API so that test code doesn’t need to know which type of driver/browser is being used.
Page Pattern
The page pattern provides a great way to separate test implementation (how to drive the page) from test specification (the logical actions we want to complete – e.g. enter data in a form, navigate to another page etc). This makes the intent of tests clear:
homePage.setUsername("mytestuser"); homePage.setPassword("password"); homePage.clickLoginButton();
Blocking Calls
WebDriver’s calls are also blocking – calls to submit(), for example, wait for the form to be submitted and a response returned to the browser. This means we don’t need to do anything special to wait for the next page to load:
WebDriver driver = new FirefoxDriver(); driver.get("http://www.google.com/"); WebElement e = driver.findElement(By.name("q")); e.sendKeys("webdriver"); e.submit(); assertEquals("webdriver - Google Search",driver.getTitle());
Asynchronous Calls
The trouble arises if you have an asynchronous application. Because WebDriver doesn’t block when you make asynchronous calls via javascript (why would it?), how do you know when something is “ready” to test? So if you have a link that, when clicked, does some ajax magic in the background – how do you know when the magic has stopped and you can start verifying that the right thing happened?
Naive Solution
The simplest solution is by using Thread.sleep(…). Normally if you sleep for a bit, by the time the thread wakes up, the javascript will have completed. This tends to work, for the most part.
The problem becomes that as you end up with hundreds of these tests, you suddenly start to find that they’re failing, at random, because the delay isn’t quite enough. When the build runs on your build server, sometimes the load is higher, the phase of the moon wrong, whatever – the end result is that your delay isn’t quite enough and you start assert()ing before your ajax call has completed and your test fails.
So, you start increasing the timeout. From 1 second. To 2 seconds. To 5 seconds. You’re now on a slippery slope of trying to tune how long the tests take to run against how successful they are. This is a crap tradeoff. You want very fast tests that always pass. Not semi fast tests that sometimes pass.
What’s to be done? Here are two techniques that make testing asynchronous applications easier.
RenderedWebElement
All the drivers (except HtmlUnit, which isn’t really driving a browser) actually generate RenderedWebElement instances, not just WebElement instances. RenderedWebElement has a few interesting methods on it that can make testing your application easier. For example, the isDisplayed() method saves you having to query the CSS style to work out whether an element is actually shown.
If you have some ajax magic that, in its final step, makes a <DIV> visible then you can use isDisplayed() to check whether the asynchronous call has completed yet.
Note: my examples here use dojo but the same technique can be used whether you’re using jquery or any other asynchronous toolkit.
First the HTML page – this simply has a link and a (initially hidden) <div>. When the link is clicked it triggers an asynchronous call to load a new HTML page; the content of this HTML page is inserted as the content of the div and the div made visible.
<html> <head> <title>test</title> <script type="text/javascript" src="js/dojo/dojo.js" djConfig=" isDebug:false, parseOnLoad:true"></script> <script src="js/asyncError.js" type="text/javascript"></script> <script> /* * Load a HTML page asynchronously and * display contents in hidden div */ function loadAsyncContent() { var xhrArgs = { url: "asyncContent.htm", handleAs: "text", load: function(data) { // Update DIV with content we loaded dojo.byId("asyncContent").innerHTML = data; // Make our DIV visible dojo.byId("asyncContent").style.display = 'block'; } }; // Call the asynchronous xhrGet var deferred = dojo.xhrGet(xhrArgs); } </script> </head> <body> <div id="asyncContent" style="display: none;"></div> <a href="#" id="loadAsyncContent" onClick="loadAsyncContent();">Click to load async content</a> <br/> </body> </html>
Now the integration test. Our test simply loads the page, clicks the link and waits for the asynchronous call to complete (highlighted). Once the call is complete, we check that the contents of the <div> is what we expect.
@RunWith(SpringJUnit4ClassRunner.class) public class TestIntegrationTest { @Test public void testLoadAsyncContent() { // Create the page TestPage page = new TestPage( new FirefoxDriver() ); // Click the link page.clickLoadAsyncContent(); page.waitForAsyncContent(); // Confirm content is loaded assertEquals("This content is loaded asynchronously.",page.getAsyncContent()); } }
Now the page class, used by the integration test. This interacts with the HTML elements exposed by the driver; the waitForAsyncContent method regularly polls the <div> to check whether its been made visible yet (highlighted)
public class TestPage { private WebDriver driver; public TestPage( WebDriver driver ) { this.driver = driver; // Load our page driver.get("http://localhost:8080/test-app/test.htm"); } public void clickLoadAsyncContent() { driver.findElement(By.id("loadAsyncContent")).click(); } public void waitForAsyncContent() { // Get a RenderedWebElement corresponding to our div RenderedWebElement e = (RenderedWebElement) driver.findElement(By.id("asyncContent")); // Up to 10 times for( int i=0; i<10; i++ ) { // Check whether our element is visible yet if( e.isDisplayed() ) { return; } try { Thread.sleep(1000); } catch( InterruptedException ex ) { // Try again } } ] public String getAsyncContent() { return driver.findElement(By.id("asyncContent")).getText(); } }
By doing this, we don’t need to code arbitrary delays into our test; we can cope with server calls that potentially take a little while to execute; and can ensure that our test will always pass (at least, tests won’t fail because of timing problems!)
JavascriptExecutor
Another trick is to realise that the WebDriver instances implement JavascriptExecutor. This interface can be used to execute arbitrary Javascript within the context of the browser (a trick selenium finds much easier). This allows us to use the state of javascript variables to control the test. For example – we can have a variable populated once some asynchronous action has completed; this can be the trigger for the test to continue.
First, we add some Javascript to our example page above. Much as before, it simply loads a new page and updates the <div> with this content. This time, rather than show the div, it simply sets a variable – “working” – so the div can be visible already.
var working = false; function updateAsyncContent() { working = true; var xhrArgs = { url: "asyncContentUpdated.htm", handleAs: "text", load: function(data) { // Update our div with our new content dojo.byId("asyncContent").innerHTML = data; // Set the variable to indicate we're done working = false; } }; // Call the asynchronous xhrGet var deferred = dojo.xhrGet(xhrArgs); }
Now the extra test case we add. The key line here is where we wait for the “working” variable to be false (highlighted):
@Test public void testUpdateAsyncContent() { // Create the page TestPage page = new TestPage( getDriver() ); // Click the link page.clickLoadAsyncContent(); page.waitForAsyncContent(); // Now update the content page.clickUpdateAsyncContent(); page.waitForNotWorking(); // Confirm content is loaded assertEquals("This is the updated asynchronously loaded content.",page.getAsyncContent()); }
Finally our updated page class, the key line here is where we execute some Javascript to determine the value of the working variable (highlighted):
public void clickUpdateAsyncContent() { driver.findElement(By.id("updateAsyncContent")).click(); } public void waitForNotWorking() { // Get a JavascriptExecutor JavascriptExecutor exec = (JavascriptExecutor) driver; // 10 times, or until element is visible for( int i=0; i<10; i++ ) { if( ! (Boolean) exec.executeScript("return working") ) { return; } try { Thread.sleep(1000); } catch( InterruptedException ex ) { // Try again } } }
Note how WebDriver handles type conversions for us here. In this example the Javascript is relatively trivial, but we could execute any arbitrarily complex Javascript here.
By doing this, we can now flag in Javascript, when some state has been reached that allows the test to progress; we have made our code more testable, and we have eliminated arbitrary delays and our test code runs faster by being able to be more responsive to asynchronous calls completing.
Have you ever had problems testing asynchronous applications? What approaches have you used?
Hi, I encountered a problem when sending keys to input box, do you have any idea of how to resolve it?
I post the question on my site: http://ksblog.org/index.php?q=web-driver-sends-the-wrong-key-to-input-box
Awesome posting, Sir! To anyone who wants to wait until some element.isDisplayed(), I recommend having a look at class WebDriverWait which is also part of Selenium. And by the way: After RenderedWebElement diappeared, WebElement now has the method isDisplayed().
I agree with Robert that WebDriverWait is really useful and simple.
I have some related post in my blog, you can read them for reference.
Use WebDriverWait to wait for AJAX/Javascript asynchronized refresh
One Useful Class for WebDriver – A Package Class for WebDriverWait
I only just stumbled upon WebDriverWait today – previously I’ve been using sleep() but not quite in the brute-force sleep-for-x-seconds you describe:
void WaitUntil (By by)
{
…
IWebElement element = null;
do
{
try
{
element = _driver.FindElement(by);
if (element.Displayed)
break;
}
catch (NoSuchElementException e)
{
// wait a bit longer in case the element is created
}
count++;
Thread.Sleep(1000);
} while (count <= 5); // 5 is the approx max seconds to wait
// finally, it has to be visible now, or else!
if (element == null || !element.Displayed)
throw new ElementNotVisibleException("WaitUntil element must be visible");
}
Now, in chunks of 1 second, my tests will wait until the element appears which makes them as fast as necessary.
This doesn’t work as I get Exception during casting firefox/chrome driver to RenderedWebElement. I use xpath instead of id.
RenderedWebElement e = (RenderedWebElement) (driver.findElement(By.xpath(“//*[@id=’ctl00_CP_checkoutSections_ctl03_ucPaymentEdit_UCEditCreditCard_AddCreditCardUC_CardTypeContainer_DdlCardType’]”)));
I think RenderedWebElement doesn’t exist anymore. I think all WebElements have the required isDisplayed method now, so no need to cast.
Hi,
nice post. But what is actually different to the Thread.sleeps? All I see is that you iterate 10 times and after that you do a Thread.sleep.
Why is that a guarantee that the WebElement is displayed in this time?
I think WebDriverWait is the thing we are all looking for. It really waits for an element until it displays or the timeout is over.
Hi,
Thanks. Unfortunately this post was written back in the days before WebDriverWait. You’re quite right – that’s the correct way to do it. I’ve just written a new version of this post to bring things more up to date: https://blog.activelylazy.co.uk/2012/01/29/testing-asynchronous-applications-with-webdriverwait/
hi .. this is a very nice post, I guess we can use this to handle most Ajax elements , do you have any such content for GWT?
Not yet – definitely something to look at! Thanks!
Hi,
Using the Javascript example here…
Can we also say like this ;
public void waitForNotWorking() {
// Get a JavascriptExecutor
JavascriptExecutor exec = (JavascriptExecutor) driver;
// 10 times, or until element is visible
for( int i=0; i<10; i++ ) {
if( (Boolean) exec.executeScript("return document.getElementById("asyncContent");)){
return;
}
try {
Thread.sleep(1000);
} catch( InterruptedException ex ) {
// Try again
}
}
}
You could, but honestly now I think you’re better off using WebDriverWait: https://blog.activelylazy.co.uk/2012/01/29/testing-asynchronous-applications-with-webdriverwait/