How much architecture is enough?

Software architecture is hard. Creating a simple, consistent, flexible environment in which we can solve the customer’s ever-changing problems is no easy task. Keeping it that way is harder still. Striking the right balance between all the competing demands takes real skill – so what does it take to create a good architecture? How much architecture is enough?

Software Architecture

First, I’m drawing a distinction between software architecture and enterprise architecture. By software architecture I mean the largest patterns and structures in the code you write – the highest level of design detail. I do not mean what is often called enterprise architecture: what messaging middleware to use, how are services clustered, what database platforms to support. Software architecture is the stuff we write that forms the building blocks of our solution.

The Over Architect

I’m sure we’ve all worked with him: the guy who could over think hello world. When faced with a customer requirement his immediate response is:

we need to build a framework

Obviously the customer’s problem is too simple for this genius. Instead, we can solve this whole class of problems. Once we’ve built the framework, we just need to plug the right values into the simple 400 line XML configuration file and hey presto! customer problem solved.

Sure, we’ve only been asked to do a one-time CSV import of some customer data. But think of the long-term, what will they ask for next? What about the next customer? We should write a generic data import framework that could take data in CSV, XML or JSON; communicating over HTTP, FTP or email. We can build rich, configurable validation logic. We can write the data to any number of database platforms using a variety of standard ORM frameworks. Man, this is awesome, we could be busy for months with this!

Whatever. You Ain’t Gonna Need It!

But sometimes, the lure of solving a problem where you’re the customer, is much more intellectually stimulating than solving the boring old customer’s problems. You know, the guy who’s paying the bills.

The Over Architect generalises from a sample size of one. Every problem is an opportunity to build a more general solution, despite having no evidence for what other cases might need to be solved. Every problem is an opportunity to bring in the latest and greatest technology – whether or not its a good fit, whether or not the company’s going to be left supporting some byzantine third party library that’s over kill for their simple use. An architect fully versed in CV++

The Under Architect

On the other hand, the Under Architect looks at every customer problem and thinks:

we could re-use what we did for feature X

Where by “re-use” he means copy & paste, change as necessary. There’s no real architecture, just patterns repeated ad infinitum. Every new requirement is an opportunity to write more, new code. Heaven forbid we go back and change any of that crufty old shit. No, we’ll just build shiny, brand new legacy code.

We’re building a web application: so we’ll need some Controllers, some Views and some Models. There we go, MVC – that counts as an architecture, right? Oh, we need a bit more. Well, we’ve got some DAOs here for interacting with the database. And the business logic? Well, the stuff that’s not wrapped up in the controllers we can put in FooManager classes. Sure, these things look like monolithic god classes – but its the best way to aggregate all the related functionality together.

Lather, rinse, repeat and before you know it you have a massive application with minimal structure. The trouble is, these patterns become self-perpetuating. It’s hard to start pulling out an architecture when all you have is a naming convention.

The Many Architects

The challenge in many software teams is everyone thinks it’s their job to come up with new architecture or start building a new framework. The code ends up littered with half-finished, half-forgotten frameworks. Changing anything becomes a nightmare: was all this functionality used? We have three different ways of importing data, via three different hand-rolled frameworks – which ones are used? How much of each one is used? Can I refactor them down into one? Two? What about the incompatibilities and subtle differences?

Without a clear vision changing the code becomes like archeology. As you delve down through the layers you uncover increasingly crufty old code that nobody dares touch any more. It becomes less of a software architecture and more of a taxonomy problem – like Darwin trying to identify a million different species by their class structure.

The Answer

What’s the answer? Well I’m sorry, but I just don’t buy this agile bullshit about “emergent architecture”. Architecture doesn’t emerge, it has to be imposed, often onto unwilling code.

Architecture requires a vision: somebody needs to have a clear idea about where the software is headed. Architecture needs patience: as we learn more about the problem and the solution, the architecture will have to adapt. Architecture needs consistency: if the guy calling the shots changes every year or two, you’ll be back to the Many Architects problem.

Above all, I think good architecture needs a dictator. Some, single person – taking responsibility for the architecture. They don’t need to be right, they just need to have a clear vision of where the architecture should head. If the team are on board with that vision then the whole team are pulling in the same direction, guided by one individual taking the long view.

Central Architecture Group

This sounds like I’m advocating a central architecture group? Hell no. The architect needs to be involved in the code, hands-on, day-to-day, so he can see the consequences of his decisions. He needs the feedback from how the product evolves and how our understanding of the problem evolves. The last thing you need is a group of ivory tower architects pontificating about whether an Enterprise Service Bus is going to solve all our problems. Hint: it won’t, but firing the central architecture group might.

Conclusion

Getting software architecture right is a hard problem. If you keep your code DRY and SOLID, you’re heading in the right direction. If someone has the vision for where the code should head and the team work towards that, relentlessly cleaning up old code – then maybe, just maybe you’ve got a chance.

 

Implementing a rich domain model with Guice

The anaemic domain model is a really common anti-pattern. In the world of ORM & DI frameworks we naturally seem to find ourselves with an ORM-managed “domain” that is all data and no behaviour; coupled with helper classes that are all behaviour and no data, helpfully injected in by our DI framework. In this article I’ll look at one possible approach for implementing a rich domain model – this example uses Guice, although I’m sure Spring etc would have different ways of achieving the same thing.

The problem

All the source code can be found on github. The “master” branch shows the original, badly factored code. The “rich-domain” branch shows the solution I describe.

Anaemic domain model

First, our anaemic domain model – TradeOrder.java. This class, as is traditional with Hibernate, has a load of annotations describing the data model, fields for all the data, accessors and mutators to get at the data and nothing else of any interest. I assume, in this domain, that TradeOrders do things. Maybe we place the order or cancel the order. Somewhere along the line, the key objects in our domain should probably have some behaviour.

@Entity
@Table(name="TRADE_ORDER")
public class TradeOrder {
    @Id
    @Column(name="ID", length=32)
    @GeneratedValue
    private String id;

    @ManyToOne
    @JoinColumn(name="CURRENCY_ID", nullable=false)
    @ForeignKey(name="FK_ORDER_CURRENCY")
    @AccessType("field")
    private Currency currency;

    @Column(name="AMOUNT", nullable=true)
    private BigDecimal amount;

    public TradeOrder() { }

    public String getId() { return id; }

    public Currency getCurrency() { return currency; }
    public void setCurrency(Currency currency) { this.currency = currency; }

    public BigDecimal getAmount() { return amount; }
    public void setAmount(BigDecimal amount) { this.amount = amount; }
}

Helper class

In this really simple example, we need to figure out the value of the order – i.e. the number of shares we want to buy (or sell) and the price per share we’re paying. Unfortunately, because this involves dependencies the behaviour doesn’t reside in the class it relates to, instead its been moved into an oh-so-helpful helper class.

Take a look at FiguresFactory.java. This class only has one public method – buildFrom. The goal of this method is to create a Figures from a TradeOrder.

    public Figures buildFrom(TradeOrder order, Date effectiveDate) throws OrderProcessingException {
        Date tradeDate = order.getTradeDate();
        HedgeFundAsset asset = order.getAsset();

        BigDecimal bestPrice = bestPriceFor(asset, tradeDate);

        return order.getType() == TradeOrderType.REDEMPTION
            ? figuresFromPosition(
                  order,
                  lookupPosition(asset, order.getFohf(), tradeDate),
                  lookupPosition(asset, order.getFohf(), effectiveDate),
                  bestPrice)
            : getFigures(order, bestPrice, null);
    }

Besides the “effective date” (whatever that might be), the only input this method takes is the TradeOrder. Using the copious number of getters on TradeOrder it asks for data to operate on, instead of telling the TradeOrder what it needs. In an ideal, object-oriented system, this would have been a method on TradeOrder called createFigures.

Why did we end up here? It’s all the dependency injection framework’s fault! Because the process of creating a Figures object requires us to resolve prices and currencies, we need to go and lookup this data – using injectable dependencies. Our anaemic domain can’t have dependencies injected, so instead we inject them into this little helper class.

What we end up with is a classic anaemic domain model. The TradeOrder has the data; while numerous helper classes, like FiguresFactory, contain the behaviour that operate on that data. It’s all very un-OO.

A better way

Data record

The first step is to create a simple value object to map rows from the database – I’ve called this TradeOrderRecord.java. This looks much like the original domain object, except I’ve removed the accessors & mutators to make it clear that this is a value object with no behaviour.

To make constructing these record objects easier, I’ve used karg, a library written by a colleague of mine – this requires us to declare the set of arguments we can use to construct the record with, and a constructor that accepts a list of arguments. This greatly simplifies construction and avoids us having a constructor which takes 27 strings (for example).

@Entity
@Table(name="TRADE_ORDER")
public class TradeOrderRecord {
    @Id
    @Column(name="ID", length=32)
    @GeneratedValue
    public String id;

    @Column(name="CURRENCY_ID")
    public String currencyId;

    @Column(name="AMOUNT", nullable=true)
    public BigDecimal amount;

    public static class Arguments {
    	public static final Keyword<String> CURRENCY_ID = newKeyword();
    	public static final Keyword<BigDecimal> AMOUNT = newKeyword();
    }

    protected TradeOrderRecord() { }

    public TradeOrderRecord(KeywordArguments arguments) {
    	this.currencyId = Arguments.CURRENCY_ID.from(arguments);
    	this.amount = Arguments.AMOUNT.from(arguments);
    }
}

The rich domain

Our goal is to make TradeOrder a rich domain object – this should have all the behaviour and data associated with the domain concept of a “TradeOrder”.

Data

The first thing TradeOrder will need is, internally, to store all the data associated with a TradeOrder (at least as a starting point, the unused fields hint that we might be able to simplify this further).

public class TradeOrder {
    private final String id;
    private final String currencyId;
    private final BigDecimal amount;

We make the data immutable. Immutable state is generally a good thing – and here it forces us to be clear that this is a fully populated TradeOrder, and since it has an id, it is always associated with a row in the database. By making TradeOrder immutable the obvious question is – how do I update an order? Well, there are numerous ways we could choose to do that – but that is a different story for a different time.

We also do not need accessors. Since we plan on putting all the behaviour that relates to TradeOrder on the TradeOrder class itself, other classes should not need to ask for information, they should only need to tell us what they want to achieve.

Note: there is one (now deprecated) accessor – that hints at a further behaviour that ought to be moved.

Dependencies

Besides the fields to store data, TradeOrder will also have fields representing injectable dependencies.

    private final CurrencyCache currencyCache;
    private final PriceFetcher bestPriceFetcher;
    private final PositionFetcher hedgeFundAssetPositionsFetcher;
    private final FXService fxService;

Some people will find this offensive – mixing dependencies with data. However, personally, I think the trade-off is worth it – the benefit of being able to define our behaviours on the class they relate to is worth it.

Behaviour

Now we have the data and the dependencies all in one place, it is relatively easy to move across the methods from FiguresFactory:

    public Figures createFigures(Date effectiveDate) throws OrderProcessingException {
        BigDecimal bestPrice = bestPriceFor(this.asset, this.tradeDate);

        return this.type == TradeOrderType.REDEMPTION
            ? figuresFromPosition(
                  fohf,
                  lookupPosition(this.asset, fohf, this.tradeDate),
                  lookupPosition(this.asset, fohf, effectiveDate), bestPrice)
            : getFigures(fohf, bestPrice, null);
    }

Construction

The last thing we need to tackle is how to create instances of TradeOrder. Since the fields for data and dependencies are all marked as final, the constructor must initialise them all. This means we need a constructor that takes the dependencies and a TradeOrderRecord (i.e. the value object we read from the database):

    @Inject
    protected TradeOrder(CurrencyCache currencyCache,
                         PriceFetcher bestPriceFetcher,
                         PositionFetcher hedgeFundAssetPositionsFetcher,
                         FXService fxService,
                         @Assisted TradeOrderRecord record) {
        ...
    }

This isn’t particularly pretty, but the key thing to note is the @Assisted annotation. This allows us to tell Guice that the other dependencies are injected normally, whereas the TradeOrderRecord should be passed through from a factory method. The factory interface itself looks like this:

    public static interface Factory {
    	TradeOrder create(TradeOrderRecord record);
    }

We don’t need to implement this interface, Guice provides it automatically. TradeOrder.Factory becomes an injectable dependency we can use from elsewhere when we need to create an instance of TradeOrder. Guice will initialise the injectable dependencies as normal, and the assisted dependency – TradeOrderRecord – is passed through from the factory. So our calling code doesn’t need to worry that our rich domain needs injectable dependencies.

    @Inject private TradeOrder.Factory tradeOrderFactory;
    ...
    TradeOrderRecord record = tradeOrderDAO.loadById(id);
    TradeOrder order = tradeOrderFactory.create(record);

Conclusion

By mixing dependencies and data together into a rich domain model we are able to define a class with the right behaviours. The obvious code smell in TradeOrder now is that the detailed mechanics of creating a Figures is probably a separate concern and should be broken out. That’s ok, we can introduce a new dependency to manage that – as long as the TradeOrder is still the starting point for constructing the Figures object.

By having the behaviours in a single place our domain model is easier to work with, easier to reason about and it’s easier to spot cases of duplication or similarity. We can then refactor sensibly, using a good object model, instead of introducing arbitrary distinctions into helper classes that are function libraries, not participants in the domain.

 

Shame driven development

I always aspire to write well-crafted code. During my day job, where all production code is paired on, I think our quality is pretty high. But it’s amazing how easy you forgive yourself and slip into bad habits while coding alone. Is shame the driving force behind quality while pairing?

We have a number of archaic unit tests written using Easy Mock; all our more recent unit tests use JMock. This little piece of technical debt means that if you’re changing code where the only coverage is Easy Mock tests you first have to decide: do you fix up the tests or, can you hold your nose and live with / tweak the existing test for your purposes? This is not only distracting, but it means doing the right thing can be substantially slower.

Changing our Easy Mock tests to JMock is, in principle, a relatively simple task. Easy Mock declares mocks in a simple way:

private PricesService prices = createMock(PricesService.class);

These can easily be converted into JMock-style:

private Mockery context = new Mockery();
...
private final PricesService prices = context.mock(PricesService.class);

EasyMock has a slightly different way of declaring expectations:

prices.prefetchFor(asset);
expect(prices.for(asset)).andReturn(
    Lists.newListOf("1.45", "34.74"));

These need to be translated to JMock expectations:

context.checking(new Expectations() {{
    allowing(prices).prefetchFor(asset);
    allowing(prices).for(asset);
        will(returnValue(Lists.newListOf("1.45", "34.74")));
}});

This process is pretty mechanical, so as part of 10% time I started using my scripted refactoring tool – Rescripter – to mechanically translate our EasyMock tests into JMock. Rescripter let’s you run code that modifies your Java source. But this is more than just simple search & replace or regular expressions: by using Eclipse’s powerful syntax tree parsing, you have access to a fully parsed representation of your source file – meaning you can find references to methods, locate method calls, names, parameter lists etc. This is exactly what you need given the nature of the translation from one library to another.

This was inevitably fairly exploratory coding. I wasn’t really sure what would be possible and how complex the translation process would eventually become. But I started out with some simple examples, like those above. But, over time, the complexity grew as the many differences between the libraries made me work harder and harder to complete the translation.

After a couple of 10% days on this I’d managed to cobble together something awesome: I’d translated a couple of hundred unit tests; but, this was done by 700 lines of the most grotesque code you’ve ever had the misfortune to lay your eyes upon!

And then… and then last week, I got a pair partner for the day. He had to share this horror. Having spent 10 minutes explaining the problem to him and then 15 minutes explaining why it was throwaway, one-use code so didn’t have any unit tests. I was embarrassed.

We started trying to make some small changes; but without a test framework, it was difficult to be sure what we were doing would work. To make matters worse, we needed to change core functions used in numerous places. This made me nervous, because there was no test coverage – so we couldn’t be certain we wouldn’t break what was already there.

Frankly, this was an absolute nightmare. I’m so used to having test coverage and writing tests – the thought of writing code without unit tests brings me out in cold sweats. But, here I was, with a mess of untested code entirely of my own creation. Why? Because I’d forgiven myself for not “doing it right”. After all, it’s only throwaway code, isn’t it? It’s exploratory, more of a spike than production code. Anyway, once its done and the tests migrated this code will be useless – so why make it pretty? I’ll just carry on hacking away…

It’s amazing how reasonable it all sounds. Until you realise you’re being a total and utter fucktard. Even if it’s one-use code, even if it has a relatively short shelf-life 

the only way to go fast, is to go well

So I did what any reasonable human being would do. I spent my lunch hour fixing this state of affairs. The end result? I could now write unit tests in Jasmine to verify the refactoring I was writing.

Not only could I now properly test drive new code. I could write tests to cover my existing legacy code, so I could refactor it properly. Amazing. And all of a sudden, the pace of progress jumped. Instead of long debug cycles and trying to manually find and trigger test scenarios, I had an easy to run, repeatable, automated test suite that gave me confidence in what I was doing.

None of this is new to me: it’s what I do day-in day-out. And yet… and yet… somehow I’d forgiven myself while coding alone. The only conclusion I can draw is that we can’t be trusted to write code of any value alone. The shame of letting another human being see your sorry excuse for code is what drives up quality when pairing: if you’re not pair programming, the code you’re writing must be shameful.

Introduction to Rescripter

Are you a Java developer? Do you use Eclipse? Ever find yourself making the same mindless change over and over again, wishing you could automate it? ME TOO. So I wrote an Eclipse plugin that let’s you write scripts that refactor your source code.

It does what now?

Some changes are easy to describe, but laborious to do. Perhaps some examples would help:

  • Renaming a method and renaming every call to that method (ok, Eclipse has built-in support for this)
  • Replacing a call to a constructor with a static factory method (ok, IntelliJ has built-in support for this, but Eclipse doesn’t)
  • Moving a method and updating callers to get a reference to the target class
  • Replacing use of one library with a similar one with a different API

Basically anything that involves the same, small change made multiple times across your source code.

How does it work?

After installing the Rescripter Eclipse plugin write some JavaScript to describe your change; when you run this, the script can make changes to your source code using Eclipse’s built-in refactoring and source code modification support.

Why Javascript? Because its a well understood language which, via Rhino, integrates excellently with Java.

An example would help about now

Ok, so imagine I have my own number class.

public class MyNumber {
	private int value;

	public MyNumber(String value) {
		this.value = Integer.parseInt(value);
	}

	public static MyNumber valueOf(String value) {
		return valueOf(value);
	}
}

My source code becomes littered with calls to the constructor:

MyNumber myNumber = new MyNumber("42");

As part of some refactoring, if I want to change how these numbers are created or implemented I may want to replace all calls to the constructor with calls to the static factory method. For example, this would let me introduce a class hierarchy that the factory method knows about, or any number of other changes that cannot be done if client code calls the constructor directly.

Unfortunately, Eclipse doesn’t provide a way to do this refactoring – however, Rescripter let’s you do it. We can describe what we want to do quite simply:

Find all references to the constructor, then for each reference:

  • Add a static import to MyNumber.valueOf
  • Replace the method call (new MyNumber) with valueOf
So what does this look like?
var matches = Search.forReferencesToMethod("com.example.MyNumber(String)");

var edit = new MultiSourceChange();
foreach(filter(matches, Search.onlySourceMatches),
    function(match) {
        edit.changeFile(match.getElement().getCompilationUnit())
            .addImport("static com.example.MyNumber.valueOf")
            .addEdit(Refactor.createReplaceMethodCallEdit(
                 match.getElement().getCompilationUnit(),
                 match.getOffset(), match.getLength(),
                 "valueOf"));
    });
edit.apply();

The first line finds references to the MyNumber(String) constructor.

Line 4 introduces a loop over each reference, filtering to only matches in source files (we can’t change .class files, only .java files).

Line 7 adds our static import.

Lines 8-11 replace the constructor call with the text “valueOf”

Our original call now looks like:

MyNumber myNumber = valueOf("42");

Now having replaced all uses of the constructor I can make it private and make whatever changes I need to the static factory method.

That’s great, but can it…?

Yes, probably. But maybe not yet. Rescripter is very basic at the minute, although you can still run some pretty powerful scripts making broad changes to your source code. If you’re having problems, have a feature suggestion or bug report – either post a comment or drop me a mail.

Page Objects in Selenium 2.0

Want to learn more about WebDriver? What do you want to know?

So you’ve written your first Selenium 2.0 test, but is that really the right way to build tests? In the second article in this series we’ll look at the difference between test specification and test implementation and how Selenium achieves this with page objects.

Specification vs Implementation

There’s an important difference in tests between the specification of what to test versus the implementation of how to test.

For example, my test specification might be “When the user enters their username and password and clicks the login button, then they are logged in and can see their recommendations”. This describes a scenario – it’s a specification of what the test should do. However, the test implementation has to deal with things like:

  • The username field is named “f_username”
  • The password field is named “f_password”
  • The login button is found via the CSS “#loginButton input”

If I change the layout of my login page, does the specification change? Hell no – I still need to provide credentials and click the login button. But has my implementation changed? Almost certainly!

Separating test specification from test implementation makes tests more robust. If I change how login works, I don’t want to have to change every single test that needs a logged in user – there’s probably a few of them! Instead, I only want to change the implementation – which fields to use, which buttons to press – in a single, common location.

Using Page Objects

In Selenium, you can separate specification from implementation by using page objects. That is, unlike in our previous example where the test code is littered with implementation details (how to find the keyword field or how to find the submit button), instead we move this logic into a page object.

If we built a page object to represent the Amazon home page, what would it look like?

public class AmazonHomePage {
    public static AmazonHomePage navigateTo(WebDriver driver);
    public AmazonSearchResultsPage searchFor(String searchTerm);
}

The homepage has two responsibilities that our test cares about:

  1. Given a browser, navigate to the page
  2. Once on the page, search for a specific search term

Notice that the searchFor operation returns a different type of page object – because after entering the search we will be on a new page: the search results page. What does that page look like?

public class AmazonSearchResultsPage {
    public String getTopResultTitle();
}

We only need one thing from our search results page – the title of the top result.

Tests with Page Objects

The page object encapsulates all the logic about how to perform certain actions. Our test now only needs to deal with what to actually do – so what does our test look like now?

AmazonHomePage homePage = AmazonHomePage.navigateTo(driver);
AmazonSearchResultsPage resultsPage =
    homePage.searchFor("iain banks");
assertThat(resultsPage.getTopResultTitle(), is("Transition"));

This is a pretty neat specification for our test: given a user on the amazon home page, when the user searches for ‘iain banks’, then the title of the top result is ‘Transition’.

There’s no implementation details in our test now. If any of the implementation details around searching change – we only need to change our page object. This is particularly important when the page objects are reused by multiple tests. Instead of having to manually change dozens of tests, we make our change in a single location – the page object.

Implementing Page Objects

By removing the implementation details from the test, they now sit within the page object. So how could we implement the page object interfaces we defined above? We could simply reuse the logic we had in our tests previously:

public AmazonSearchResultsPage searchFor(String searchTerm) {
    // Enter a search term
    WebElement keywordsField =
        driver.findElement(By.name("field-keywords"));
    keywordsField.sendKeys(searchTerm);

    // Click go
    WebElement goButton =
        driver.findElement(By.cssSelector("#navGoButton input"));
    goButton.click();
    ...

However, Selenium gives us another way to do this – we can also define the web elements declaratively, by using annotations. Although either approach is valid, using annotations tends to be neater. So let’s have a look at the top of the homepage class now:

public class AmazonHomePage {

	@FindBy(name="field-keywords")
	private WebElement keywordsField;

	@FindBy(css="#navGoButton input")
	private WebElement goButton;

Here we define the same two WebElements we had in our test before. But rather than call the web driver to find the elements for us explicitly, they’re now pre-populated based on the annotation.

To implement the interface we described above, we simply need to invoke the sendKeys and click methods on these elements as we did previously:

public AmazonSearchResultsPage searchFor(String searchTerm) {
    keywordsField.sendKeys(searchTerm);
    goButton.click();
    return PageFactory.initElements(driver,
        AmazonSearchResultsPage.class);
}

On lines two and three we type in our search term to the field we declared above and then click the go button. But what on earth is happening on lines four and five? This piece of magic – the PageFactory – is what allows us to use annotations to declare our elements.

Page Factory

The page factory instantiates our page object (AmazonSearchResultsPage) and finds the annotated fields. The fields are then initialised to the associated elements on the page. This is how we’re able to simply use the fields without having to initialise them. The returned search results page is then a fully populated page object, ready to be used by the test as before.

There’s one more method to see on the homepage – the navigateTo method. This also uses the page factory – this time to initialise our AmazonHomePage.

	public static AmazonHomePage navigateTo(WebDriver driver) {
		driver.get("http://www.amazon.co.uk/");
		return PageFactory.initElements(driver,
             AmazonHomePage.class);
	}

And there we have it – a better amazon search example – now using page objects and annotations. This is a much better way to structure tests than having implementation and specification intermingled within the test.

As before, all the examples are available on github. This article was part 2 in a series, if there’s a specific aspect of using Selenium you’d like to see covered in a future article let us know in the comments below.