Cross-functional teams

Cross-functional teams aren’t a new idea. And yet, somehow, we still don’t seem to have got the memo.

I was listening to the excellent Scott Hanselman’s podcast “Hanselminutes” last week, he had Angie Jones on to talk about automation. Among all the great advice around ensuring that automation is a first-class citizen in your development process one thing stood out for me

you need to get your automation engineers into your scrum team

I don’t know why, but it surprised me. Are there really companies out there up to speed enough to be doing test automation; yet so behind the times with agile that they think it’s a good idea to have a dedicated team of automation engineers, removed from the rest of the dev team?

Cross-functional teams are a pretty central idea to agile – breaking down silos and ensuring that everyone that is required to produce an increment of working software is aligned and working together. It’s certainly not a new idea, but it’s clearly an idea that we’re still struggling to absorb.

But then, looking back, I remember working for a certain large company that decided they needed to “do more test automation”. So they hired a room full of automation engineers, who sat two floors away from the dev team, in a room hidden away (we honestly didn’t know they were even there for weeks, maybe even months). This team were responsible for creating an automated test pack for the application (rather than use the one the test automation engineers in the scrum teams had been working on for the last few years). But… they weren’t even talking to the scrum teams. So they were constantly chasing to keep up. As you can imagine, hilarity ensued. I say hilarity. Arguments, really. Then anger. Eventually laughter as we realised all this effort was wasted because the scrum teams wouldn’t – in fact couldn’t – support this new automation code.

Clearly not getting the idea of cross-functional teams is an age old problem. Compare this to a more recent client of mine – one that had a genuinely more cross-functional team. Not only did the scrum team have its own automation engineer, the developers (actual developers, not re-branded testers) were encouraged to work on the test automation tools – to everyone’s benefit. Test tooling written to the same standards as production code, with the insight and experience of the test automation specialist. This is moving beyond cross-functional teams into cross-skilled teams. Not only is every skill set you need within the same scrum team, but each individual can have multiple skills, taking on multiple roles.

Sure, you still need specialists. But with generalizing specialists you get the best of both worlds: the experience of specialists in their area, with the flexibility and breadth of ideas that come from the whole team being able to work on whatever is required. When the entire team can swarm on any area you have a very flexible team, if we need a big push for test automation the entire team can focus on it. Similarly with plenty of pairing and rotation everyone on the team will see every area and every role, allowing everyone’s unique perspective to improve the product and the process.

But then, a counter-example, the same client suffered from another age-old silo: operations. I thought devops had killed this silo, but it seems not. If the scrum team can’t release an increment of software to actual users then it isn’t a fully self-contained, self-sufficient, cross-functional team. A scrum team working with a separate test automation team seems like a crazy idea; and yet, somehow, a scrum team working with a separate operations team is much more normal, much more accepted. But it’s the exact same problem: if you don’t have everyone you need in the same scrum team then you’re going to get bottlenecks. You’re going to get communication problems. You’re going to get a “them-vs-us” attitude.

Every time I’ve come up against this the typical argument against operations staff being embedded within scrum teams is that they’re not working on “your stuff” all the time, so the rest of the time they’d be busy doing other stuff, unrelated to “your team” or they’d be bored. Well, maybe if we freed up that extra capacity we could release more often? Maybe they could be working on making it quicker, easier, safer to release more often? Maybe they could be more deeply involved with development when we’re making decisions which affect what they’re going to release and how it’s going to wake them up in the middle of the night. Maybe they could even help with other, non-production environments? The neglected, little siblings of production that every company seems to struggle to pay enough attention to.

Maybe, even, over time the team evolves from having the operations specialist to having team members cross skilled into operations. Under the watchful eye of the specialist could we, shock horror, let testers touch production? Could the BA manage a release? In some industries this is completely impossible for regulatory reasons, but in all the others its “impossible” for merely arbitrary reasons.

Breaking down silos is never easy – but I think it’s an interesting reflection of how far we’ve come that some silos seem frankly ridiculous now, while others just seem old-fashioned. I still hold out for the distant dream of genuinely cross-functional teams. Whenever I’ve seen this actually happen the lack of bottlenecks and mis-communication makes everything so much faster, so much easier. In the end a cross-functional team is better than silos. But a cross skilled team is better still, if you can manage it.

Downsizing society

The world is becoming increasingly automated. Jobs that were once done by people are now frequently done by machines instead. This started off in manufacturing, but the coming of self-driving cars will put huge numbers of people out of work; even lawyers are being replaced by machines. Some reports suggest that machines could do 50% of jobs within the next 30 yearsFifty percent! What will we do when half the population have been made redundant? How will we cope with such a restructuring of society?

To an economist, a job is an income. To a human being, it is much more than that. It provides a sense that you matter in society, that people beyond your family rely on you and even admire you.

If 50% of jobs are being done by machines, the question is what will people do instead? This is an important question, because we identify as our job. Our jobs define us. Let me introduce you to Louise, she’s a doctor. And this is Barry, he’s an estate agent. I bet you have different ideas of who those two people are. What about when they’re both unemployed? Unemployable. Made redundant from society.

How would so many people survive without jobs? Where’s the money to live coming from? The only possible answer seems to be some kind of universal basic income. The idea that everyone, employed or not, receives a fixed amount of money each month from the government – replacing all existing benefits and all the bureaucracy involved in administering them. With a UBI, the 50% of people without jobs would at least have some money with which to buy food and heating. But who wants to live on handouts? Who wants to be defined as unemployed? Even if the majority of people are in the same situation.

This inevitably results in a two-tier society: those that earn little to nothing on top of the basic income; and those increasingly rare few that still have jobs the machines aren’t able to do, yet. We have the non-working class, and a diminishing middle class. How can this cause anything other than resentment, envy and anger?

This is to say nothing of the challenges of funding a universal basic income. While ideas like funding it through a wealth tax make a lot of sense, could they ever succeed? Would the wealthy half ever vote for it? Would the big businesses (and their very wealthy owners) that bankroll governments stand for it?

As Brexit and Trump have shown, voters can vote for what previously seemed impossible. And we’ve barely scratched the surface of the anger and disenfranchisement that automation will bring upon us. However, with both Brexit and Trump people voted out of hope. Hope for something better. For respect. For status. For jobs. Could Farage evoke such a passionate response from a platform of “more handouts”? Of massive wealth redistribution, the likes of which no socialist government has ever even proposed? It seems improbable.

If we don’t introduce a UBI, what’s going to happen? The jobs that are left aren’t going to be spread evenly. London will always have disproportionately more jobs and lower unemployment than, say, Sunderland or Hull. What’s going to happen in these areas as unemployment becomes endemic? Rising anger seems inevitable. Riots. A public whipped into a frenzy of hatred against some group of “others”?

With so much anger, extreme politics is only going to increase. Does a majority of the public already feel left behind? Made redundant by automation? So far this has given us Brexit and Trump. It’s only going to get worse. Who are we going to wage war on? Syria? ISIS? China? Germany? This is the armageddon outcome. So many people feel so disenfranchised that we inadvertently start world war three.

What other outcomes are there? Another possibility is some kind of neo-luddite movement railing against technology. So far everyone’s anger at disappearing jobs has been directed at migrants. When people realise its actually technology taking their jobs, could that anger end up directed at technology instead? You can imagine someone like Farage standing against technology. Of banning automation. Of holding back progress to protect jobs. While this couldn’t hold back the tide forever, it could delay the inevitable march of technology for some years.

The flip-side of this neo-luddite revolution, is the inevitable flight of technology outside of such a regime. With the future being held back in one place, technologists will move somewhere else; to somewhere innovation is encouraged instead of criminalised. Some enterprising country, say Switzerland, would reduce taxes for technology companies to encourage them to relocate. The remaining jobs, and the people to do them, all move to an employment enclave. This exacerbates the split in society: between the wealthy employed few, and the many under-employed poor. This is the “Switzerland outcome”. A physical as well as economic split in society.

Of course, there is always the possibility that the jobs that disappear are replaced by new jobs. Jobs in manual labour become replaced with jobs like “social media consultant” that would have been unimaginable in Victorian England (some would say still are). But the signs aren’t good: in the wake of the 2007 financial crisis jobs aren’t returning in great numbers – they’re being done by robots instead. This might be the first economic recovery in history that hasn’t been accompanied by an increase in employment.

Eventually we’re bound to discover a universal basic income, or something like it. It might take a very long time – a time in which the social strife could be immense. But eventually, barring armageddon, we will have to find a way to move to a post-work society; that means finding a way for people without work to live happy, fulfilled lives.

But in a life without work, how will people find meaning in their lives? With the ready availability of automation, some people might move back onto the land – to become 21st century peasant farmers. With the help of machines it could be quite possible for a family to provide for itself and maintain a decent standard of living through farming alone. Of course, with our heavily urbanised society plenty of people have neither the space nor the wherewithal to do this; but for some this could be a good alternative to the slums from where jobs have vanished.

The ready availability of time frees people up for any project they wish to embark on. The utopian outcome is enabling everyone to become creative, to embark on personal projects and self-expression. A society full of people doing what people do best: being human and creative. Is this realistic? Some people would relish the opportunity to dedicate themselves to creative pursuits. But plenty of others would not, instead looking for the structure and security their jobs used to offer.

Could this usher in an era of grand projects? Where people band together to achieve some lofty goal? Not for financial compensation, but for the joy of being part of something bigger. With everyone’s basic needs met, instead of working for money people look for meaning. They will take part in activities that inspire them, for free. The cost of labour at this point is basically zero, for the first time in human history. The only constraint: the end goal has to inspire people. Improving the efficiency of a government department? Hardly. Flying to Mars? Almost certainly.

But there will still be plenty of people who feel they can’t contribute but need structure in their life. Iain M. Banks’ Culture series describes a post-work society where the machines run everything; but in Banks’ universe people dedicate themselves full-time to leisure. While no doubt attractive to some, this life seems without structure, without purpose. Can people really live like this? Work has defined us, given structure to our day, given us meaning. Without this, who are we?

It is likely that the end result is some mixture of all of these outcomes. Each individual finding their own way in an increasingly diverse world – the simple answer of finding a job, any job, is no longer possible; instead people are forced to find their own answers to hard questions: what am I going to do with my life?

As we approach this future, what are we to do? How can we prepare for the upcoming earthquake in society? It seems inevitable that this will be a tumultuous time. How can we smooth the transition to a post-work society?

Code awareness levels

Writing code is all about working at multiple levels of abstraction concurrently. But as well as working at multiple levels of abstraction there are also multiple levels of awareness of the code.

The most basic level of code awareness is just making the code work. Does this line of code compile, run and do what I intended it to do? While learning to code this lowest level of awareness consumes almost all our capacity: it is difficult to focus on anything else other than just typing in the correct syntax.

Once we’ve mastered typing in code that works there is another level of awareness: is the intent of this line of code clear? Will another human being be able to understand this? Hell, will I be able to understand it in a month’s time? This is where code readability begins. While writing code we expend some of our mental capacity on making sure that the code will be readable.

Code legibility though is about more than just each single line of code in isolation. We also need to consider whether this line makes sense in the context of the rest of this method, maybe we should extract a new method? Does this line of code make sense in the context of the rest of this class? Does the behaviour belong somewhere else? Does the design that I’m discovering actually make sense? Is it consistent? Is the pattern I’m implementing consistent with the rest of the application? Besides making each line of code compile there are numerous higher levels – rising through each of the larger structures in the application: the method, the class, the module, the component, the system.

Besides code legibility there are other factors to be aware of: will failures in this code be easy to understand, will exceptions help me get to the right part of the code quickly, what happens if things I depend on fail? Will this code perform well, in what scenarios and with what frequency will it be called? Is this code secure, are there ways for an attacker to take advantage of this code?

Writing good code requires developers to be aware of these levels all the time. However, if we’re tired, hungover or fed up we probably won’t give much time to these concerns. We’ll just make the code work that’s in front of us and ignore the bigger picture. When we’re lacking mental capacity, we tend to stop worrying about the higher levels and focus on the lower level, easier problems. “I’ll come back later and fix the design”, we tell ourselves.

This is where pairing can help. I don’t think it’s true that the pairing partner is only thinking about the bigger picture; but the non-driving partner certainly has spare capacity to think about these things and act as a conscience against the driver missing something or getting tired.

I like to use hands-on coding exercises in interview situations because it helps highlight how much awareness a developer has. Generally I’m looking for developers that, for a given problem, have enough capacity to solve the problem quickly and well. Less capable developers will generally take longer and/or solve the problem badly. In an interview situation though, there are extra levels of awareness, besides just solving the problem: is what I’m doing creating a good impression? Am I solving the problem in the right way (e.g. using TDD)? Is the way I’m solving the problem showing off the right skills?

For example a while back I interviewed a candidate who used “yield return”. This is an unusual thing to see, so I asked why use it? “To start a discussion about whether or not you should ever use it”. This I found quite impressive: a candidate with enough mental capacity to not only solve the problem at hand, in a good way; but still with enough spare capacity left to realise that there was an opportunity to use an unusual construct purely to provoke a discussion of it – effectively driving the interview through code, to show off the depth of their knowledge and experience. This is the kind of person you want on your team: someone with enough spare mental capacity to make sure the code is clean and the design correct.

I think this lack of capacity is why the code we write when we’re tired or when we’re learning a new language or learning a new framework is bad: all our capacity is focused on just making the code work, we don’t have spare capacity to also think about whether the code is readable or whether the design makes sense. When we’re struggling to just make the code do what we want, to just keep the compiler happy, is it any wonder the outcome is a mess?


If you want to see how much mental capacity I can bring to your company, why not hire me? I’m looking for new roles right now – so drop me a line dave@activelylazy.co.uk.

Continuum of Design

How best to organise your code? At one end of the scale there is a god class – a single, massive entity that stores all possible functions; at the other end of the scale are hundreds of static methods, each in their own class. In general, these two extremes are both terrible designs. But there is a continuum of alternatives between these two extremes – choosing where along this scale your code should be is often difficult to judge and often changes over time.

Why structure code at all?

It’s a fair enough question – what is so bad about the two extremes? They are, really, more similar to each other than any of the points between. In one there is a single class with every method in it; in the other, I have static methods one-per class or maybe I’m using a language which doesn’t require me to even group by classes. In both cases, there is no organisation. No structure larger than methods with which to help organise or group related code.

Organising code is about making it easier for human beings to understand. The compiler or runtime doesn’t care how your code is organised, it will run it just the same. It will work the same and look the same – from the outside there is no difference. But for a developer making changes, the difference is critical. How do I find what I need to change? How can I be sure what my change will impact? How can I find other similar things that might be affected? These are all questions we have to answer when making changes to code and they require us to be able to reason about the code.

Bounded contexts help contain understanding – by limiting the size of the problem I need to think about at any one time I can focus on a smaller portion of it, but even within that bounded context organisation helps. It is much easier for me to understand how 100 methods work when they are grouped into 40 classes, with relationships between the classes that make sense in the domain – than it is for me to understand a flat list of 100 methods.

An Example

Let’s imagine we’re writing software to manage a small library (for you kids too young to remember: a library is a place where you can borrow books and return them when you’re done reading them; a book is like a physically printed blog but without the comments). We can imagine the kinds of things (methods) this system might support:

  • Add new title to the catalog
  • Add new copy of title
  • Remove copy of a title
  • User borrows a copy
  • User returns a copy
  • Register a new user
  • Print barcode for new copy
  • Fine a user for a late return
  • Pay outstanding fines for a user
  • Find title by ISBN
  • Find title by name, author
  • Find copy by scanned id
  • Change user’s address
  • List copies user has already borrowed
  • Print overdue book letter for user

This toy example is small enough that written in one-class it would probably be manageable; pretty horrible, but manageable. How else could we go about organising this?

Horizontal Split

We could split functionality horizontally: by technical concern. For example, by the database that data is stored in; or by the messaging system that’s used. This can work well in some cases, but can often lead to more god classes because your class will be as large as the surface area of the boundary layer. If this is small and likely to remain small it can be a good choice, but all too often the boundary is large or grows over time and you have another god class.

For example, functionality related to payments might be grouped simply into a PaymentsProvider – with only one or two methods we might decide it is unlikely to grow larger. Less obviously, we might group printing related functionality into a PrinterManager – while it might only have two methods now, if we later start printing marketing material or management reports the methods become less closely related and we have the beginnings of a god class.

Vertical Split

The other obvious way to organise functionality is vertically – group methods that relate to the same domain concept together. For example, we could group some of our methods into a LendingManager:

  • User borrows a copy
  • User returns a copy
  • Register a new user
  • Fine a user for a late return
  • Find copy by scanned id
  • List copies user has already borrowed

Even in our toy example this class already has six public methods. A coarse grained grouping like this often ends up being called a SomethingManager or TheOtherService. While this is sometimes a good way to group methods, the lack of clear boundary means new functionality is easily added over time and we grow ourselves a new god class.

A more fine-grained vertical grouping would organise methods into the domain objects they relate to – the recognisable nouns in the domain, where the methods are operations on those nouns. The nouns in our library example are obvious: Catalog, Title, Copy, User. Each of these has two or three public methods – but to understand the system at the outer level I only need to understand the four main domain objects and how they relate to each other, not all the individual methods they contain.

The fine-grained structure allows us to reason about a larger system than if it was unstructured. The extents of the domain objects should be relatively clear, if we add new methods to Title it is because there is some new behaviour in the domain that relates to Title – functionality should only grow slowly, we should avoid new god classes; instead new functionality will tend to add new classes, with new relationships to the existing ones.

What’s the Right Way?

Obviously there’s no right answer in all situations. Even in our toy example it’s clear to see that different structures make sense for different areas of functionality. There are a range of design choices and no right or wrong answers. Different designs will ultimately all solve the same problem, but humans will find some designs easier to understand and change than others. This is what makes design such a hard problem: the right answer isn’t always obvious and might not be known for some time. Even worse, the right design today can look like the wrong design tomorrow.

How many builds?

I’m always amazed at the seemingly high pain threshold .net developers have when it comes to tooling. I’ve written before about the poor state of tooling in .net, but just recently I hit another example of poor tooling that infuriates me: I have too many builds, and they don’t agree whether my code compiles.

One of the first things that struck me when starting to develop on .net was that compiling code was still a thing. An actual step that had to be thought about. Incremental compilers in Eclipse and the like have been around for ages – to the point where, generally, Java developers don’t actually have to instruct their IDE to compile code for them. But in Visual Studio? Oh it’s definitely necessary. And time consuming. Oh my god is it slow. Ok, maybe not as slow as Scala. But still unbelievably slow when you’re used to working at the speed of thought.

Another consequence of the closed, Microsoft way of doing things is that tools can’t share the compiler’s work. So ReSharper have implemented their own compiler, effectively. It incrementally parses source code and finds compiler errors. Sometimes it even agrees with the Visual Studio build. But all too often it doesn’t. From the spurious not-actually-an-error that I have to continually instruct ReSharper to ignore; to the warnings-as-errors build failures that ReSharper doesn’t warn me about; to the random why-does-ReSharper-not-know-about-that-NuGet-package-errors.

This can be infuriating when refactoring. E.g. if an automated refactor leaves a variable unused, I will now have a compiler warning; since all my projects run with warnings-as-errors switched on, this will fail the build. But ReSharper doesn’t know that. So I apply the refactoring, code & tests are green: commit. Push. Boom! CI is red. But it was an automated refactor for chrissakes, how’ve I broken the build?!

I also use NCrunch, an automated test runner for Visual Studio (like Infinitest in the Java world). NCrunch is awesome, by the way; better even than the continuous test runner in ReSharper 10. If you’ve never used a continuous test runner and think you’re doing TDD, sort your life out and setup Infinitest or NCrunch. It doesn’t just automate pressing the shortcut key to run all your tests. Well, actually that is exactly what it does – but the impact it has on your workflow is so much more than that. When you can type a few characters, look at the test output and see what happened you get instant feedback. This difference in degree changes the way you write code and makes it so much easier to do TDD.

Anyway I digress – NCrunch, because Microsoft, can’t use the result of the compile that Visual Studio does. So it does its own. It launches MSBuild in the background, continually re-compiling your code. This is not exactly kind on your CPU. It also introduces inconsistencies. Because NCrunch is running a slightly different MSBuild on each project to the build VisualStudio does you get subtly different results sometimes; which is different again from ReSharper with its own compiler that isn’t even using MSBuild. I now have three builds. Three compilers. It is honestly a miracle when they all agree that my code compiles.

An all-too-typical dev cycle becomes:

  • Write test
  • ReSharper is happy
  • NCrunch build is failing, force reload NCrunch project
  • NCrunch builds, test fails
  • Make test pass
  • Try to run app
  • VisualStudio build fails
  • Fix NuGet problems
  • NCrunch build is now failing
  • Force NCrunch to reload at least one project again
  • Force VisualStudio to rebuild the project
  • Then the solution
  • Run app to sanity check change
  • ReSharper now shows error
  • Re-ignore perennial ReSharper non-error
  • All three compilers agree, quick: commit!

Normally then the build fails in CI because I still screwed up the NuGet packages.

Then recently, as if this wasn’t already one of the outer circles of hell. The CI build was failing for a bizarre reason. We have a command line script which applies the same build steps that CI runs, so I thought I’d run that to replicate the problem. Unfortunately the command line build was failing on my machine for a spectacularly spurious reason that was different again than the failure in CI. Great. I now have five builds which don’t all agree on whether my code compiles.

Do you really hate computers? Do you wish you had more reasons to defenestrate every last one of them? Have you considered a career in software development?