Feeds:
Posts
Comments

Posts Tagged ‘agile’

Agile places an emphasis on the importance of the team. The team make the decisions: what do we work on today, how do we tackle our constraints, even who should be in the group. But yet some research seems to suggest that individuals are more effective than teams.

For example in “59 seconds” Richard Wiseman questions the effectiveness of brainstorming – groups tend to focus on mundane, easily agreed upon suggestions; or be swayed by uncreative, charismatic team members.

How do we reconcile this conflict? If groups tend to lack creativity and flexibility in their thinking, why do agile teams appear to be more creative, more flexible and above all more effective? Is it an illusion, or does agile actually help teams achieve more?

Just one developer?

The trouble with software is its rarely a lone sport anymore. There aren’t many fields where one developer on his own can make a significant contribution. But where one developer can make meaningful progress you will get the best bang for buck. As soon as you add a second team member you need much more communication (ok, I might sit and talk to myself sometimes, but I talk much more when there’s another human being there). By the time you’re adding a third, fourth or fifth developer, you’re spending loads of time just talking and drawing on whiteboards and standing around having meetings.

If I’m the only person to have touched the code, when it crashes – I know exactly whose fault it is. As soon as there are more developers, we get to play blamestorming. “Well it works fine on my PC”, “It worked last time I ran it”, “You checked in last – it must be your bug”. You start to get the diffusion of responsibility that Wiseman talks about. People don’t feel personally responsible for the output, so they don’t feel compelled to make it better: half assed is good enough, it’s not my problem.

The truth is most activities of any size nowadays require a team of people to work on, which immediately raises the question of who works on what, when.

Fluidity

Maybe agile helps teams be more effective by letting the team be more fluid. Rather than the smartest people getting stuck on one problem or in one area, the fluidity and constant reassessment of agile allows the smart people to automatically refocus to where they need to be. But critically, it doesn’t need someone to micromanage the situation and tell them to work on the most important things – people will “self-organise” and naturally gravitate to where they can help most.

At the daily standup Harry says:

Jim – are you doing ok with the checkout flow? You’ve never done anything like that before so would it help if I came and paired with you today? The order history page can wait until next week so we can hit our target for Friday.

Magic: a “self-organising team”. Imagine some asshat manager had said that! Jim would feel like an idiot, Harry gets to feel awkward so tries not to ride roughshod over Jim’s work – both get dragged down and demotivated; the end result is slow, sloppy work and a miserable team. Instead, because the team came up with the idea, everyone’s happy about it and the work gets done as quickly as possible.

Because different people are always offering help – either because they’re nosy and want to know how something works, or because they’re some smartass know-it-all that’s good at everything – the fact that the smartest people are quickly rotating round the group’s biggest problems isn’t always plain to see. Everyone is moving around; but most of the movement is noise: the important thing is that the brightest, most capable people are moving to where they are needed most.

Maybe the fluidity simply creates a socially acceptable way for the smart people on the team to leap from problem to problem without the rest of the team feeling stupid.

Who’s the rockstar?

I hate the term, but if agile teams are more effective because the “rockstar” developers are working on all the important stuff – that suggests everyone else is working on the unimportant stuff. Now, if your company has time to pay idiots to work on stuff that nobody wants – maybe I can offer you some overpriced consultancy?

But that doesn’t happen, does it? Perhaps because the “rockstar” on the team, is probably only good at playing guitar (stretching the tortured analogy). I’ve heard him play a drum solo: it’s shit. But the drummer? Yeah, he’s not too bad at that. Everyone on a team has different strengths, and will do best at certain tasks. As a manager, it’s almost impossible to try and assign people to tasks to get the best out of everyone and deliver the most value possible. You’re basically trying to allocate resources centrally, which turns out to be pretty hard.

Instead by delegating resource allocation to the team, the team decide who would be best on each activity; the team take responsibility for delivering as much value as quickly as possible. Even if that sometimes means people are working on tasks they’re not suited for – those who are better at it might be working on something more valuable. Sometimes you need a drummer, even if they’re not the best drummer in the band.

Costs

Regular task switching and lots of pairing is great for creating an environment where developers can move from task to task easily. But this comes with a cost – I can’t immediately pick up where James left off, I need to talk to him to find out what he was doing and where he got to, I need to learn and understand the code he wrote yesterday before I can write more. This has a cost to it.

What about the diffusion of responsibility? If six different people all work on the same feature, won’t we find that nobody really cares whether it works, because everyone blames the other guys? Well, assuming we’re all professional developers, I’m sure we wouldn’t sink to such childish behaviour. But it’s surprising how easy you can become detached from the goal you’re aiming for – the overall benefit you’re trying to deliver for the customer. You know what’s left to do, so you do your little bit. You don’t think about the overall goal and what the customer actually wanted. You take your eye off of quality for a split second and bang! You screwed up.

I suspect diffusion of responsibility is a genuine problem in agile teams – which is why shared ownership is emphasised. We all own this code – so treat it as your own. In another light, it’s the craftsmanship ethic – to leave the code a little better than you found it. Don’t just assume the other guy knew what he was doing: fix it, properly. Without this, the diffusion of responsibility would lead to chaos.

To tolerate all these costs: task switching, diffused responsibility, communication and coordination overhead – there simply must be a massive benefit. The upside of having the right people on the right task at the right time must outweigh all those downsides.

But does it always? Does it on your team?

Read Full Post »

Software architecture is hard. Creating a simple, consistent, flexible environment in which we can solve the customer’s ever-changing problems is no easy task. Keeping it that way is harder still. Striking the right balance between all the competing demands takes real skill – so what does it take to create a good architecture? How much architecture is enough?

Software Architecture

First, I’m drawing a distinction between software architecture and enterprise architecture. By software architecture I mean the largest patterns and structures in the code you write – the highest level of design detail. I do not mean what is often called enterprise architecture: what messaging middleware to use, how are services clustered, what database platforms to support. Software architecture is the stuff we write that forms the building blocks of our solution.

The Over Architect

I’m sure we’ve all worked with him: the guy who could over think hello world. When faced with a customer requirement his immediate response is:

we need to build a framework

Obviously the customer’s problem is too simple for this genius. Instead, we can solve this whole class of problems. Once we’ve built the framework, we just need to plug the right values into the simple 400 line XML configuration file and hey presto! customer problem solved.

Sure, we’ve only been asked to do a one-time CSV import of some customer data. But think of the long-term, what will they ask for next? What about the next customer? We should write a generic data import framework that could take data in CSV, XML or JSON; communicating over HTTP, FTP or email. We can build rich, configurable validation logic. We can write the data to any number of database platforms using a variety of standard ORM frameworks. Man, this is awesome, we could be busy for months with this!

Whatever. You Ain’t Gonna Need It!

But sometimes, the lure of solving a problem where you’re the customer, is much more intellectually stimulating than solving the boring old customer’s problems. You know, the guy who’s paying the bills.

The Over Architect generalises from a sample size of one. Every problem is an opportunity to build a more general solution, despite having no evidence for what other cases might need to be solved. Every problem is an opportunity to bring in the latest and greatest technology – whether or not its a good fit, whether or not the company’s going to be left supporting some byzantine third party library that’s over kill for their simple use. An architect fully versed in CV++

The Under Architect

On the other hand, the Under Architect looks at every customer problem and thinks:

we could re-use what we did for feature X

Where by “re-use” he means copy & paste, change as necessary. There’s no real architecture, just patterns repeated ad infinitum. Every new requirement is an opportunity to write more, new code. Heaven forbid we go back and change any of that crufty old shit. No, we’ll just build shiny, brand new legacy code.

We’re building a web application: so we’ll need some Controllers, some Views and some Models. There we go, MVC – that counts as an architecture, right? Oh, we need a bit more. Well, we’ve got some DAOs here for interacting with the database. And the business logic? Well, the stuff that’s not wrapped up in the controllers we can put in FooManager classes. Sure, these things look like monolithic god classes – but its the best way to aggregate all the related functionality together.

Lather, rinse, repeat and before you know it you have a massive application with minimal structure. The trouble is, these patterns become self-perpetuating. It’s hard to start pulling out an architecture when all you have is a naming convention.

The Many Architects

The challenge in many software teams is everyone thinks it’s their job to come up with new architecture or start building a new framework. The code ends up littered with half-finished, half-forgotten frameworks. Changing anything becomes a nightmare: was all this functionality used? We have three different ways of importing data, via three different hand-rolled frameworks – which ones are used? How much of each one is used? Can I refactor them down into one? Two? What about the incompatibilities and subtle differences?

Without a clear vision changing the code becomes like archeology. As you delve down through the layers you uncover increasingly crufty old code that nobody dares touch any more. It becomes less of a software architecture and more of a taxonomy problem – like Darwin trying to identify a million different species by their class structure.

The Answer

What’s the answer? Well I’m sorry, but I just don’t buy this agile bullshit about “emergent architecture”. Architecture doesn’t emerge, it has to be imposed, often onto unwilling code.

Architecture requires a vision: somebody needs to have a clear idea about where the software is headed. Architecture needs patience: as we learn more about the problem and the solution, the architecture will have to adapt. Architecture needs consistency: if the guy calling the shots changes every year or two, you’ll be back to the Many Architects problem.

Above all, I think good architecture needs a dictator. Some, single person – taking responsibility for the architecture. They don’t need to be right, they just need to have a clear vision of where the architecture should head. If the team are on board with that vision then the whole team are pulling in the same direction, guided by one individual taking the long view.

Central Architecture Group

This sounds like I’m advocating a central architecture group? Hell no. The architect needs to be involved in the code, hands-on, day-to-day, so he can see the consequences of his decisions. He needs the feedback from how the product evolves and how our understanding of the problem evolves. The last thing you need is a group of ivory tower architects pontificating about whether an Enterprise Service Bus is going to solve all our problems. Hint: it won’t, but firing the central architecture group might.

Conclusion

Getting software architecture right is a hard problem. If you keep your code DRY and SOLID, you’re heading in the right direction. If someone has the vision for where the code should head and the team work towards that, relentlessly cleaning up old code – then maybe, just maybe you’ve got a chance.

 

Read Full Post »

There’s been a lot of chatter recently on the intertubes about whether some developers are 10x more productive than others (e.g. here, here and here). I’m not going to argue whether this or that study is valid or not; I Am Not A Scientist and I don’t play one on TV, so I’m not going to get into that argument.

However, I do think these kinds of studies are exactly what we need more of. The biggest challenges in software development are people – individual ability and how we work together; not computer science or the technical. Software development has more in common with psychology and sociology than engineering or maths. We should be studying software development as a social science.

Recently I got to wondering: where are the studies that prove that, say, TDD works; or that pair programming works. Where are the studies that conclusively prove Scrum increases project success or customer satisfaction? Ok, there are some studies – especially around TDD and some around scrum (hyper-performing teams anyone?) – but a lazy google turns up very little. I would assume that if there were credible studies into these things they’d be widely known, because it would provide a great argument for introducing these practices. Of course, its possible that I’m an ignorant arse and these studies do exist… if so, I’m happy to be educated :)

But before I get too distracted, Steve’s post got me thinking: if the variation between individuals really can be 10x, no methodology is going to suddenly introduce an across the board 20x difference. This means that individual variation will always significantly dwarf the difference due to methodology.

Perhaps this is why there are so few studies that conclusively show productivity improvements? Controlling for individual variation is hard. By the time you have, it makes a mockery of any methodological improvement. If “hire better developers” will be 5x more effective than your new shiny methodology, why bother developing and proving it? Ok, except the consultants who have books to sell, conferences to speak at and are looking for a gullible customer to pay them to explain their methodology – I’m interested in the non-crooked ones, why would they bother?

Methodologies and practices in software development are like fashion. The cool kid down the hall is doing XP. He gets his friends hooked. Before you know it, all the kids are doing XP. Eventually, everyone is doing XP, even the old fogies who say they were doing XP before you were born. Then the kids are talking about Scrum or Software Craftsmanship. And before you know it, the fashion has changed. But really, nothing fundamentally changed – just window dressing. Bright developers will always figure out the best, fastest way to build software. They’ll use whatever fads make sense and ignore those that don’t (DDD, I’m looking at you).

The real challenge then is the people. If simply having the right people on the team is a better predictor of productivity than choice of methodology, then surely recruitment and retention should be our focus. Rather than worrying about scrum or XP; trying to enforce code reviews or pair programming. Perhaps instead we should ensure we’ve got the best people on the team, that we can keep them and that any new hires are of the same high calibre.

And yet… recruitment is a horrible process. Anyone that’s ever been involved in interviewing candidates will have horror stories about the morons they’ve had to interview or piles of inappropriate CVs to wade through. Candidates don’t get an easier time either: dealing with recruiters who don’t understand technology and trying to decide if you really want to spend 8 hours a day in a team you know little about. It almost universally becomes a soul destroying exercise.

But how many companies bring candidates in for half a day’s pairing? How else are candidate and employer supposed to figure out if they want to work together? Once you’ve solved the gnarly problem of getting great developers and great companies together – we’ll probably discover the sad truth of the industry: there aren’t enough great developers to go round.

So rather than worrying about this technology or that; about Scrum or XP. Perhaps we should study why some developers are 10x more productive than others. Are great developers born or made? If they’re made, why aren’t we making more of them? University is obviously poor preparation for commercial software development, so should there be more vocational education – a system of turning enthusiastic hackers into great developers? You could even call it apprenticeship.

That way there’d be enough great developers to go round and maybe we can finally start having a grown up conversation about methodologies instead of slavishly following fashion.

Read Full Post »

Last month we ran our first company coding dojo – this was only open to company staff, but attendance was good (around a dozen people).

For those that have never heard of it, a coding dojo – based on the idea of a martial arts dojo – is an opportunity for programmers to improve their skills. This means getting a group of developers together, round a big screen, to work through a problem. Everything is pair programmed, with one “driver” and one “co-pilot”. Every so often the pair is changed: the driver returns to the audience, the co-pilot becomes the driver and a new co-pilot steps up. That way everyone gets a turn writing code, while the rest of the group provide advice (no matter how unwelcome).

For the first dojo we tackled a problem in Scala - this was the first time using Scala for most people, so a lot of time was spent learning the language. But thanks to Daniel Korzekwa, Kingsley Davies & DJ everyone got to grips with the language and we eventually got a solution! The session was a lot of fun, with a lot of heated discussion – but everyone felt they learned something.


Afterwards, in true agile style, we ran a quick retrospective. The lessons learned showed the dojo had been an interesting microcosm of development – with us making the same mistakes we so often see in the day job! For example, we knew we should start with a design and went as far as getting a whiteboard; but failed to actually do any design. This led to repeated rework as the final design emerged, slowly, from numerous rewrites. One improvement for next time was to do just in time design – in true agile style.

We also set out to do proper test-first TDD. However, as so often happens, this degenerated into code-first development with tests occasionally run and passing rarely. It was interesting to see how quickly a group of experienced developers fall out of doing TDD. Our retrospective highlighted that next time we should always write tests first, and take “baby steps” – by doing the simplest thing that could possibly make the test pass.

Overall it was a great session and very enjoyable – it was fascinating to see the impact of ignoring “best practices” on something small where the results are so much more immediate.

Read Full Post »

I spent last week at the Agile 2010 Conference. It was my first time at a conference this size; I definitely found it interesting and there were some thought provoking sessions – but there weren’t many deeply technical talks. As others have asked, what happened to the programmers?

Bob Martin wrote that

Programmers started the agile movement to get closer to customers not project managers

He also commented on how few talks were about programming

< 10% of the talks at #agile2010 are about programming. Is programming really < 10% of Agile?

People have already commented on how cost is a factor in attending a conference like this – especially for those of us outside the US who have expensive flights to contend with, too. This is certainly a factor, but I wonder if this is the real problem?

Do developers attend a conference like Agile 2010 to improve their craft? How much can you cover in a 90 minute session? Sure, you can get an introduction to a new topic – but how much detail can you get into? Isn’t learning the craft fundamentally a practical task? You need hands on experience and feedback to really learn. In a short session with a 100+ people are you actually gonna improve your craft?

Take TDD as an arbitrary example. The basic idea can be explained fairly quickly. A 90 minute session can give you a good introduction and some hands on experience – but to really grok the idea, to really see the benefit, you need to see it applied to the real world and take it back to the day job. I think the same applies to any technical talk – if its interesting enough to be challenging, 90 minutes isn’t going to do it justice.

This is exacerbated by agile being such a broad church; there were developers specialising in Java, C#, Ruby and a host of other languages. Its difficult to pitch a technical talk that’s challenging and interesting that doesn’t turn off the majority of developers that don’t use your chosen language.

That’s not to say a conference like Agile 2010 isn’t valuable, and I’m intrigued to see where XP Universe 2011 gets to. However, I think the work that Jason Gorman is doing on Software Craftsmanship, for example, is a more successful format for technical learning – but this is focused clearly on the technical, rather than improving our software delivery process.

Isn’t the problem that Agile isn’t about programming? It is – or at least has become – management science. Agile is a way of managing software projects, of structuring organisations, of engaging with customers – aiming to deliver incremental value as quickly as possible. Nothing in this dictates technical practices or technologies. Sure, XP has some things to say about practices; but scrum, lean, kanban et al are much more about the processes and principles than specific technical approaches.

Aren’t the biggest problems with making our workplaces more agile – and in fact the biggest problems in software engineering in general –  management ones, not development ones? Its pretty rare to find a developer that tells you TDD is bad, that refactoring makes code worse, that continuous integration is a waste of time, that OOD leads to worse software. But its pretty common to find customers that want the moon on a stick, and want it yesterday; managers that value individual efficiency over team effectiveness, that create distinct functional teams and hinder communication.

There is always more for us to learn; we’re improving our craft all the time. But I don’t believe the biggest problems in software are the developers. Its more common for a developer to complain about the number of meetings they’re asked to attend, than the standard of code written by their peers.

Peers can be educated, crap management abides.

Read Full Post »

Older Posts »

%d bloggers like this: