I can haz dependencies?

I hate IoC containers. Spring? Evil. Guice? The devil’s own work. Why? Because it leads to such slack, lazy, thoughtless programming.

Why the hate?

Ok, perhaps I better explain myself a bit. IoC is a great idea. What annoys me, is the way IoC frameworks end up getting used by normal people. I’ve ranted previously about how IoC containers lead us to implement anaemic domain models. The trouble is, once you have a hammer, everything starts to look like a nail. Especially those pesky fingers. Once you have a dependency injection framework, everything starts to look like a dependency that needs to be injected. Need to implement some business logic? First create a new class, test drive it, then make it injectable, inject it into the class where the calling code needs it, test driving it natch, then bingo – you just hit yourself on the finger.

Now I’ve got two classes, basically closely coupled, but the IoC container hides that fact from me. I see a nice, clean interface being injected in. Aren’t I a good little OO developer? No, you’re stupid and you’re lazy.

Before you know it, your class has a dozen or more dependencies, each of which have a dozen dependencies, each of which have a dozen dependencies, each of which… you get the picture. You’ve managed to build a rats nest of a dependency graph, little by little. What you’ve TDD’d isn’t a design. The technical name for it is The Big Ball of Mud.

An Alternative

Instead, I think dependency injection works best at application seams, at architectural boundaries. Say, for example, you’re building a web app. You’ve created a TradeEntryController that allows users to, well, enter trades. The TradeEntryController naturally has loads of dependencies on the rest of the system. It needs to fetch valid assets to invest in and prices, it needs to know what your balance is so you can’t buy more shares than money in the bank etc etc. A perfect example where life without an IoC container could become really cumbersome.

But, I don’t think you need one. I think what your controller needs is a few, specific dependencies – that define the architectural boundary the controller lives within. Above the controller is a HTTP request, a session and all that blah blah. Within it, is business logic. Below it is the database. So, the dependencies we inject should represent only the architectural context in which the controller operates. For the most part, this will be common to all my controllers – not just trade entry. Controllers for managing balances, lists of assets, user accounts – these all depend on knowing stuff about their session, and to be able to talk to the next layer down: the database (or in an n-tier setup, perhaps some web services).

So, why not just inject those dependencies?

public class TradeEntryController {
    public void setSessionManager(ISessionManager sessionManager) { ... }
    public void setTradeDatabase(ITradeDatabase tradeDatabase) { ... }
    public void setAccountDatabase(IAccountDatabase accountDatabase) { ... }
    public void setAssetDatabase(IAssetDatabase assetDatabase) { ... }
}

Then in my controller, I can fetch user information from the SessionManager; I can get the list of assets from the AssetDatabase; I can check the user’s balance via the AccountDatabase; and I can record the trade via the TradeDatabase. So far, so much the same as a normal IoC container.

So what’s different?

Rather than manage these dependencies via an IoC container. I think you should push them in manually. Yes, I’m suggesting you write your own dead simple dependency injection framework. What? Am I mad? Quite probably, but bear with me.

public interface ICanHazTradeDatabase {
    void setTradeDatabase(ITradeDatabase tradeDatabase);
}

public class TradeEntryController
    implements ICanHazTradeDatabase, ICanHazAssetDatabase...
{
    ...
}

public class ControllerFactory {
    public Controller createController(Class clazz) {
        Controller c = clazz.newInstance();
        if (c instanceof ICanHazTradeDatabase)
            ((ICanHazTradeDatabase) c).setTradeDatabase(tradeDatabase);
        if (c instanceof ICanHazAssetDatabase)
            ((ICanHazAssetDatabase) c).setAssetDatabase(assetDatabase);
        if ...

        return c;
    }
}

The exact mechanics of ControllerFactory of course depend on your MVC framework, but hopefully the idea is clear: when we instantiate a controller, we check it against a known set of interfaces and push in very specific dependencies. Is it pretty? Not really. Is it easy to write? Of course. Does it push dependencies into your controller? Well, yes. Where do they come from? Well, that’s an exercise for the reader. But I’m sure you can find a way to make ControllerFactory a singleton and instantiate all your dependencies in one place.

The Point

What, exactly, is the point of all this? Well, as a developer writing a controller – I can get easy access to all the dependencies that represent the architectural context I’m running within. The databases, services, message brokers, email server, blah blah blah that the application as a whole depends on. They’re right there – I just add the interface, one method and bang – ICanHazCheeseburger.

More interesting, is what I can’t do. I can’t decide that my TradeEntryController needs a TradePricingCalculator and inject that as a dependency. Well, I could, but I’d be making TradePricingCalculator available everywhere, and I’ve got a little more work to do than I would if I was using plain old Spring or Guice – I’ve an interface to create, a couple of lines to add to some scarily named GlobalControllerFactory. Why is this important? It adds some friction. It makes something bad hard to do. I’m forced instead to think about creating a TradePrices object and adding some functionality to it. I’m forced to have a rich domain, because I can’t just move all my functionality off into a TradePriceCalculatorVisitorFactoryManagerBuilder.

The choices we make and the technologies we choose make some things easy and other things hard. We need to think carefully about whether the things we make easy should be easy. It’s always possible to do the right thing, but sometimes we need to make it easier than doing the wrong thing.

Oh my God, what have I done?

As I write this, I’m sitting in the hotel apartment in Italy. I’ve left my wife & 16 week old son in the UK while I spend two weeks working for a new company in Italy. In January, all three of us move out here for at least the next three years.

Oh. My. God. What the fuck have I done?!

The Bad

Day two was pretty bad. The full enormity of it all suddenly hit home. I can’t make myself understood – I need others to do everything for me. I can imagine how my son must feel – no way of communicating and totally reliant on others. To be honest, it’s a wonder I didn’t start screaming at the top of my lungs and shit my pants.

After work I had to navigate, in the fog and dark, a route I’d never driven before – to find my way home. But before I could kick back and relax with a nice glass of wine, I first had to do battle with the supermarket and buy the wine. As well as food for the evening. Plus also pans to cook the food in. And knives to prepare it with. And… and… and… Clearly people staying in this apartment don’t cook. I mean, to call it under-stocked is to do a dis-service to apartments without kitchens. Totally fucking useless. Anyway. Easily solved. Just more money down the pan.

Then when I got home, the useless toilet paper carrier bags snap and the shopping crashes to the floor. Everything survives. Oh, except the wine. God-fucking-damnit I needed that.

After a day of hearing how critical it will be to live in the right area; and for my wife not to be bored out of her mind by moving here – I find out that the tantalizing prospect of my wife being able to continue working for the same company but from Italy, had been snatched away from us. This means my wife will be unemployed. In a country she doesn’t speak the language. With an economy going down the toilet. We expected this. But it’s still annoying.

I mostly spend the evening worrying what on earth I’m doing out here. How are we going to survive? Life in the UK with a 3 month old baby has been hard. How will we cope in a country we don’t know our way around, where we can’t speak the language? What the fuck will we do?

The good

Day three on the other hand, has been much better. After a day feeling useful at work – I mooch home, stop via the supermarket and buy yet more things I’d forgotten / couldn’t find the day before. So today I get home and can actually cook a meal (a bolognese, how appropriate). I even get a glass of wine. Or three. And suddenly today feels like a much better day!

But why? I’m starting to get to know a few of the ex-pats. Starting to get an idea of where we ought to live. Starting to feel that my wife might not be completely isolated. It sounds like there really is the community out here I’d heard about / hoped for. Ok, Sam still needs to get involved and settle in – but at least today I think it’s possible.

And at work the team – to my great relief – actually seem pretty fucking awesome. Sure, there are challenges. Sure, there are some issues. But they seem to basically want to do the job right. They seem to basically understand what it takes. I’m not sure why that hasn’t always happened, maybe they think they don’t have permission? But perhaps that’s where I come in. After all, if Software Craftsmanship is about anything – it’s about raising the bar, it’s about encouraging other developers to do the right thing, it’s about giving them permission to do it right. So here’s hoping this is a great opportunity to help a team excel, to do the job I believe in and have an influence in an industry I’ve loved for years.

All-in-all – I’m hoping I’ve made an awesome, massive, life-changing decision that will be full of win for all three of us. Of course, having lunch with and meeting people you know off the telly puts the squee level into fucking over-drive. Oh. My. Fucking. God. I really am working here.

Tomorrow will now be an utter piece of shit disaster making me wish I was back in Blighty. Hey ho. Gotta love the roller coaster of life.

Apologies to people expecting this to be a blog about technology. Normal techie-ranty-goodness will resume shortly. Life-with-a-capital-L just slightly more on my mind at the minute.

Update: further posts on this topic will be at my new blog, specific to our move to Italy: English South Of Milan.

Ability or methodology?

There’s been a lot of chatter recently on the intertubes about whether some developers are 10x more productive than others (e.g. here, here and here). I’m not going to argue whether this or that study is valid or not; I Am Not A Scientist and I don’t play one on TV, so I’m not going to get into that argument.

However, I do think these kinds of studies are exactly what we need more of. The biggest challenges in software development are people – individual ability and how we work together; not computer science or the technical. Software development has more in common with psychology and sociology than engineering or maths. We should be studying software development as a social science.

Recently I got to wondering: where are the studies that prove that, say, TDD works; or that pair programming works. Where are the studies that conclusively prove Scrum increases project success or customer satisfaction? Ok, there are some studies – especially around TDD and some around scrum (hyper-performing teams anyone?) – but a lazy google turns up very little. I would assume that if there were credible studies into these things they’d be widely known, because it would provide a great argument for introducing these practices. Of course, its possible that I’m an ignorant arse and these studies do exist… if so, I’m happy to be educated 🙂

But before I get too distracted, Steve’s post got me thinking: if the variation between individuals really can be 10x, no methodology is going to suddenly introduce an across the board 20x difference. This means that individual variation will always significantly dwarf the difference due to methodology.

Perhaps this is why there are so few studies that conclusively show productivity improvements? Controlling for individual variation is hard. By the time you have, it makes a mockery of any methodological improvement. If “hire better developers” will be 5x more effective than your new shiny methodology, why bother developing and proving it? Ok, except the consultants who have books to sell, conferences to speak at and are looking for a gullible customer to pay them to explain their methodology – I’m interested in the non-crooked ones, why would they bother?

Methodologies and practices in software development are like fashion. The cool kid down the hall is doing XP. He gets his friends hooked. Before you know it, all the kids are doing XP. Eventually, everyone is doing XP, even the old fogies who say they were doing XP before you were born. Then the kids are talking about Scrum or Software Craftsmanship. And before you know it, the fashion has changed. But really, nothing fundamentally changed – just window dressing. Bright developers will always figure out the best, fastest way to build software. They’ll use whatever fads make sense and ignore those that don’t (DDD, I’m looking at you).

The real challenge then is the people. If simply having the right people on the team is a better predictor of productivity than choice of methodology, then surely recruitment and retention should be our focus. Rather than worrying about scrum or XP; trying to enforce code reviews or pair programming. Perhaps instead we should ensure we’ve got the best people on the team, that we can keep them and that any new hires are of the same high calibre.

And yet… recruitment is a horrible process. Anyone that’s ever been involved in interviewing candidates will have horror stories about the morons they’ve had to interview or piles of inappropriate CVs to wade through. Candidates don’t get an easier time either: dealing with recruiters who don’t understand technology and trying to decide if you really want to spend 8 hours a day in a team you know little about. It almost universally becomes a soul destroying exercise.

But how many companies bring candidates in for half a day’s pairing? How else are candidate and employer supposed to figure out if they want to work together? Once you’ve solved the gnarly problem of getting great developers and great companies together – we’ll probably discover the sad truth of the industry: there aren’t enough great developers to go round.

So rather than worrying about this technology or that; about Scrum or XP. Perhaps we should study why some developers are 10x more productive than others. Are great developers born or made? If they’re made, why aren’t we making more of them? University is obviously poor preparation for commercial software development, so should there be more vocational education – a system of turning enthusiastic hackers into great developers? You could even call it apprenticeship.

That way there’d be enough great developers to go round and maybe we can finally start having a grown up conversation about methodologies instead of slavishly following fashion.

Software craftsmanship roundtable

Last night was the second London Software Craftsmanship round table. This was a great session with lots of lively discussion. We seemed to cover a wide variety of topics and I came away with loads of things I need to follow up on.

It’s amazing, as soon as you start discussing things with other developers you pick up on lots of different ideas, tools and technologies you’d not previously explored. These sessions are proving to be a great way to find out some of the cool stuff people are doing. So what did we discuss?

*phew* I think that covers it. If I missed anything, let me know below.

So, are you coming to the next one? What should we discuss?

 

The true cost of technical debt

Whether you like to think of it as technical debt or an unhedged call option we’re all surrounded by bad code, bad decisions and their lasting impact on our day to day lives. But what is the long term impact of these decisions? Are we really making prudent choices? Martin Fowler talks about the four classes of technical debt – from reckless and deliberate to inadvertent and prudent.

Deliberate reckless debt

Deliberate reckless technical debt is just that: developers (or their managers) allowing decisions to be made that offer no upside and only downside – e.g. abandoning TDD or not doing any design. Whichever way you look at it, this is just plain unprofessional. If the developers aren’t capable of making sensible choices, then management should have stepped in to bring in people that could. Michael Norton labels this “cruft not technical debt“. If we’re getting no benefit and simply giving ourselves an excuse to write crappy code then it’s not technical debt, it’s just crap.

Inadvertent debt

Inadvertent technical debt is tricky. If we didn’t know any better, how could we have done any differently? Perhaps industry standards or best practices have moved on. I’m sure once upon a time EJBs were seen as a good idea, now they look like pure technical debt. Today’s best practice so easily becomes tomorrow’s code smell.

Or perhaps we’d never worked in this domain before, if we had the domain knowledge when we started – maybe the design would have turned out different. Sometimes technical debt is inevitable and unavoidable.

Prudent deliberate debt

Then there’s prudent, deliberate debt. Where we make a conscious choice to add technical debt. This is a pretty common decision:

We need to recognise the revenue this quarter, no matter what

We’ve got marketing initiatives lined up so we’ve got to hit that date

We’ve committed to a schedule so don’t have time for rework

Sometimes, we have to make compromises: by doing a sub-standard job now, we get the benefit of finishing faster but we pay the price later.

Unlike the other types of technical debt, this is specifically a technical compromise. We’ve made a conscious decision to leave the code in a worse state than we should do. We know this will slow us down later; we know we need to come back and fix it in “phase 2”; but to hit the date, we accept compromise.

Is it always the right decision?

1. Compromise is always faster in the short-term and slower in the long-term

Given a choice between what we need now and some unspecified “debt” to deal with later – the obvious choice is always to accept compromise.

2. Each compromise is minor, but they compound

Unlike how most people experience “debt”, technical debt compounds. Each compromise we accept increases the cost of all the existing debt as well. This means the cost of each individual compromise may be small, but together they can have a massive impact.

First of all I decide not to refactor some code. Next time round, because its not well factored, it’s harder to test, so I skip some unit tests. Third time round, my test coverage is poor, so I don’t have the confidence to refactor extensively. Now it’s getting really hard to test and really hard to change. Little by little, but faster and faster, my code is deteriorating. Each compromise piles on the ones that went before and amplifies them. Each decision is minor, but they add up to a big headache.

3. It’s hard to quantify the long term cost

When we agree not to rework a section of code, when we agree to leave something half-finished, when we agree to rush something through with minimal tests: it’s very difficult to estimate the long term cost. Sure, we can estimate what it would take to do it right – we know what the principal is. But how can we estimate the interest payments? Especially when they compound.

How can I estimate the time wasted figuring out the complexity next time? The time lost because a bug is harder to track down. The extra time it takes because I can’t refactor the code so easily next time. The extra time it takes because I daren’t refactor the code next time; and the extra debt this forces me to take on. How can I possibly quantify this?

At IMVU they found they:

underestimated the long-term costs [of technical debt] by at least an order of magnitude

Is it always wrong?

There are obviously cases where it makes sense to add technical debt; or at least, to do something half-assed. If you want to get a new feature in front of customers to judge whether its valuable, it doesn’t need to be perfect, it just needs to be out there quickly. If the feature isn’t useful for users, you remove it. You’ve saved the cost of building it “perfectly” and quickly gained the knowledge you needed. This is clearly a cost-effective, lean way to manage development.

But what if the feature is successful? Then you need a plan for cleaning it up; for refactoring it; making sure it’s well tested and documented sufficiently. This is the debt. As long as you have a plan for removing it, whether the feature is useful or not – then it’s a perfectly valid approach. What isn’t an option is leaving the half-assed feature in the code base for years to come.

The difference is having a plan to remove the debt. How often do we accept compromise with a vague promise to “come back and fix later” or my personal favourite “we’ll fix that in phase 2”. My epitaph should be “now working on phase 2”.

Leaving debt in the code with no plan to remove it is like the guy who pays the interest on one credit card with another. You’re letting the debt mount up, not dealing with it. Without a plan to repay the debt, you will eventually go bankrupt.

The Risk of Technical Debt

Perhaps the biggest danger of technical debt is the risk that it represents. Technical debt makes our code more brittle, less easy to change. As @bertvanbrakel said when we discussed this recently:

Technical debt is a measure of code inflexibility

The harder it is to change, the more debt laden our code. With this inflexibility, comes the biggest risk of all: that we cannot change the code fast enough. What if the competitive or regulatory environment suddenly changes? If a new competitor launches that completely changes our industry, how long does it take us to catch up? If we have an inflexible code base, what will be left of our business in 2, 3 or 4 years when we finally catch up?

While this is clearly a worst case scenario, this lack of flexibility – this lack of innovation – hurts companies little by little. Once revolutionary companies become staid, unable to react and release only derivative products. As companies find themselves unable to innovate and keep up with the ever changing landscape – they risk becoming irrelevant. This is the true cost of technical debt.

Without a plan to repay the technical debt; with no way to reliably estimate the long term cost of not repaying it, are we really making prudent, deliberate choices? Isn’t the decision to add technical debt simply reckless?