Monday, December 5, 2011

You are not 90% done

Have you ever had a programmer tell you that they're 90% done?

It happens all the time. Don't believe them.

I'm a programmer. I should know.

You see, there's a funny thing with software. It's very fluid, and can end up being very complicated.

For simple tasks, it's fairly easy to estimate the effort required, based on previous experience.

An example often used in project management literature is "digging a ditch".

If an average man can dig a 10 meter long ditch in 1 hour, then some excellently simple math can tell you that:
  • That same man can dig a 20 meter ditch in 2 hours
  • 30 men can dig a 300 meter ditch in 1 hour
  • 500 men can dig 50 100 meter ditches in 1 hour

Unfortunately there are a lot of people that still try to apply this same logic to software development.

It simply doesn't work. Here's why. Let me tell you a little story.

Mark the Manager gives Peter the Programmer a specification for a new, complex feature in an existing, complicated piece of software.

Mark asks for an estimate, to which Peter replies, "Well, there are a lot of unknowns, but based on the spec, I think 2-3 weeks. Worst case scenario? 4 weeks."

Mark would like it done faster, but agrees to the estimated timeline and gives the green light.

Peter starts working steadily, 40 hours a week, with no vacation, personal, or sick time.

Here are his progress reports:
  • Week 1: 35% complete -- Perfectly on schedule
  • Week 2: 65% complete -- A bit behind, but still on track
  • Week 3: 85% complete -- Uh oh, but at least we'll have it next week
  • Week 4: 90% complete -- Ok, now we're late
  • Week 5: 95% complete -- Customers are getting anxious
  • Week 6: 97% complete -- Customers are irate
  • Week 7: 99% complete -- Customers are leaving
  • Week 8: 100% complete -- We are bankrupt

Disaster! What went so terribly wrong?

Perhaps the following visualization will help:

This is such a common curve in software development, it should have a name. If it doesn't already, I propose calling it the "Programmer Optimism Curve" or POC.

So what do we do to prevent this from happening? There is clearly no silver bullet solution.

I've written about my 5-step software development estimation method. That can certainly help.

The "story point" methodology from Agile can also work well when you have a long project with the same team members.

Jeff Atwood has some good suggestions about listing todo items too.

Of course, "knowing is half the battle," so now that you're aware of the problem, you can start taking active steps to keep it from sinking your ship.