Yep, Estimating is hard. It isn’t easy and it isn’t without peril. Unfortunately it is a fact on 95% of the projects we work on. So given that, I’m not sure that telling people not to estimate is productive. There are a lot of misinformation about estimating though. Not all estimates are evil and not all are a waste of time. Here are the three estimating rules that I follow that have served me well.
1. Estimate Iteratively
I can’t figure out for the life of me why people who discuss Agile projects assume that if you estimate you must be BEUF. (Big Estimate Up Front) I always estimate iteratively with the estimate getting more refined as I go along. Usually the stages are:
Proposal Estimate – At the very start of the process when you don’t know much
Plan Estimate – Creating an executable plan and estimate before you start after you know more about scope
Execution Estimate – Re-planning with the final team just before execution
Iteration Estimate – Re-planning before each Iteration
With an iterative approach to estimating you can provide a line of sight to the clients as to what they can expect and you can also communicate how the whole project will evolve. Even the estimates.
2. Estimate with the team
This is a no-brainer. Always, always, always estimate with the team. If your team composition changes, I would re-estimate with the new team. Friends don’t let friends estimate alone. It is as much a learning experience as anything else.
3. Estimate the Minimum Viable Product
This is probably the most important rule. Let me be clear. You have no business starting or being involved on a Software Development project if you can’t estimate that you can at least deliver the Minimum Viable Product. (MVP) Yes, I know it is hard and difficult and fraught with errors, but an estimation based on years of experience if better than no opinion at all. It can save wasting money on a project that had no hope of success. And it doesn’t need to be a very detailed estimate at the start, just a high-level estimate that confirms the team believes it can be done. (see rule #1)
Some Estimating Falsehoods
There are some common Estimating Falsehoods out there. I have taken these from the Blog post on estimating by Matt Rogish that takes the point that estimating is harmful.
Here we go:
“In really terrible places to work, someone other than the developer will actually do the estimating. This estimate will then be given to the developer as an implicit (or at especially evil places, explicit) time not to exceed. This is such a depressing situation I cannot dare think of it further.” – Correct. This is really a problem with bad leadership rather than estimating. Don’t throw the baby out with the bath water.
“Most estimation is drawn from past experience. What if the developer doesn’t have this experience? How can they estimate? This is where things get tricky. And by tricky, I mean “completely made up.” Sure, for things that have been done before by someone the developer can draw some parallels and make a guesstimate. But what about the stuff that has never been done before (presumably, the secret-sauce that makes you money)? Usually you just take a Wild-Ass Guess (WAG-estimation) and hope you’re not wrong. Given the rarity of being punished for under-promising and over-delivering this WAG tends to be a massive over-estimation.” – This is a common Falsehood. As mentioned in Rule 2, no one estimates alone. Senior developers help the less experienced and share their wisdom. This point also again highlights just plain old bad management.
“Completely spec out the entire system ahead of time, spend a lot of time researching and estimating the problem, determine project dependencies, and you can determine the “finish date” before writing a line of code! It’s seductive and taps into the part of our brain that craves order and dependability.” – See Rule 1 – Estimate Iteratively. I can’t imagine any Agile professional would fall into this line of thinking.
“If the value of software we’re writing is so low, is it worth being written? If the project has such tight time constraints that a schedule slip will doom the project then you’re in a world of hurt. (The solution is agile: work on the most important things first, have an always-working project, and cut scope to hit a date.)
This exposes a major weakness in most software companies: the inability to determine project value. Given a project of unknown value, the conservative business will then attempt to minimize cost. Unfortunately as we’ll see, cost minimization ends up becoming very expensive in the long run.” – This is a strange point to me. We as software developers can’t estimate our projects, but YOU the business need to estimate the value. Well we can’t have it both ways. I agree Agile is the solution, IF we estimate we can complete the Minimum Viable Product within the current budget. See Rule #3.
“I’m not entirely sure what this means, but I’ve never seen the folks who are writing the specs, requirements, etc. ever being asked to estimate how long each spec will take to write. Or have the CEO give a date when the company will hit some arbitrary metric, although see the perversity in public company earnings estimates and reporting.” – I’ve seen both on all the projects and companies I’ve worked for. CEO bonuses are commonly tied to unrealistic estimates on an arbitrary timeframe.
“Ultimately companies require estimation because they don’t trust anyone. They don’t trust developers to deliver results. They don’t trust mid-management to motivate their reports. The underlying reality is that the folks “in charge” are worried that everyone is going to rip off the company and only by holding people to arbitrary deadlines are they assured stuff will get done.
Committing to a date allows them to sleep better at night and shifts the blame from them to the folks on the front lines. “ – Wow. If anyone works at a place like this, leave. But don’t blame the estimates. If you don’t produce estimates, this company will just find another way to pull a fast one on you.
“Estimation is ultimately a futile effort. Software, more or less, is like writing poetry. Or solving mathematical proofs. It takes as long as it takes, and it’s done when it’s done. Perfect, or imperfect, estimation won’t change how long it takes to complete the work. If that sounds horrible to you, then go do something else.” – I could not disagree more. It is correct that estimation won’t change the amount of work it will take to complete something, but that isn’t the point. The point is about giving some imperfect information to the client to help him/her make business decisions. You remember the client right? The one who is going to pay the bills and we are trying to deliver value to right? I would agree that truly unique software solutions are like writing Poetry. I was on a R&D project recently and the estimating was challenging to say the least. But 95% of Software Development projects are following semi-established patterns. The projects are not routine, but nor are they like creating something fundamentally new all the time.
“Estimation actually slows down development because estimation isn’t free. The developer is incentivised to take as long as possible estimating in an effort to not get beaten when they miss the estimate. Why waste that time when the metric the developer is attempting to hit ultimately generates negative value?” – See Rule 1 on estimating iteratively and this is just an example of bad management once again.
Summary
Estimation is difficult, never correct, and is fraught with danger. But if you are like me, the vast majority of my clients require them. So the question is not why can’t we stop doing estimates?, but rather how can we do them better?
In almost all cases, the problems with estimating can be traced to bad leadership on management. That is where I believe our focus should be.
Agile PMO 