Project Estimates versus Targets

I was going to have another Blog post that talked about Agile estimating, but I think a Blog about estimates versus targets should be done first. In short, I think the confusion and mingling of the terms have led to many ‘perceived’ project failures. As usual, let’s start at the start. 🙂

Definitions

Estimate – An opinion and educated guess as to the amount of work, people, and ultimately duration. An estimate can only considered an estimate if practitioners have created it. Ideally, an estimate is only an estimate if the project team has created it. But it the consulting world, this is sometimes impossible. The most I can hope for is that skilled and competent technical resources have created the estimate, even if they are not going to be on the project team.

Target – A opinion and uneducated guess as to the amount of work, people, and ultimately duration. A target is different from an estimate in that it is usually created by non-Practitioners. The target is usually created to put a stake in the ground to reflect an Enterprise objective or goal, Regulatory requirements or the criteria required so that a business case holds.

The Problem

The problem that I have seen over and over again in the Information Technology industry is the inter-mingling of targets and estimates. Many times we ourselves are the problem. In an effort to be non-confrontational we accept targets as now the project estimate. Subsequently, the corporation wonders why our Software Development projects never meet estimates. In my experience, Software Development teams meet estimates pretty well, where we struggle is meeting targets. Is this any surprise? I’m sure if I created a target for the annual budgeting process for a large corporation I also would be way off because I can’t appreciate all the processes and factors that need to be considered. (Hey, an analogy I can use!) 🙂

I have usually found that even if a project team acquiesces to a target provided/imposed, the project itself usually comes in at about the estimate the project team created. Typically, the project team can estimate accurately unless the project is in a totally new technology or domain.

But what If I am not given a choice?

This is a situation that probably occurs more often that not. I would estimate that most of the times project teams adopt a target as an estimate, they do so because of direction from Senior Management. So what can we do?

I would recommend the following approach:

1. Education – Start diligent and professional educational communications with Senior Management. Ensure they become aware of the differences between targets and estimates.
2. Language – Use the language and terminology in all discussions and documentation. This is part of the education plan to communicate the difference between the two terms.
3. Track both – Even if the project is executing according to an imposed target, track the project actuals against both the target and estimate. Evaluate after the project is complete and build up data from multiple projects. Don’t use this data to support an unprofessional ‘I told you so’ communication. This is very bad form. But this data will be interesting to present in a professional way. Executives don’t like surprises and being called out on the carpet when their initiatives always exceed the plan. Presenting these figures in a professional and respectful manner may convince executives to accept estimates rather than targets. (when possible)
4. Use estimates internally – Even if targets have been imposed on your team, I would recommend to still create estimates and then use these estimates to track progress with your team. Tracking with targets is unfair to the team. It will create an atmosphere of mistrust and unhappiness. The team will feel that they are constantly failing. The team wants to feel they are doing a good job. Tracking by estimates gives the best opportunity to do that and to measure progress.  Ideally create these estimates with Planning Poker and manage in an Agile way, but that is a topic for another Blog.

Decision Point

These are strategies to move the company and Senior Management in the right direction to have successful projects. It is not a short-term strategy will take multiple projects. If Senior Management chooses to not listen after you have been able to gather the information and present your case, every individual has to decide if the culture of the company is right for them.

But I believe that unless we go through the process to inform Senior Management, we are also part of the problem. The worst thing we can do is just to accept the targets as estimates.

My Agile Adoption Story

My first experience with an Agile project that was more Agile than Waterfall was in 2006. I phrase it in that way as all projects I have been on have had some aspects of Agile. But I digress, back to the story…

In December of 2005 a fixed price RFP was released for a new Parks Reservation Service that the Government of Manitoba required completion for the spring of 2006. (April 3rd to be exact) We were very diligent in planning to even respond to the RFP as we knew what a challenge the requested time frames would provide. So we met with the proposed team and held a risk session/planning session/estimating session to determine if we felt this was do-able and if we all were comfortable with bidding. It turned out that we thought this was certainly do-able, but it would be a challenge right from the start. We took a vote in that session and it was unanimous that we should bid. We thought that we needed to execute the project in a way that was more Agile than we ever had done before to be successful. This meant to us at the time:

• Team Chemistry and continuity
• At least 1 client on site full time with full decision making authority
• Daily stand-ups
• Fully empowered team (This is normally the way we operate)
• Very lean Use Case requirement documents.
• Demos at least every week to show progress
• Team Planning and Estimating
• Visual Project Management

What were we missing?

You may have noticed several current key Agile practices that were not in our list. Such as:

• Test Automation
• Continuous Integration
• Kan Ban
• User Stories
• Planning Poker
• Pair Programming

But before we get to that. Let’s provide some context for the project.

The Problem

• Dimensions of Parks and Campgrounds
• 57 Campgrounds
• About 5,500 campsites
• 11 Campgrounds currently computerized
• Requirement was for a Web Application that need to be used by three types of clients:
• General Public to make reservations over the internet
• Call Centre employees to make reservations on behalf of campers.
• Campground attendants to manage inventory, make reservations on behalf of campers, and check-in/check-out campers
• Web Application needed to have the following main components:
• Welcome/Login
• Shopping Cart
• Campground Management
• Inventory Management
• Reservation Search
• My Account
• Reports
• Reservation Maintenance
• Reports
• Policy Configuration
• Integration with online payment provider

Kick off to Deployment in 92 days

To meet the required deadline, we made the following project execution decisions:

• We partnered with three other companies to provide expertise so we could hit the ground running. (We could have learned this expertise but we did not have the luxury of time to do so)
• We did not have experience in test automation and felt it would be too risky to learn on this project. To mitigate this we allocated an Application Architect to manually perform Quality Assurance activities daily.
• We allocated an additional Technical Architect to Performance Test the application for the first three months.
• We leveraged Protegra Frameworks to speed up development velocity.
• We had a Technical Spike to validate and confirm what GIS mapping solution we would use.
• We separated the project into three phases to be able to deliver the functionality when it was required:
• Phase 1 – Public and Call Centre Focus
• Functionality focused primarily on the reservation process and reservation maintenance (i.e. making reservations, changes, cancellation, payment, refund, etc.)
• Phase 2 – Campground focused
• Focused on management of campgrounds (i.e. check campers in, check campers out, move campers, camping permits, revenue reconciliation, etc.)
• Phase 3 – Admin site enhancements
• Focused primarily on providing more back office support to Parks head office users

The Solution

The Solution was a C#, .NET Solution. The database was SQL Server and the architecture was a standard three-tier architecture.

Project Details

• Project team included over 20 developers at maximum
• First Phase was delivered in 3 months.
• Included investigation and integration of GIS mapping technology
• Phase 1 alone was over 7,400 hours of development work, all three Phases were over 12,000 hours.
• Quality of the application has exceeded industry standards with only a few defects since go live

The Outcome

The solution was deployed to production on time on April 10th, 2006. For those of you keeping track, this was a week later than initially planned due the time required to train the users. The solution was ready to deploy on April 3rd. This was after the project learned a month before go-live that the code had to be frozen two weeks prior to go live to allow for the acquisition of PCI certification to take on-line payments.

The project was awarded the PMI project of the year (Manitoba Chapter) for 2006.

What We Learned

• Client decision maker on site is key to a Lean project.
• Liaise with the clients as much as possible
• Empowering the development team was crucial.
• Traceability of requirements is key with formal organizations
• The team succeeded because they trusted one another

What we would do differently

• Quality could be further improved with the use of TDD, automated test and build tools
• Form in sub-teams of 6-8 to maximize efficiency. Assign a Client decision maker to each one of these sub teams.
• Leverage Visual Project Management to an even greater extent
• Develop in Iterations, not increments. We really developed fully functioning increments.
• Use the following Agile practices:

• Test Automation – We mitigated this risk by assigning one of our top Application Architects, but I would not recommend this approach unless you have no alternative.
• Continuous Integration – We probably got away with this because we coded in increments. If you are doing iterative development, I believe you need Continuous Integration.
• Kan Ban – We did not use a traditional Kan Ban board. We now use these on all projects.
• User Stories – Use Cases were still too large and they were really why we developed in increments. I don’t think you can deliver in Iterations without User Stories.
• Planning Poker – We had a very senior team who felt very comfortable in a joint planning session and challenging each other’s estimates. I wouldn’t recommend this going forward though as this team was very unique.
• Pair Programming – Strangely enough, this is one Agile practice that we rarely do. We typically use it more for a mentoring opportunity. The book is still out on this one though as I would like to try it on a project.

Why we succeeded

In some ways we succeeded primarily because of the excellent people and team we had. I believe that is still a key factor in the success of projects. That said, Agile practices do result in better solutions and everyone on the team realizes that we could have had an even better solution and project with them. (And certainly a more stable pace)

I believe we also succeeded because we had the client decision maker on site. This was not just any decision maker though. The Client was a person that had lived and breathed working with the old system, creating the RFP, and knowing what she wanted from a new system. In short, our client on site was also our Product Owner. Very powerful…

Check it out

If you want to check out the solution, you can find it at the following link:

http://prsasr.gov.mb.ca/

Agile Project Estimate Guarantee

Over the last few days I’ve noticed more discussion around the concepts on whether Agile teams should estimate or not. This includes my latest Blog on the subject:

User Story Points versus User Story Hours

Yesterday I read the following Blog post on the 5 reasons you should stop estimating User Stories:

5 Reasons why you should stop estimating

Given the amount of attention this blog has garnered, I felt I needed to provide an opposing viewpoint. The Blog post was very interesting and compelled me to write as I think these discussion points are missing one crucial factor. To summarize, the blog post proposed that these were the 5 reasons why we should stop estimating User Stories:

1)      Estimating User Stories is a waste and the time can be better spent elsewhere

2)      The estimates will be used for blame and cause the team to focus solely on the deadline

3)      Don’t give estimates as these are promises that are hard to keep. In the end they are going to be incorrect anyway.

4)      Don’t put additional pressure on the development team and cause them to compromise quality to make the deadline. (similar to #2)

5)      The estimates may cause a shift in the project’s priorities due to the fear and uncertainty of large items.

What struck me was what was missing from these reasons for not estimating, namely the client. None of the reasons explained why estimating User Stories created less value for the client. And really isn’t that why the project exists?

Estimates, who gets to decide?

The Client does. Sometimes when reading about Agile principles, I fear we are losing that perspective and we are deciding for the client what has value. The reason I believe in Lean so profoundly is that it provides the focus on why we are doing Agile practices. It helps to ensure that we understand the why of what we are doing at all times.

Let’s review the Blog points:

1) Estimating User Stories is a waste and the time can be better spent elsewhere

Whether estimating is a waste is something that only the client can best decide. Estimating is just another project activity that the client will determine if it is required.

So far every client I have worked with have required estimates at some level or another. Whether or not we think the time is best spent elsewhere is immaterial. The client defines value and the value in their value stream. So far, every client I have worked with has found value in estimates as they need to decide whether the initiative satisfies the business case and whether they can acquire the required budget.

2)  The estimates will be used for blame and cause the team to focus solely on the deadline

The fact that estimates can be or may be used for blame is more a comment on a dysfunctional system than a comment on estimates. If estimates can be used in this manner, I would imagine any other deliverable could also be used in this manner. I believe it is the role of a good project leadership team to ensure this doesn’t happen. Not producing estimates is treating a symptom, not the problem.

3) Don’t give estimates as these are promises that are hard to keep. In the end they are going to be incorrect anyway.

Estimates are hard, so let’s not do them. They are going to be incorrect anyway. Let’s just wait until they are actuals and then we can report them and we will have eliminated our risk. But whose risk are we referring to? We have just transferred all of the risk from the project team to the client.  If we are professionals in our craft of Software Development, shouldn’t we be able to provide our expert assessment on the situation? If not, why should the client trust us?

4) Don’t put additional pressure on the development team and cause them to compromise quality to make the deadline.

My opinion on this one is similar to #2. This is more a comment on a dysfunctional system than a comment on estimates.  The comment implies that the team will sacrifice quality to meet an estimate.

This sounds like another instance of treating the symptom and not the problem. If a project manager is holding the team to estimates and not engaging the client with new ideas, then the Project Manager is probably not the right fit. If the client is doing this, well that is certainly the client’s call. It is after all their solution. Quality for the client may be meeting the deadline.

5) The estimates may cause a shift in the project’s priorities due to the fear and uncertainty of large items.

Valid point. But I think this is the correct behaviour. Large stories will shift the priorities because of the perceived large risk they entail. This I believe is consistent with the principles of Agile to take on these large tasks early to minimize risk and learn fast/fail fast. I don’t think this is necessarily related to estimating. In fact, this is a reason why we need to do estimates. Estimates will ensure we identify the large epics as high risk items and address them early. This will help us to fail fast. If we do not estimate, we may leave these this stories as a lower priority and they will be done later and possibly cause project rework.

Next Steps

One critical benefit about providing an estimate to the client is that it completes the commitment cycle. Typically clients commit budget and teams commit to an estimated solution vision. This estimated solution vision is at a very high level that identifies the features and interactions between the features. It is not big requirements or design up front. But the team should have an understanding of the solution vision before starting or the potential is that excessive rework may be required after a considerable amount of work has already been completed.

In short, preparing that estimates forces the project team to understand and envision the entire solution because we are now committed.

Typically Planning Poker sessions produce estimates, but the real by-product is the consolidated solution vision that the entire team, including the client, has after the session. (In addition to the prioritized User Stories) That is the real value. Not producing estimates leaves the project in much poorer state.

The Guarantee

Why do we believe so strongly in this? Because we focus on the end state or the ultimate Solution. By focusing just on the Software Development process and activities you can Sub-optimize and although the Software Development process can be better (however you define better), the overall value of the project can be diminished or possibly no longer apply. Not producing estimates is one example of potentially many where having a focus just on the Software Development process may adversely affect the raison d’etre of the overall initiative.

Just like outsourcing the act of Software Coding can result in less than optimal solution elsewhere in Software Development, not estimating a Software Development projects can create a less than optimal solution for the entire initiative or business.

Protegra believes so strongly that Agile projects can be successfully estimated that we have an offering to guarantee scope and budget with our clients when we are able to execute Custom Software Development Projects in our Lean and Agile Methodology. It is something we have been doing now for over 10 years.

Agile Project Characteristics – Two Controversial Factors

In my last post I listed the Agile Dozen Characteristics and Practices that I felt were critical on all projects. In that list though, there were two characteristics that I did not expect myself and that may raise the eyebrows of Agile purists when mentioned. Before there is discussion on all the factors and I present the experience reports, I thought it would be of value to discuss the following two characteristics:

• Developer to Designer Ratio
• Team meeting estimates

1) Developer to Designer Ratio

Where to start on this one. The first immediately apparent item is that there is a difference between developers and designers. Yes, I would say that there is room for both of these roles and that I believe that perhaps all of the design and development activities can’t be pulled by any member of the team. I know this principle is tantamount to heresy in some of the Agile circles, but I believe that context is everything and there are no absolutes. Based upon the problem domain, team composition and solution complexity, there may be a requirement for a role that owns the solution vision at a very high level. Now this does not imply the creation of detailed requirements and architects and designers defining exactly what developers need to code, but merely implies that there is a role that provides the overall vision of the solution at a high level and then works with the entire team as they flesh out user stories to ensure that the solution remains cohesive and consistent. This designer can then also provide leadership in the architecting the automated test suite as well.

This has worked well on our projects. I’d really like to hear back from others on this concept…

2) Team meeting estimates

This was the other characteristic that I thought would be somewhat controversial. Now I firmly believe in Planning Poker and collaborative estimating techniques, but at the end of the day, no matter how the estimates have been created, a successful project must deliver to the estimates that they have made. 😐 Whether this is a Traditional Project where estimates are defined at the task level through to an Agile project where User Stories and tasks are defined in points and through planning poker, eventually the execution must somewhat match the plan. (now matter how defined or loose the plan is) If this continues to happen through many iterations, the clients will get concerned and possible business plans that depend on the iteration outcomes may be compromised. To demand 100% meeting of the estimates is not feasible, but if the project consistently deliver less than 50% of the plan then some action needs to be taken. (Which could be to just be less optimistic and committing to less)

What do people think of these two more traditional characteristics? I’d love to hear feedback!

Building a better monster – Naturally Agile

When we last left our intrepid Project Manager (Evil Scientist) he was fighting off the hordes of rioting Business Users and negotiating change requests. Not a good scene and something that could certainly be avoided.

It turns out that the answer to how we can build a better project has been done forever by nature. Nature in itself is intrinsically iterative and abhors big bang implementations. (except for ‘the’ big bang, but that is a topic for a different blog entry)

To start at the beginning, let us review what the evil scientist was trying to do. The very objective of the project had two essential criteria:

1. To create life
2. To have this new life follow his direction

Each of these essential criteria has a complementary concept in the Agile world. Let’s discuss both of them in sequence.

1. Creation of Life

As you can probably surmise, my analogy for being able to manage a project is exactly how nature creates life. The creation of life project has the following external milestones:

• Visioning at Conception
• Implementation at Delivery

Now there is an immense amount of work being done between these two external milestones and thanks to modern medicine we have more of an eye to this progress than previous generations. Although the creation of life is a 40 week project (for humans), the following systems are initially formed at the following milestones:

• Nervous System – 8 weeks
• Circulatory System – 4 weeks
• Skeletal System – 6 weeks
• Respiratory System – 6 weeks
• Digestive System – 6 weeks

As you can see, although the implementation date is far in the future, nature really delivers initial functionality on all major sub-systems 8 weeks into the project. (I know I am hugely simplifying this, but I’m hoping you will cut me some slack. :)) So by 20% into the project nature has an end to end prototype (pun intended), that nature can then add subsequent functionality to before the final release. Some of this additional functionality is the refinement of these major systems which includes:

• Full Skeletal Structure – 1o weeks
• Hearing – 17 weeks
• Breathing of Fluid – 28 weeks
• Ability to Suck Thumb – 28 weeks

One important piece of information is that from weeks 33-40 it is reported that there are no major qualitative changes. During this period there is just more refinement and ‘hardening’ of the baby. (And possibly some refactoring?)

Let me leave you with two questions:

1. How much better shape would our projects be in if we had an end to end prototype built by the time we are 20% into the project?
2. How much better shape would our projects be in if we were only hardening our solution in the last 20% of a project?

2. Project Management Style

The other comment I would like to make quickly is that the Project Manager style of the Evil Scientist is definitely ‘old school’. I fundamentally believe that the new Project Manager style is more of a parent. Now I do not mean this to imply that the child project is beneath the parent in knowledge at all. But that the Project Manager is meant to facilitate the learning and growth of the project. And by doing that the project will also teach the Project Manager. At the end we really do need to look at projects as not something to be led, but rather as something that learn, grow, and succeed.

Perhaps we should term this concept Embryonic Project Management? Should the 20% milestones nature uses at the start and end of the project simulated in a project?