The Misleading Expectation on Estimates

When we talk about software development, the subject of “estimations” is a broad field of misunderstanding and misleading expectation. Especially when we talk about agile methodologies like scrum or kanban, there are many misunderstandings and misconceptions. A common misconception is to force teams to do “scrum by the book”.

Whenever we are in the role as of stakeholder, we want to know how long things take. Even though some developers believe this is an unacceptable request, I believe it applies to anyone of us, and I am going to share two simple examples why.

Once in your life, you might decide to buy a house. The assumption is that it requires some renovation before you move in for living. It is obvious that you want to know how long it takes to renovate your building, and it means that you must deal with an estimate from some workers.

In case you own a car or a bicycle, once in a while you ask for a maintenance check. Of course, you want to know how long it takes. Some companies ask for your phone number so that you get a call as soon as they are done.

There are many more examples like planning a road trip with Google Maps or when your car shows the distance you can reach with the remaining fuel. All those examples have one thing in common: the initial intent is a valid and reasonable intent. The hypothesis is, that, also in software development, a request for an estimate is valid and reasonable.

Of course, you are expecting an accurate estimate in all examples because it has an impact on your life. In these examples it leads to higher expenses or more time spent for commuting to work. In both cases, you are disappointed in case the estimate was too optimistic. On the other hand, you pick another company if an estimate is too pessimistic. Therefore, it is worthy to take a closer look at this topic.

Understanding Estimates

When we talk about estimates, we are usually referring to effort which is required to develop, maintain or modify software. It is used for planning or budgeting so that time or money is derived from estimated effort. Effort can be expressed by person hours, story points, t-shirt sizes or any variation of that.

A well-known routine in software teams is to do refinement meetings on a regular basis. In a refinement meeting, the team starts with questions towards a product owner about all kind of requirements. Sometimes stakeholders are invited as well to refinement meetings to keep the communication loop small. Afterward, the team agrees on a certain estimate and continues with another feature.

How the team comes to an estimate is highly dependent on their learned routines. Those routines are very different from team to team. Some teams establish voting systems, some do separate estimations by discipline, some agree on an algorithm how to calculate one final number from multiple estimates.

Another very common approach is the “no-estimate” approach. The assumption is, that every ticket takes the same amount of time on average. For that reason, every ticket gets the same estimate independent of its actual effort. The goal of a refinement meeting is to get a common understanding of the topic rather than too much time on aligning on the most precise estimate. As soon as the discussion is over, a ticket gets a “checked” and the team continues with the next ticket.

Either way, it is important to get an understanding of how estimates are biased.

Misleading Expectation and its Impact

I have observed several facets which lead to wrong expectation.

We must keep in mind that we are dealing with high uncertainty. Usually, a new feature has not yet been built in such context. A good analogy is: Even though you are an excellent cook, you are facing challenges when you try out a new recipe for the first time.

Secondly, any estimate is highly influenced by psychological factors. Several factors influence estimates, for example the environment, the expectations of stakeholders, the relationship among team members or their individual personal mood. Those factors cannot be completely reduced.

It is easy to estimate what is known, but it becomes difficult to estimate what is known as unknown. And we must also accept that we need to face unknowns which were not known before. Even the best experts cannot cover such a case!

Another issue is the misunderstanding of accuracy and precision. While accuracy is expected, developers are pushed towards precision. In this context, I have observed software teams which were pushed again and again into refinement meetings because they got the feedback that their estimates are not “accurate” enough. After several refinement meetings, their estimates have become more precise but at the same time, there has been no evidence that those estimates have also become more accurate. With some effort into a “proof-of-concept”, this could easily have been circumvented.

A very similar problem occurs when teams have the routine to calculate a final number out of several estimates. At a first glance, this looks very convincing, but it does not show the interaction within the team. A team with well-established collaboration habits is a lot more efficient and a team with unhealthy conflicts will always perform worse than such estimates. It is detached from the team’s reality and for that reason misleading.

I would also highly recommend avoiding any kind of election of estimates. The best idea can be easily down-voted when the majority has a different opinion. There is no escape hatch to show fact-driven evidence. A better approach is always to run a small experiment to gather evidence instead. Knowledgeable people can be easily overruled when teams do election of estimates. Especially in big teams this approach looks beneficial. However, members of big teams build clusters based on their personal preference and relationship which highly influences any voting. This is a big cause of misleading estimates.

Consequence of misunderstanding is that decisions would have been taken differently. For instance, depending on risks depending oning roadmaps would have been planned differently. Therefore, we need a strategy how to reduce bias.

Strategies for Improving Estimate Accuracy

To increase accuracy of estimates, we need to get a mapping between planned working items and actual tasks which have been done by the team. We, human beings, have our habits and routines, so it is very likely that planned working items look very similar to actual tasks which have been worked on. Also, my observation is that software teams are good at writing down their actual to-dos. When we ask for feedback about a new feature, it should be easily possible to write down what needs to be done to release a feature. In the past, I have done one-on-one meetings with individual members of the team. In very short brainstorm session, we have evaluated the amount of work and I mentioned explicitly that an incomplete list is acceptable. In a second step, it is worthy to check how many additional items we must consider as optional so that we get a feeling of how much time buffer we must consider on top. The benefit of one-on-one meetings is that I can schedule them more flexibly and the conversations are more open and trustful. The disadvantage is that the total amount of time to get the final estimate is much higher.

If parts of a feature are unknown, and it seems like a challenge to write down any to-do, I suggest planning “proof-of-concept” or “spikes”. It should be clear for everyone that it is perfectly acceptable for all work to be thrown away. My experience is that this rarely happens. Still, the goal should be to get more insides about working items instead of building the feature itself.

Conclusion

Estimates are always a snapshot of a current status quo. Experience and knowledge in a project changes over time. Estimates need re-evaluation, and dependent budget and plans need readjustments, otherwise delays in projects are guaranteed! It is fine to start with vague estimates on a new project as long as low reliability is accepted and as long as estimates are refined as soon as more information is available.