Sprint Estimation Plan: Don’t Estimate Stories
This is part three in a series on estimating. Part one was “Don’t Estimate Software Defects” and Part two was “Don’t Estimate Spikes”. And, part three will be about estimating stories.
I’m not a big fan of estimating stories in sprint planning. I don’t do it, nor do I re-estimate stories in sprint planning. I estimate stories in a separate estimating meeting and usually at least a couple sprints in advance, if not more. There are a few reasons why (re)estimating stories during sprint planning is a dangerous practice: In sprint planning, we are thinking at a lower level of detail with far greater knowledge about the story, the code base and the system than we had when we estimated the rest of the backlog. You cannot correctly estimate such a story in sprint planning relative to some other story estimated with far less detail. Such a practice leads to velocity inflation and risky release planning.
Different Level of Detail
Once we get to sprint planning, we may have implemented a spike related to the story and we’re digging into the story details. We’ve thought about the design of the code for the story. We’re digging into the tasks. We know a whole lot more about this story than we did when we estimated the rest of the backlog.
So, if I estimate or reestimate the story in sprint planning, I CAN’T estimate it relatively to the rest of the backlog. I don’t know that level of detail about the rest of the stories. I’m now thinking at a different level of detail, using a different kind of thinking than when we estimated the other stories.
We know our estimates are wrong. We knew they were wrong when we estimated in release planning, and we know they are still wrong when we get to sprint planning. That’s why we have velocity — an empirical measure that takes all that wrongness into account.
During sprint planning, we break the stories down into tasks, estimate those tasks, and compare the task estimates against our capacity. It’s that, not points, that keep us from overcommitting in this sprint. No need to change the estimate.
“So what?” you ask. Bear with me and I’ll tell you why you care.
Greater Knowledge and Skill
If you are making release commitments, if you have a release plan, or if you follow our other advice, then your release backlog was likely estimated many weeks ago. All those estimates didn’t have the benefit of what we know now. Many of those estimates are suspect. Likely wrong. If we allow estimate changes once we hit sprint planning, there will be even more estimate changes in sprint planning. This is not the only wrong estimate. If this is happening too often (discovering poor estimates), we should reconsider how well groomed the backlog is and whether we should reestimate the whole remaining product backlog.
“So what?”, you ask “I just like my estimates to be right anyway.” Consider how reestimating in sprint planning might affect our velocity and therefore release planning…
Velocity Inflation
New estimates are most likely going to be higher, not lower. If an estimate is too big, development is not concerned. They might not mention it. If an estimate is too small, development is very concerned.
Think about how often you change or want to change an estimate in sprint planning. Hold that number in your head for a moment. Maybe it’s one story every other sprint? Maybe more? Perhaps 1 out of 20? Also consider how big of an increase it usually is. Is it from a 3 to a 5? From a 1 to a 2? From a 5 to an 8? Perhaps it’s usually a 1.6x increase?
Think about the rest of the product backlog for a moment. If we tend to increase 5% of the stories 1.6x AT THE LAST MINUTE, then your velocity relative to your remaining product backlog is TOO BIG. The remaining backlog is estimated relative to the original points, not relative to the reestimated points.
Here’s an Example
Example: Suppose we typically do around 10 stories and 20 points per sprint, and the distribution in size is something like [1, 1, 1, 2, 2, 2, 3, 3, 5]. Suppose it’s a 3 point story that becomes a 5. If we complete all those stories, should we say our velocity is 20 or 22? Say we have 400 points remaining in our release backlog. Can we implement 22 points out of that release backlog every sprint? I would say no, you can only knock out 20 backlog points each sprint.
To look at it another way, you think you have 400 points in your backlog, but since you are going to reestimate them on the fly, you really have 440 points to do. You won’t get done in 400/22 iterations. You’ll get done in 440/22 iterations. Or 400/20.
Our team’s ability to produce, their capacity, is stable or at least not increasing rapidly. That is our cash on hand. The price of stories, however, can increase more rapidly due to inflation. Silly analogy: I want to buy a backlog that is originally priced at $400. I can’t pay for it all at once, so I start putting aside some money each pay period. I can put aside $22 per period toward this backlog, but by the time I’ve saved up enough for the original $400, I find that the store has raised the price to $440. Bummer.
Conclusion
If I’m making a release commitment, I want to be absolutely sure I’m not overcommitting. I want to under commit and over deliver. I’m going to evaluate risks, reduce risk early, put risk mitigation stories in my backlog (create options) and reserve buffer for those risks I’m accepting instead of mitigating. I’m going to anticipate that I don’t know all the stories that we need to do. We haven’t thought of some of the dark matter we’re going to have to deal with, so I reserve a little buffer for that. That’s a reasonable and prudent approach. I absolutely don’t want velocity inflation to consume those little buffers. I need them for what I set them aside for. If I allow velocity inflation to use up my contingency, then I’m running a riskier project than I think I am.
It is for similar reasons that I don’t estimate newly discovered defects or unplanned spikes. If you aren’t careful, you’ll inflate your velocity and under estimate the size of your remaining backlog.
Comments (9)
George Dinwiddie
The more generalized version of this is not to re-estimate after you start doing the work, since you’re gaining knowledge that can’t be known when you estimate prior to that, and therefore make these estimates no longer comparable with estimates made prior to starting. The worst case is when people want to modify the estimates after the fact. “That story was a lot harder than we expected, so we really should get credit for an 8, not a 3.”
Using this more generalized version makes it apply to situations where the work is done differently from the workflow you describe. For example, I find relatively few teams do task breakdown during sprint planning, opting for JIT task breakdown when working on the story. Teams working this way don’t do task estimation, as it no longer provides value, and merely requires time and effort. Generally they slice the stories smaller if they’re experiencing tasks large enough that estimating task duration would seem to help. Working with small stories rather than task breakdowns requires more collaboration within the team, but also encourages limiting work-in-progress and concentrates on working software over work expended.
Also, it’s not always appropriate to “under commit and over deliver.” In a lot of corporate environments, that’s the safest and easiest way to appear competent. In other environments, the organization is less risk averse and will accept managing uncertainty for the tradeoff of pushing harder to accomplish more, sooner.
The generalization of this aspect is that our reason for estimating is to predict when a given amount of work will be done or how much work will fit in a given timebox. If we remember the question we want to answer with estimations, it’s easy to see why we don’t want to change our estimates later. At least, we don’t want to change them on a routine basis; there may be exceptions where we’d prefer to do something else.
When we forget what question we’re trying to answer, it’s easy to fall into considering estimates as “credit to the team” or “a measure of productivity.” These lead us to patterns of behavior that do not enhance the production of working software.
Thanks, George! I like that generalization. That’s helpful and is a better way to word my intent.
Iakiv
Hi Andrew!
I have some questions:)
When I first got familiar with the ideas from your post (from JIRA official documentation by the way) I liked them so much.
But I still did not manage to put them into life.
Our team started the project when it was already 6 years old. The system is pretty complex and big… And also – pretty bulky at frontend.
I encounter the following things that confused me when I tried to provide estimate only at the point of time when we don’t know details:
1. Because of bulky frontend, lack of ui/ux designer involved, lack of PO resources, priorities that may change unexpectedly – usually frontend developers estimate like this: if we go the simplest way it would be 3, if more “smart” – 5, if the “smartest” – 8.
2. Because of the old and complex project and because of we are a new team in it – backend guys may estimate like this: 100 sp, infinity, or 3 sp if it happens that implementation is easy and 40sp otherwise… Even if the story sounds like “little changes from user perspective”.
That’s why we often wait for more clarification for PO in “1” cases and wait for at least some investigation from developers in “2” cases in order to provide “initial” estimate. We don’t change it after…
Do you have any thoughts on this? Is there a way not to “wait” in our case?
I would love to find one…
Thanks
Hi Iakiv
> “Because of bulky frontend, lack of ui/ux designer involved, lack of PO resources,… we often wait for more clarification for PO…”
I assume you are working to fix that. You have to have a good PO and PO Team (https://www.leadingagile.com/2011/03/why-a-product-owner-team/) and have a well groomed backlog (https://www.leadingagile.com/2014/02/managing-risk-uncertainty-agile-2/).
> “Because of the old and complex project and because of we are a new team in it – backend guys may estimate like this…. That’s why we often… wait for at least some investigation from developers… Do you have any thoughts on this? Is there a way not to “wait” in our case?”
That’s why we have Spikes and why we identify and implement Spikes early FOR THE TYPES OF Organizations I usually work with, which are those that need to coordinate the work of multiple teams and need some predictability. If your context is different, my advice here might not apply to you. In my context, I have someone technical work with the PO to groom these stories, identify spikes, help estimate, and answer the complexity question. The PO team needs to understand it’s lead-time for grooming and groom 2, 3, or maybe more sprints in advance depending on the situation.
There’s no substitute for having the right people in the right roles doing the job that needs to be done.
Thomas Subera
Hi Andrew,
i am not entirely sure if i am in the same boat with you there. If it’s just about a 1.6x increase i don’t really care and so does the team.
But what if the story increases rapidly say from 1 to 5 within the first sprint planning phase. I agree that most information about a story is unclear, but that’s why we always talk about every new story before the first estimation so everything should be clear to the team.
But sometimes it happens and it is of no ones fault. The PO just thought it as clear that feature a need things b and c, the team on the other side feels deceived because vital information about the story was missing. It’s just a feature but well.. i really need this other thing to work too for this. (complexity costs *bing*).. And in this cases the PO did nothing wrong. There is still one story.. but more complex than the team thought of. (and no way to cut the story into smaller ones)
Now you say that if the story is estimated afterwards it will change it’s behaviour to the backlog.. well right it should. Because instead of a velocity of 20 the team can only produce 16 in this sprint which will make everyone wonder if somethings wrong.
I must confess it’s not a daily basis problem, more one or two in the year but nether less i would ask your opinion to it.
thanks in advance
Hi Thomas. Thank you for engaging on this.
Before I get to the heart of your question, let me comment on your 1st sentence. The figures I used in the post are just examples and I tried to make them conservative. The actual numbers for any given team could surely be worse. But sticking with those numbers for a moment, those figures worked such that someone who make a commitment in the face of this situation would be off by 10% of the duration of the release, or more if my numbers are conservative. +/- 10% is often acceptable, but if you are inadvertently planning on coming in at the 10% late mark, you could end up much worse. You need that +/- 10% for unforeseen things (such as the example in the body of your comment), not for preventable mathematical errors.
Now, I’ll address your real question: In your example, there was a miscommunication, a misunderstanding. It happens. Here’s where we aren’t on the same page, with your comment “Because instead of a velocity of 20 the team can only produce 16 in this sprint which will make everyone wonder if somethings wrong.” You don’t want anyone to ask questions or to think anything is wrong. I think we should make everyone wonder if something is wrong. Let’s discuss the goof in the retrospective and improve our process. Let’s reduce the future occurrence of this type of goof. Don’t say that everything is okay. Don’t change the number so that no one will ask questions.
Good luck to you!
andrew
Jeff
Hello
But how to deliver this gap of requirement in time if the 10% buffer isn’t enough? we propose to the po to add this as tasks under the next sprint?
Po often rejects this
Hi Jeff. I’m not sure if you are writing about overcommitting to a sprint or to something larger, such as a release plan or a quarterly planning cycle. I don’t worry much about a small overcommitment at a sprint level for a single/rare sprint. If it doesn’t happen that often, I should have enough reserve in the larger plan to absorb a goof for one sprint.
Any time in the course of events someone in an organization realizes they have overcommitted to something bigger, such as a quarterly planning cycle, they should have a conversation about what to do about it. I suggest the best thing to do about it would be to reduce some scope. They could renegotiate the date, but extending the date isn’t really in the spirit of smaller batches, fast feedback, MVP, or agility. They could mandate overtime, but that usually isn’t good as it often destroys safety and morale and quality. They could add people, but that often makes projects later and isn’t in the spirit of keeping teams together (stable), another practice I recommend.
Why does your PO reject adding unfinished tasks to the next sprint? I would reject adding unfinished tasks to the next sprint without dragging the unfinished story along with the tasks, or without splitting the story. I assume the PO still wants those tasks done. If so, what choice does he have?
Jamie Smith
If you don’t change your story points, when you come to use that as your golden reference point to relatively size future stories, surely it would be wrong? Or are you saying, the team recognise when a story sizing is wrong and avoid using that particular one as a golden reference point?