Compressed Backlog Refinement

Lots has been written about backlog refinement (what we in the US used to call grooming), and a lot of it is good. There is lots to say about this practice. However, I’ve not seen any treatment on whether you should do it differently for your initial backlog. Therefore, I’m setting out in this post and in the next to answer these questions:

• How do you refine and estimate the initial backlog?
• How do you refine and estimate additional stories or subsequent backlogs for work that comes along later?
• Wouldn’t those approaches be the same?

Usually, when I spin up a new agile team, they don’t have an initial backlog. Nevertheless, the parent organization just about always knows what they want the team to do The vision or mission or charter is there–though it might not be communicated clearly. Sometimes there are documented requirements in what is often called a PRD, MRD, or BRD. Whatever the case, if there is no backlog of user stories, I typically hold a story writing workshop to get the ball rolling. Now that we have a crude backlog, it needs to be refined.

This post is not about where the ideas come from or even about how to convert what already exists into user stories. The focus here is about how to conduct that initial refinement meeting, then what might be different in subsequent refinement sessions.

For the initial refinement, I recommend compressed refinement. After that, I recommend continuous refinement.

Compressed Backlog Refinement

When refining that initial backlog, often after spinning up a new agile team, there is usually someone in the chain of command wondering, “Why isn’t anyone coding yet?” There is a great deal of pressure to just get started already. The team is probably anxious to get started too, unless they are still trying to wrap up the prior project. At the same time, we want to know when we expect to get done with this new work,  when we can get to the next release or how much of the backlog we’ll have when the release date rolls around. It’s not sufficient for my clients to tell them they’ll get what they’ll get when they get it.  My clients need predictability. We also need to refine the backlog to identify dependencies, risks and items with long lead time. Anyway, it takes some time to create, refine and estimate a backlog, to form teams, to do the sprint-0 stuff and get rolling. We want to do a good job refining the backlog, but we can’t take forever to get it done.

Therefore, we do a compressed approach to backlog refinement.

Stories are written, we have a backlog of crude stories, and we need to refine them all quickly. In the compressed refinement approach, we’ll have a series of refinement meetings. It could be half-day to full-day meetings for the better part of a week. The whole team attends. Please bring in lunch, preferably a good lunch. You may find yourself updating stories in your agile management tool in real-time during the meeting. That’s generally unproductive, so strike the right balance. Some edits might have the whole team collaboratively involved, arguing over wording and meaning. Other edits can be done by someone else in the room on the side. Of course, other edits will be made offline.

This compressed approach tends to be very detailed and slow because we’re going from crude stories to well refined stories during the meeting. Since we’ve talked about them in such depth, we might as well estimate with planning poker as we go.

Because our story discussions are so detailed when using this approach, you might be able to get by with an abbreviated sprint planning meeting for the first 2 or 3 sprints.

Punctuated Refinement

Organizations that operate in project mode rather than product mode tend to do this compressed refinement approach over and over for each project that comes along. They spend little time getting the backlog ready for the next project while the current project is still underway.

This is backlog refinement done in a big batch. This is rolling wave planning with big tidal waves. That’s usually not the best approach. After this 1st compressed refinement, let’s move on to continuous refinement.

Not Talking About Progressive or Elaboration

I’m not talking about progressive elaboration or even elaboration. Elaboration in an agile sense is more about decomposing epics into features and features into user stories and then providing the details for the stories (the acceptance criteria) and their visual specifications. Adding the term Progressive to Elaboration means we do it more or less just in time over the course of time.

Here’s how we may do both Progressive Elaboration and Compressed Refinement: We may still progressively elaborate epics and feature, then have an intense period of backlog refinement, kick off a project, yet still do more progressive elaboration of user stories, their acceptance criteria, and visual specifications over time. What I’m writing about is whether we refine and estimate most of our stories in a compressed period of time, say, a week, maybe two or whether we do that refinement over time and further in advance.

Recommendation

There are some downsides to this. A compressed refinement does not allow for the long lead-times that may be necessary to do some research, figure some things out, learn some new technology, put the necessary architectural run-way in place, work out contracts with 3rd parties, or get other groups to build out dependent pieces. Also, we sometimes need additional time for things to soak in. We need to take a step back and look again at the big picture. Because of these things, we may end up with a poorly refined backlog or a poor plan.

For these reasons, I recommend making the switch from project thinking and punctuated refinement to product thinking and continuous refinement. That’s the topic of my next post.