Replacing Backlog Grooming
The Problem
Over the last few years, I’ve worked with numerous teams. One thing they all struggle with is backlog grooming. They all know they need to do it. Unfortunately, they all seem to struggle with when to do it or who should do it. The most interesting struggle with backlog grooming happened two years ago. The “story time” meetings took place at the beginning of a month-long sprint. The manager stated, the work to be completed and delivered during that sprint had to be refined within the same sprint. This helped explain why the team thought they needed month-long sprints.
When I asked why they would try to refine work the first two weeks of the sprint and then complete that work the second two weeks, you know what their answer was? “It said to do it like that in the Scrum Guide!”
After I clarified their misunderstanding, we established a cadence to continuously mature the backlog. A few select people would participate in the scheduled meetings. We would reserve capacity from each sprint to get that work ready for future sprints. The team was able to shorten their sprints to 2 weeks. They more than doubled their delivery rate without increasing defect rates. With that as an example, over the last few years, I have evolved my practice of backlog grooming.
Let’s look at some key dates in the evolution of backlog grooming.
Evolution of Backlog Grooming
2005: “grooming the product backlog” is mentioned by Mike Cohn on what is now the Scrum Users Yahoo Group;
I always have teams allot some amount of time to “grooming the product backlog” to make sure it’s ready for the next sprint.
2008: A formal description of “backlog grooming” is given by Kane Mar, under the name Story Time, and recommending it as a regular meeting
I call these meetings “Story Time” meetings….Although they are not a formal part of Scrum, I’ve found that Story Time greatly improves project planning and reduces confrontational planning meetings, which are all too common for many teams. A Story Time meeting should be held at the same time and location every single week and involve the entire team, including the Product Owner and ScrumMaster. The sole intention of these weekly meetings is to work through the backlog in preparation for future work.
2011: The practice of “backlog grooming” is called “backlog refinement” and promoted to an “official” element of Scrum with its inclusion in the Scrum Guide
Product Backlog refinement is the act of adding detail, estimates, and order to items in the Product Backlog. This is an ongoing process in which the Product Owner and the Development Team collaborate on the details of Product Backlog items. During Product Backlog refinement, items are reviewed and revised. The Scrum Team decides how and when refinement is done. Refinement usually consumes no more than 10% of the capacity of the Development Team. However, Product Backlog items can be updated at any time by the Product Owner or at the Product Owner’s discretion.
2014: Derek Huether from LeadingAgile evolved the practice of backlog grooming with one of his clients, to allow the practice to work better at scale, calling it a “Progression” workshop.
When operating at scale, my client deals with different problems than a standard Scrum team. They’re dealing with separate lines of business. They’re dealing with multiple delivery teams for each line of business—to include external vendors—not to mention a portfolio roadmap that has annual plan items and budgets. Our strategy encapsulated the entire product delivery value stream, while ensuring we had enough architectural runway. We progressed work to be consumed by delivery teams, via a series of workshops.
Progression Workshops
Our Progression Workshops differ slightly from the story time meeting detailed by Kane Mar and the refinement meeting mentioned in the Scrum Guide. Counter to Story Time, we don’t invite the entire team. Instead, we have elevated the workshop to a group some have come to know as a Product Owner (PO) team. The group of people within the PO team will vary, depending on the line of business. Yes, there will be a Product Owner (Product Lead) and facilitator, but from there we’ll include the development lead, testing lead, and an architect.
We’ve found two key challenges when operating at scale. First, is there a well defined backlog that is ready enough to be consumed by different delivery teams? Second, is the work being queued up to the delivery teams free and clear of other teams. That is, have we decomposed it in such a way that we’ve minimized dependencies on other teams? Beyond that, in order to achieve some degree of architectural runway, we continually refactor existing platforms. Architectural changes are not only made incrementally but we require an architect to be present at every progression workshop.
Counter to the Scrum Guide, I’m not going to be proscriptive as to how much capacity the PO Teams do/should commit to progression workshops. The goal is to have enough work ready for delivery teams to consume for a few sprints.
When progressing work, we do expect some artifacts to be generated, to contribute to the teams understanding of what will be developed, tested, and delivered. Below is a partial list of potential artifacts. To be clear, we do not expect all of these to be generated.
Potential Deliverables
- System Context Diagram
- Dependency and Risk Work Items
- System Architecture Guidance Acknowledgement
- Use Case Diagram and document
- Business Process Flow
- Known Business Rules
- High Level Technology Alignment
- Architecture Backlog for Planned Work
- As-Is Data Contracts
- Feature Work Items Assigned to Delivery Team
- Feature Business Value and Acceptance Criteria
- Feature Stack Rank
- Test Strategy
Conclusion
So, that’s the high-level view of the Progression Workshop. Most of the time, a feature will require two or more progression workshops before work is ready to be consumed by a delivery team. Once features progress to a defined level of shared understanding, the delivery teams assist in the decomposition of features to user stories. In this way, work is decomposed to the right level of detail for each delivery team.
I’m curious, how have you scaled feature development and backlog grooming in your organizations? What mechanics outside of the standard Scrum process have you found useful to refine work to be completed by delivery teams? Have you evolved Story Time or backlog refinement?
Comments (9)
Larry Apke
I have always separated backlog grooming from story grooming. I suggest to the teams I coach that they spend one hour a day in team-related activities. The first 15 minutes is for daily standup. The remaining 45 minutes is scheduled for the other ceremonies – formal iteration planning, review, retrospective, backlog grooming and story grooming. Given this schedule there are (in two week iteration) seven days for backlog and story grooming. Backlog grooming takes one or two of these meetings and concentrates on the backlog in its entirety – the proper sequencing (ranking/priority) and sizing (points). The remaining meetings are used to groom the stories which consists of understanding acceptance criteria and taking a first cut at implementation strategy (ie tasks and hours).
I have found this regular schedule works very well for teams. It gives us ample time to do our scrum ceremonies and understand upcoming work so that our formal iteration planning meeting is effective and friction free. It also allows team to know when we will take time from them and when they should have time to do their implementation work, maximizing their time so they can focus.
I have written about this a few different times on my blog – http://www.agile-doctor.com – and encourage those who might be interested to check it out.
Larry,
I like your solution. I see Agile Teams like I see kids. Both want their freedom but I see they function best with some structure in place. Be explicit, get a shared understanding, operate within the constraints.
I’m looking forward to reading your blog post.
Charles Bradley
Hi Derrick, overall good article as Product Backlog Refinement is definitely a key practice that will improve flow and quality. I commend you there.
However, I want to challenge you to go one more step in your already great improvement efforts…
> Yes, there will be a Product Owner (Product Lead) and facilitator, but from there we’ll include the development lead, testing lead, and an architect.
>…
> Architecture Backlog for Planned Work
>…
> Feature Work Items Assigned to Delivery Team
> work is ready to be consumed by a delivery team.
>…
> enough work ready for delivery teams to consume for a few sprints.
>…
> Once features progress to a defined level of shared understanding, the delivery teams assist in the decomposition of features to user stories.
Why isn’t the “delivery team” involved from the beginning? Why don’t the technical members(lead developer, architects, lead testers, etc) of the “PO Team” come from the Scrum teams themselves? Why is there a need for a handoff? Isn’t there waste in these kinds of handoffs? Why isn’t the PO working directly with the Scrum Teams on this?
Much of what you talked about smacked a lot of waterfall to me… a separate “requirements team”, technical members involved in refinement that are not on Scrum teams themselves, not letting the Scrum teams self organize to choose who helps refine and how it is done, etc.
So, please accept this as a polite challenge to take your improvement steps even one step further! (as this is how it is intended I assure you, polite challenge to go higher, I have no axe to grind here)
For some guidance on Refinement at Scale, I’d recommend you look to Large Scale Scrum, specifically here:
See #’s 5 and 6 on Page 9 of the PDF here:
http://www.crosstalkonline.org/storage/issue-archives/2013/201305/201305-Larman.pdf
Also, see:
http://bit.ly/1ooh4Pr
(See sections on Requirement Workshops and Joint Refinement Workshops)
Anyway, hopes this helps in your improvement journey. It sounds like you’ve helped make some great improvements already. I wish you well!
Charles Bradley
Charles, I can see now that I didn’t describe the Product Owner (PO) team well enough in my post. The members of the PO teams are coming from the delivery (Scrum and Kanban) teams. There isn’t one single Product Owner team. Thank you for calling me out on that. There are members of the delivery team who do attend the workshops, mostly to play the part of the Napoleon Corporal. (I call them that, not the client)
For those unfamiliar with the term:
Charles, thank you very much for your insights. I’m going to check out those links you posted.
I’d recommend that if we are going to continue this conversation, we set some context. The size and scale of the organizations we work with and the level of product complexity in the companies we are trying to help is beyond ‘getting the delivery team involved from the beginning’. That is the problem with these kinds of debates… unless you’ve walked in someone else’s shoes, it’s difficult to have a relevant point of view. Scrum is an interesting theory that only works as described in some contexts. LeSS is describing an idealized end state that cannot be applied out of the box in most of the organization we work in. Scrum as a base package is not scaleable. SAFe fundamentally punts on solving the real problem, but can be effective in some large contexts, but will limit agility over time. I cannot emphasize enough here that unless you’ve been on the ground supporting the specific customers we are supporting, it’s tough to have a relevant perspective. There is no theory in what we are talking about, only on the ground pragmatic dealing with reality.
Charles Bradley
For some reason I wasn’t able to respond directly to your comment Derek, tried clicking the “reply arrow in a circle thingie” but that didn’t do anything.
Anyway, thanks for your response. I’m glad to hear that the delivery teams are self organizing to choose who participates in the refinement sessions.
The Napolean Corporal metaphor, while well intentioned, leads me to believe that there is some large hierarchy difference(Generals vs. Corporals) between PO’s and the delivery team members… I’m sure that’s not how you intended it, but I’m also not sure the metaphor sends the right message. I also recognize that all metaphors break down at some point, but implication of hierarchy is kind of antithetical to self organization and collaboration among equals. I see PO’s and Dev Team members as equal partners in a collaboration wrt requirements.
(That might be the first time I’ve ever used the word “antithetical” in a sentence, had to look it up to make sure I’m using it right!)
Hey Charles, thanks for the follow-up. To the navigation bug, I submitted a request to get that fixed. Thanks for the QA. :-) With this client, the PO’s are members of the delivery teams but there is certainly a skills gap. Though there are efforts to coach up members of the teams, the fact remains the core participants of the Product Owner teams have much more institutional knowledge that some of the members of the delivery teams. We need representation from those teams to make sure the work it detailed out enough so they will be able to consume it with minimal delay.
I hear your concern about the General vs. Corporals. The metaphor does break down a little. Since I have a military background, I would describe the difference being closer to Corporals and Privates. To that, a few Staff NCOs with a little more field experience occasionally joins the workshop to provide insights to the terrain of an upcoming mission.
I had to look up antithetical. I don’t think hierarchy is antithetical to self organization. Even as a Corporal in the Marines, I could direct a group of other Marines. I could communicate the mission. The other Marines were self organized around the mission. They just may not have an much field experience as myself and therefore have a lower rank. I still have to be able to communicate the mission to the point they can be self organized.
Charles Bradley
> They just may not have an much field experience as myself and therefore have a lower rank. I still have to be able to communicate the mission to the point they can be self organized.
In a military context, I would agree with you because generally speaking all people in the military are trained in basic war fighting skills at least in the early part of their career… HOWEVER, in software, that is generally not true, and especially not true the higher you get in the hierarchy. In addition, technology changes much more rapidly than basic warfighting skills.
If mgmt in our industry was more “Manager-Teacher” in the Lean sense, that would be true — but I find that vastly untrue in industry, *especially* in big hierarchy organizations.
The cool thing is that big hierarchy organizations are dying off at an epidemic rate, so that they’ll be dead before long anyway. I just hope that we lead them in a way that doesn’t make them susceptible to that epidemic. For more on this, see Steve Denning’s articles about the declining average age of fortune 500 companies.
Your point about enabling self organization is well taken. I just wonder if these teams are truly self organizing or if they are being led by a hierarchical “powerful council” called a PO Team. I guess I would look to see the trend over time of being less dependent on people with “special skills” or “special domain history” like you describe. If that trend was going down at a reasonable pace, I guess I’d be more ok with it.
What I was missing from your article was recognition that these people are on a path, and they have more to their improvement path that they are also working toward. (next couple of steps to get even better and more truly self organizing)
Charles Bradley
Btw, Derrick, I thank you for your service to our country. That’s a big deal to me. I was just on a military base recently teaching some of our troops about Scrum and Evidence Based Management. It was very fulfilling for me, and I was thrilled to be serving *them* for once. I also reminded them that Scrum was created by two former military members. (Jeff-USAF, Ken-USN)
So, Semper Fi, Hooah, and Hooyah!
(I’m not former military, just have a lot of relatives/friends who are)