Toward a Continuous Flow

I’ve observed that most software delivery processes in the “real world” are hybrid processes. It’s rare for an organization to follow a defined method to the letter. That said, there seem to be four common reference models, and real processes often lean more heavily on one of these than the other three:

• Linear
• Iterative
• Timeboxed
• Continuous flow

The Linear model is the canonical basis for so-called “waterfall” approaches. The key underlying assumption is that effective software delivery requires work to follow a predefined sequence of steps. If everyone “does their job right,” a single pass through these steps will result in a good outcome. Due to historical issues beyond the scope of this post, few people today consider a purely Linear process to be a practical way to achieve good results. However, linear thinking still informs management and executive assumptions about how software is best delivered. Most other approaches, in practice, include some degree of linear thinking.

The Iterative model is the most widely used, and there are many variants. The key underlying assumption is that a single pass through the requirements will yield deeper understanding of the problem space, but multiple passes are necessary to create a viable solution. Some iterative methods call for an incremental refinement of the solution, starting with a prototype and building from there. Others call for incremental delivery of “vertical slices” of functionality until sufficient features are in place to satisfy customer needs. Some processes define a linear analysis and setup phase followed by a series of development iterations, and ending with a linear hardening and delivery phase. There are numerous branded and published variants of the Iterative model, and even more home-grown, unbranded variants.

Today, the Agile movement dominates industry thinking about software delivery methods. The Agile community as a whole regards the Timeboxed process model as de rigeur. Many Agilists cannot conceive of any other way to deliver software. The Timeboxed model is similar to the Iterative model with two additional (and definitional) characteristics: (1) Iterations are the same length, and (2) a potentially-shippable solution increment is produced in each iteration (at least). Several Timeboxed processes are currently in widespread use.

For several years, Lean Thinking has been influencing people’s thinking about effective software delivery methods. The Agile community has embraced this, and most rhetoric about Agile now makes reference to Lean, as well. A key underlying assumption in applying Lean Thinking to software delivery is to establish and manage flow. A practical goal is to achieve continuous flow, or to get as close as possible to it. Continuous flow is generally recognized as an improvement over the timeboxed approach.

With the majority of delivery teams currently using a timeboxed process, a practical question is how to move from a timeboxed process to a continuous flow process. I can share my own experiences in coaching delivery teams toward a continuous flow process, for what it’s worth.

Parkinson’s Law

Parkinson’s Law holds that work expands to fill the time available for its completion. When teams adopt a timeboxed process model, they settle into a routine that guarantees the work they carry out in the course of an iteration completely fills that iteration. (That’s actually true for any process model a team uses.)

Let’s say a team shifts from a Linear or basic Iterative model to a Timeboxed model. They begin with four-week iterations. After a bit of learning curve time, they settle into a routine that involves spending X amount of time in iteration planning, Y amount of time refining their backlog for future iterations, and Z amount of time on demonstrations and retrospectives. No matter how they conduct these activities, they will exactly fill the four-week time slot.

Driving down the iteration length

An effective way for a team to challenge itself to improve delivery performance is to shorten the iteration length. Now the team must complete the same amount of work at the same level of quality in three weeks instead of four. They will find ways to simplify and streamline the standard iteration management activities so that they can get them done in less time. Eventually, they settle into a routine that exactly fits the three-week time slot.

One they’ve become comfortable with a three-week iteration, it’s time to move out of the comfort zone once again. They repeat the process until they’ve settled into a comfortable two-week cadence. It’s been my experience that it’s quite feasible to streamline the “standard” sort of iteration management activities to fit into a two-week time slot. Indeed, many teams begin with two-week iterations.

Asymptotic limit of iteration length

That’s not a proper use of the term asymptotic limit, but I enjoyed typing it.

I mean to say there’s a point where the cost of iteration management overhead exceeds the value added by frequent feedback loops. And yes: it’s overhead. The fact your team plans iteratively helps them to deliver incrementally, but that doesn’t mean the planning activity itself is value-add for the customer. You won’t win a trophy just because you can check the box that says, “Iterative planning.” It’s all about delivery. As a team learns how to deliver the same value in shorter and shorter iterations, they usually do so by reducing the time and effort necessary to support iteration management overhead – activities such as iteration planning, iteration reviews, and heartbeat retrospectives.

Some of that improvement is probably obvious – you don’t need to spend as much time to plan a two-week iteration as you do to plan a four-week iteration. But a good deal of improvement comes from learning to achieve the goals of these activities in simpler ways. Some of that ties back to improvements in the team’s work practices.

Getting there

For example, learning to slice User Stories into a more-or-less consistent size eliminates the need for sizing the stories during planning, and also simplifies the collection of metrics. Similarly, combining traditionally-separate activities reduces the complexity of the delivery process – for instance, collapsing coding and testing into a single activity, or specifying acceptance criteria in an executable form that serves as an acceptance test suite. Building cross-functional skills on the team also saves time by reducing coordination and communication overhead. Automating the deployment process means the team requires less time to reach a meaningful definition of done for stories.

Through a combination of different areas of improvement, teams can achieve a pretty smooth work flow within the context of their timeboxed process. It’s been my experience that the point where teams begin to question the value of the timeboxed model is a one-week iteration length. That appears to be the point where the value line crosses the cost line.

Being there

Once the team reaches a point where they can justifiably question the value added by fixed-length iterations, they’re probably already delivering in a very smooth way. This could be a logical time to establish a continuous flow process “officially.”

What about teams that aren’t invested in a Timeboxed process already? Some people assume it’s necessary to begin with a Timeboxed process and “earn their way forward” before they can adopt a Continuous Flow process. That’s not really the case. You can shift into Continuous Flow at any time it makes sense for you. If you’re already using a Timeboxed process, then the sequence of steps outlined above may be a good fit. But you can try any process you please any time you please.