Agile vs Waterfall
Agile vs Waterfall
These words have become completely overloaded when discussing product development. Lots of conversations about helping organizations improve their product development processes go sideways based on individual perspectives about the meaning of Waterfall and Agile. At this point these words don’t provide a distinction that is helpful when we are trying to figure out how to build products in organizations. This is a red-herring contradiction. When I hear someone say “we need to do that Waterfall” or “that wouldn’t work in Agile” or “we can use Agile for that project” then I want to stop and ask “what does Agile mean to you?” or “what do you mean by Waterfall?”
I want to break down this debate into a clearer discussion around specific characteristics; Type of Effort, Upfront Planning, Sequencing and Feedback, and Composition of Teams. Then talk about how understanding these characteristics helps us improve the ability to be predictable and achieve the fastest time to ROI.
Type of Effort
Product Development encompasses different types of efforts. We frequently view Agile as the model to do exploratory product development. In exploratory projects, teams are exploring new ways of solving a problem or trying to discover a new customer or market segment. Mike Cottmeyer has been calling these divergent projects. Fast time to value involves rapidly performing lots of experiments, getting feedback on your hypothesis, and adapting your approach.
At the other end of the spectrum are convergent projects. In a convergent project – there is a scope, a due date, and a budget. Agile is a very strong approach for delivering convergent projects. Organizations doing convergent projects need teams to have a pretty strong ability to make and meet commitments and they have a significant need to manage risks and coordinate dependencies. These organizations can make the trade-off decisions necessary to maximize the chances they will converge on the best outcome.
Most of the organizations we deal with need to have a pretty clear idea about what they are going to get — and when they will get it — before they decide to do a project. They are primarily performing convergent projects. You manage convergent and divergent projects differently. A frequent challenge we see is when organizations move to Agile, they treat all projects like divergent projects but they are actually in convergent projects.
Upfront Planning
Not planning isn’t an option. The question is how much planning do you need to do to be successful? It isn’t possible to be predictable if you don’t do a sufficient amount of planning. The red-herring in Agile is that you don’t do any planning. The Red-herring in Waterfall is that you get all planning done up front. Regardless of your approach to the project, base the amount of upfront planning on how much you need to do to answer the questions of, “What will I get?” and “When will we finish?”.
Sequencing and Feedback
As I stated above, most organizations need a pretty clear understanding of what they are going to get and when they will get it. We see different modes of sequencing and feedback. Many organizations accomplish predictability by sequencing the work in phases. For example, Analyze, Architect, Design, Construct, Integrate, and Test/Remediate. The intention is to make good decisions upstream that protect downstream efforts.
Testing and integration feedback, deferred until late in the project, is usually the problem. Challenges identified late put the project under duress at a point when it is difficult to respond effectively. This is the project that takes as long to complete the last 20% as it took to deliver the first 80%. In phase based projects it is very difficult to be predictable because risks tend to manifest late in a project. Time to ROI is negatively impacted when project durations run past the planned delivery.
The other form of phasing and sequencing leverages frequent delivery of working tested product. When we sufficiently plan, sequence frequent deliveries to maximize the potential for success on the project. Resolve risks and dependencies early, while there is time to respond to them. Deliver more valuable work earlier in the project. Provide the opportunity to explore options and stop after sufficiently solving the problem. We gain insight from frequent deliveries and this insight is used to make trade-off decisions to maximize return on the project. Projects using value based sequencing can be more predictable than phase based projects. Time to ROI can also be maximized in value based sequencing in those cases where a smaller subset of the potential set of features can be deployed to deliver value.
Composition of teams
We see people belong to functional groups and are matrixed into various project teams, in most organizations. It makes sense when the sequencing is in sequential phases. Individuals are often spread across multiple projects simultaneously. Using this method, we can maximize the utilization of everyone in the organization.
In another model, people belong to collaborative teams that work together to deliver working, tested product frequently. In this approach, team members have almost instant availability to each other and everything needed to deliver its next increment. People are generally dedicated to a single team so they can work collaboratively and with near immediate availability on each frequent delivery. Co-located teams help with this immediate availability.
Team composition plays a crucial role in the ability to leverage value-based sequencing. The collaborative team composition approach may have a negative impact on utilization. (I disagree with that assumption, and I’ll tell you why in a follow up blog post next week). The value gained from increased predictability and faster time to ROI often outweighs the impact of lower utilization.
Summary
Arbitrary references to Waterfall and Agile don’t provide much useful information. Be as predictable as possible and maximize time to ROI. Have clear conversations around team characteristics that are best suited to the problems we are solving. Understand if your project is convergent or divergent and determine the appropriate amount of planning. Assess if your project would benefit from phase based or value based sequencing, and whether functional separate or collaborative teams are the best way to build software.
In my experience, Agile is about collaborative teams and value-based delivery. I prefer this approach on divergent projects that require little governance and upfront planning. I also prefer this approach on convergent projects that require significant governance and upfront planning. So I am very comfortable with Agile – whether the project is a divergent or a convergent type of effort and regardless of the appropriate amount of upfront planning. Not everyone sees these the same way because not everyone has the same perspective of Agile. Let’s move away from red-herring conversations of Agile versus Waterfall and focus on meaningful characteristics.
Comments (11)
Jardena
Great article! This framework will keep countless conversations from spinning off course in our organization. We struggle daily with preconceptions about the word “Agile” to the point where we avoid the “A” word and replace it with specific practices or principles.
Adam
I’ve always thought the main disconnect has been that “Agile” describes a way of thinking whereas Waterfall is more of a specific process. My feeling is that Waterfall processes could be part of an Agile implementation. In fact, our project manager has long run projects with constant feedback, even while in a strict waterfall environment.
lee
Not planning isn’t an option. The question is how much planning do you need to do to be successful? It isn’t possible to be predictable if you don’t do a sufficient amount of planning. The red-herring in Agile is that you don’t do any planning.
Maybe you meant, “in Agile is that you don’t do any upfront planning.” Otherwise you have a paradox.
Thanks Lee. I am trying to create a distinction between how much and when you do planning and the term Agile. My point is that there is a perception that Agile means no planning – upfront or otherwise. We need to make situation appropriate decisions about what the right amount of planning is – and when it is best done.
Adam. At its simplest distinction – Agile thinking involves validating frequently – Waterfall thinking involves validating late. Everything else is on a continuum and is subject to these two underlying objectives.
Tim
i like that distinction as I had always considered Agile to be about making it up as you go along and less suited toward convergent projects
Elena Yatzeck
This is an excellent and pragmatic way to talk about your SDLC!
If it helps, I have taken to describing Agile as a philosophy which serves as an umbrella term for a lot of different methodologies which have their own names, rules, and merchandising revenue streams. Agile is existential, whereas methodology description can be either normative or descriptive, depending on whether you’re teaching a new team, or just providing tips to an experienced one.
Cheers, Dennis!
Elena,
Good to hear from you. I agree with how you parse the meaning of Agile. I like the distinction of normative or descriptive. But this also goes to my point. Most organizations we deal won’t benefit from existential and/or philosophical discussions of what is or isn’t Agile and Agile vs Waterfall. We need to be very pragmatic to move most organizations to action.
Let’s catch up soon.
Brad Kekst
Excellent article, great to hear someone get into specifics and not be religious about either methodology.
In the fourth paragraph is there any chance you transposed Agile for Waterfall (“In a convergent project – there is a scope, a due date, and a budget. Agile is a very strong approach for delivering convergent projects.”)?
Abhishek Sen
In terms of team composition, I have seen an issue with long running Agile projects which I have not experienced in waterfall. Consider an Agile project running for multiple years. Developers gain experience and mature, and expectations change and often managers are faced with the dillema of keeping the agile teams intact and making room for developers to grow and take on technical leadership roles. In a waterfall scenario, generally there are specified roles like Technical / Project lead / Architects – who perform somewhat different roles, and high performing developers with potential can aspire towards these. However in Agile there are no such roles, and over time the senior developers question thier career path and potential for growth in an Agile environment.