Encapsulation vs Orchestration: Dependencies in Agile

Dependencies will stop Agility dead in its tracks. In the presence of dependencies, you’re never going to achieve the level of Agility you desire—unless you do something about them.

So you only have two choices when it comes to this. You can either encapsulate the dependencies within the team or within the value stream, or you can orchestrate them. Another way to say this is that you either must break the dependencies, or you must manage them.

Either way, if you have dependencies, you cannot pretend they don’t exist.

The Assumptions of Scrum

So, what does Scrum say about dependencies? It assumes you don’t have them in the first place. And if you look at the way that Scrum is architected, you’ll notice that it assumes a lot, including that each team:

• Consists of six to eight people
• Focuses on a particular business problem
• Owns its technology stack
• Has a singular input
• Has a singular measurable output

In effect, Scrum assumes that you have encapsulated teams that can address any dependencies internally by self-organizing around the ceremonies of Scrum. They can then either resolve or prioritize dependencies in real-time to ensure that the team can deliver on their commitment by the end of the Sprint.

But what happens when the dependencies are outside the purview of the team?

The Reality of Scrum

Sadly, the reality for many large organizations is that when forming teams, it’s all too common for a select person or people to be jumping between multiple teams.

For example, a QA person is spread out over six teams. That’s six Sprint Planning sessions, six daily standups, six reviews, and retrospectives. At that point, there’s no room for QA work.

Each team then has to make sure that person is available to them when they need them to do the work. They’re no longer instantly available to that team, which means the team now has planning dependencies and coordination dependencies. There’s a dependency between one project and another based upon either project’s ability to stay on track.

Theoretically, you could have two teams that meet most of the criteria of an Agile team. But if you have only one QA person straddling the fence between two groups, you also have a dependency, and that’s going to limit the ability of both teams to achieve Agility.

Any time we inject a dependency into the system, we create something that has to be managed outside of the team. And that limits our ability to be Agile.

Failure Modes of Other Popular Methodologies

Many methodologies have acknowledged that dependencies are real and have come up with various ways of addressing them. But it doesn’t matter which methodology you choose because they’re all just ways to orchestrate dependencies at the end of the day.

Here is a closer look at how some of the most popular methodologies, Scrum, SAFe, and LeSS, can attempt to manage dependencies, yet usually still fail:

Scrum of Scrums

At one point in time, Scrum came up with this idea of Scrum of Scrums. The idea is that you have some number of teams, each with their own ScrumMaster. At some point, usually at the beginning of the Sprint, all the ScrumMasters meet up to coordinate the dependencies in real-time. This is a valid construct that works under some conditions.

The problem is that when you have two teams with a dependency, and you’re trying to mitigate them in real-time, you end up with late resolution of the dependency. And that tends to destabilize the velocity of the teams. When the velocity of the teams become destabilized, you end up unable to make and meet commitments.

A lot of organizations put buffering strategies into place, like setting aside extra time to deal with known dependencies. But it becomes very ad hoc and hard to manage as you try to scale the system. So, while Scrum of Scrums is a fine communication tool, it’s not a good strategy for eliminating dependencies.

SAFe

SAFe suggests that you need Program Increment (PI) Planning. Basically, you’re going to have some sort of early “big room” planning session on one end of the PI Plan and a closing ceremony on the other end. In the middle, you’re going to deliver features on some kind of cadence.

When you do an early planning meeting, the idea is that everyone’s going to get in a room and address the dependencies in advance, at least to some reasonable level of assurance. But they’re going to understand those dependencies at the beginning and hopefully front load them so that they can resolve the dependencies sooner. Or, if they’re going to fail the PI, then at least they’ll know that sooner and can pivot or reprioritize.

Where SAFe struggles in large organizations is when you have something like a banking platform. This platform is responsible for things like ACH processing transactions, reconciliation, bill payment, etc. Typically, these platforms are dealing with incredibly high volume, large, robust mainframes, and super-fast transaction processing.

Imagine if you have four products, each with seven to ten SAFe teams, and each value stream is potentially injecting dependencies into the platform So, what you’ve done is simply moved the dependencies one degree out. You haven’t done anything to break them. They’re still there, and they’re still limiting your ability to be Agile.

What if the dependencies are encapsulated within the value stream? If that’s the case, SAFe might work for you in your situation. But be on the lookout for the presence of dependencies to make sure that’s always a true statement. Because if you have value streams that intersect, meaning that you have dependencies across value streams, you might not have your value streams set up right.

Or maybe you do, and these dependencies actually exist, and you’ll need to manage them across value streams.

SAFe typically is set up with a 4-tiered governance model. But that model only works if each layer is orchestrating the dependencies for the layer directly below them. And that level of orchestration gets expensive, and you wind up with a hierarchy of teams.

Dependencies are not your friend. Even if you get to a place where you can effectively manage them.

LeSS

LeSS talks about team-level execution and the way it rolls up into bigger things. But to do a LeSS implementation, you’d have to have a tremendous amount of encapsulation at the team level and a very minimal amount of orchestration as you go up.

So, how would LeSS address dependencies?

Well, let’s say you have a few front-end teams and a few backend teams. You might have a PO team who is decomposing and sequencing the work across these teams and who’s operating off a dependency-aware backlog.

The PO team starts loading all the teams up with Feature 1 items. Then Feature 2 items. But where it gets interesting is when you get to Feature 3 and not all the teams have work to do on that Feature. What do you do with teams that don’t have work, yet have capacity? Do you move them on to Feature 4? If the teams that don’t have capacity for Feature 4 work, you can’t.

So, you wind up in a situation where each team can either be a bottleneck or starved for work. The inclination is to load up the teams that have capacity with non-value-added work just to keep them busy, but that’s probably not the best use of their time.

Ideally, what you’d want to do is move the capacity to the bottleneck. But, in reality, this often isn’t reasonable due to the distribution of skills across teams. The skills of one team might not apply to the work the other teams are doing.

So, why not just collapse them into one complete, cross-functional team that can service all the components?

What if the backend is in Cobol and the front-end is in .NET? You’d have a multiple technology problem.

On the other hand, what if you’re in an environment where it’s not just about technology and skillset, but the work is also mission-critical and can never break?

When you get into these pure-play LeSS environments where any team can touch anything, you have to ask yourself, do you have the right level of test-safety? Can you continuously deploy it? Can you be sure that it’s never broken?

Do you have the right architecture in place, the proper organizational alignment to operate this way, and do it safely? If the answer to any of these is no, then you have dependencies.

Compensating Controls

As you can gather from what you’ve read thus far, dependencies are a real thing that organizations are dealing with. And after seeing some of the common failure modes of the most popular methodologies, hopefully you’re starting to see that while most of them are attempting to address this issue, not all methodologies work in all environments. So, what do you do when your methodology of choice doesn’t prescribe a course of action for your particular environment?

To be truly Agile, by definition, you have to have encapsulated teams. That’s the ideal state, but it’s also probably not your current reality.

And if it’s not your current reality, then you know you need to manage the dependencies, but a lot of people don’t like that because it doesn’t feel very Agile.

So, there’s this tension that organizations are feeling, and they’re asking themselves what can I do about it?

The answer is you have to put some sort of compensating control in place. What’s a compensating control? A compensating control is any orchestration mechanism you implement on top of your chosen methodology to help you get to your desired end state.

How Agile Do You Want to Be? And How Quickly?

Understanding where you are today vs where you’re going and acknowledging that dependencies are going to get in the way means that your Transformation journey isn’t about implementing a methodology that’s too heavy for your future end state, or establishing a set of compensating controls that are going to hinder your Agility in the long term.

Instead, your Transformation journey is about creating the right conditions today that will help you begin incrementally and iteratively creating the right conditions for tomorrow.

The reality is that none of this Is going to happen overnight. But, knowing where you want to go and having a plan for the interim, allows your organization to decide which dependencies are worth the money to break. There will surely be dependencies that may never be worth the money to break, or aren’t economically viable enough to break.

So, you’re going to have to do something on top of your methodology to ensure you’re reaping the benefits of the methodology until it becomes reasonable to break the dependency.

When you start to view your Transformation through this lens of encapsulation v orchestration, you begin to see that each methodology is just a different way of attacking the same problem at varying degrees of heaviness.

As you begin to strike a balance between encapsulation and orchestration, you might realize that your ideal end-state is some sort of multi-methodology environment that depends on the needs of the teams and its application.

Either way, once you’re in this mindset, you can start to make better decisions to get you to your desired end-state, and you can make strategic decisions about which dependencies to break. Meanwhile, you have controls in place to maximize Agility in the parts of your organization where you still have dependencies until it’s time to deprecate those controls.

Organizational Impact of Dependencies

An organization that doesn’t understand how to balance encapsulation and orchestration doesn’t have a viable way of managing or resolving dependencies. So, they inevitably wind up doubling down on methodology as well as team-level and work-level practices. And they end up not being able to deliver on the other fundamental building blocks.

Because when you have unmanaged dependencies, you’re not going to be able to establish a stable velocity at the team level, so the organization isn’t going to know how much demand to put into the system, so the net effect is that the organization is going to overload it.

And when you can’t know the teams’ ability to deliver against the known backlog, it makes it very hard to make informed bets and tradeoffs as we go based on the known capacity of the system so that we can level the demand against it.

Then we get to the point where the teams just aren’t trustworthy, and so the organization has a hard time trusting them. Many Agilists say that we need to trust the teams, but what we really mean is that we need to put the team in a position where they can be trusted. This primarily means that we have to create the right conditions for them, which means we have to create an ecosystem in which they’re encapsulated.

Dependencies kill Agility, So, you have to either find a way to encapsulate them within the team or find a way to orchestrate them.