Why not just define the solution in advance?

If you’re familiar with our model of organizational transformation, then you know we’re fond of the metaphor of taking a journey in a specific direction, possibly (but not necessarily) ending up at the farthest imaginable point of that journey. We think of the journey as a series of expeditions, each of which aims to fulfill a portion of a vision and plan.

The metaphor is both spatial and temporal. When you picture a group of adventurers embarking on an expedition, the visualization is mainly spatial: They are marching across territory toward a goal that lies on the horizon. The horizon moves ahead of them as they march. Their concept of “the possible” depends on what they are able to see or imagine from their current position and, as they progress, they are able to see and imagine more and more possibilities.

A way forward based on Lean principles involves conducting a series of experiments. Learnings from each experiment inform the design of the next experiment. Always, there’s a goal in mind. Over time, outcomes meet needs more and more effectively. Improvement over time suggests a temporal angle on the “journey” metaphor.

Step-by-step Improvement Over Time

It’s easy to find examples of similar journeys that suggest change over time. One that I find relevant, particularly in larger, well-established IT organizations, is the tale of the Eddystone Lighthouse. You can read about it on Wikipedia. There are also many videos on YouTube about the lighthouse, and it has been featured on the Science Channel program, “Impossible Engineering.”

I see this as an example of a temporal journey of improvement because of the progression of engineering advancements reflected in the series of lighthouses built on the site from 1699 to the present. Similarly, improvement in organizational performance often involves building a series of solutions that incrementally move closer to strategic goals.

…it turns out to be faster, cheaper, and better to find the way forward through a series of experiments than to design the ultimate solution in advance.

Lighthouses and expeditions

It’s easy to get tangled up in a sea of metaphors. Even referring to the situation as a “sea” could be one metaphor too many, were it not for the fact we’re also talking about lighthouses.

This lighthouse at Eddystone was the the first to be built in the middle of the sea and was erected on a rock that which was submerged 20 hours a day. Over the course of history, it was rebuilt four different times, each quite different from its predecessors. Engineers had learned things and materials science had progressed, enabling each successive lighthouse to be better than those that had stood before.

The same pattern occurs in organizational transformation. A Scrum team on a journey from Basecamp 2 to Basecamp 3 will use the Scrum events and artifacts quite differently than a team progressing from zero to Basecamp 1. The more mature team will use Scrum in a lighter-weight fashion than the novice team. For example, they have learned how to level out their work by crafting User Stories of roughly the same size. They’re on their way to dispensing with story-level sizing. Meanwhile, the novice team may still be struggling with separating the notion of size from the notion of time, and they may have difficulty visualizing the possibility that story-level estimation is a crutch that can be made unnecessary by mastering other practices.

Also, the organization surrounding the two teams will be at different levels of proficiency with lightweight methods. You’ll often hear us speak of clarity around the backlog, or words to that effect. An expedition approaching Basecamp 3 will have learned skills in identifying worthwhile initiatives, prioritizing those initiatives, and refining backlogs that are sensible and actionable by program and delivery teams.

It’s more feasible for the delivery teams in the Basecamp 3 expedition to function in a lightweight way than for the novice teams, which are supported by organizations still early on the learning curve, still struggling to reach Basecamp 1. They may not receive actionable backlog items on a consistent basis. Everyone is trying to get a handle on quite a few unfamiliar concepts and methods. Even an advanced team would have challenges in maintaining flow and delivering value without appropriate support from the program and portfolio teams.

The two organizations just can’t build the same kinds of lighthouses. They have to advance one step at a time.

Why not just determine the final solution through research?

Sometimes, people are uncomfortable with this approach. They would prefer it if we could design the “final” solution in advance and then simply implement it. That way, they would have only one sizeable capital investment to make, and they could check the “improvement” box. All done!

The “implementation” approach may be feasible for relatively small enterprises that have fairly narrowly-bounded goals. When a larger enterprise, that has longstanding habits and entrenched processes, sets a goal to “be more effective” or “be more competitive” or “improve the customer experience” or “be able to pivot quickly,” it’s harder to visualize a Golden End State in a vacuum. Such goals are real and meaningful, but difficult to quantify, and the path to achieving them in the face of an ever-changing competitive landscape is not easy to discern.

Perhaps counterintuitively, it turns out to be faster, cheaper, and better to find the way forward through a series of experiments than to design the ultimate solution in advance. It takes less time and less money to build something, learn from it, discard it, and build another (repeating the sequence several times) than it does to learn all the possibilities and pitfalls of numerous options in advance through “research.” This has been a practical reality for a long time, far longer than the buzzword agile has been in use.

That pesky moving horizon

Now you may be asking, “If you’ve seen this pattern before and you know what to expect, why don’t you just tell us what we need to do to be at Basecamp 5? Let’s start Monday!”

That would be great. Unfortunately, things don’t seem to work that way. Combining the experiences of the LeadingAgile consultants, we’ve seen that approach many times in many kinds of organizations. We’ve tried starting with culture change; with procedural change; with technical practices. We’ve tried driving change top-down; bottom-up; by consensus or invitation; by management dictate. What’s common in those cases is that when people are told what to do, the desired change in mentality doesn’t happen. When people are invited to change their thinking, they simply don’t know how. People remain in the mindset of following orders. The only difference is they’re following different orders than before. The changes don’t penetrate deeply, and they aren’t sticky. People become frustrated with the results, and abandon the effort to change.

It seems to be important that people deeply understand the why of the change. To become aware of some of the possibilities is a good first step, but it isn’t sufficient to create meaningful and lasting improvement. People need to be able to get their heads around the potential benefits and risks of any given change. For that to be possible, they need guidance beyond the limits of their comfort zone…but not too far beyond those limits. Very radical change, introduced suddenly, will only lead to fear and frustration. The only way to reach Basecamp 5 is to walk there, step by step.

Remember the bit about the horizon moving ahead of you? It does. At the outset of the journey, you don’t have enough information to visualize possible end states. There may even be so much organizational “fog” that you can’t really tell which way to turn. The best you can do is set a direction that seems to be consistent with your goals. Then you have to take a deep breath and start walking, pausing to check your compass frequently and adjusting course accordingly.

Maybe the first few lighthouses you build will burn down or be swept away by the sea (or be destroyed by Napoleon’s army, as the case may be), but eventually you’ll build one that nothing and no one can tear down. The key is to be willing to try things that don’t turn out exactly the way you hoped, and learn from those experiences. Just keep going. As long as you have a good compass, you won’t get lost.