Zen and team organization

There’s an old Zen saying: Before enlightenment, chop wood, carry water. After enlightenment, chop wood, carry water.

A sensible way to get things done is to organize people around the work, forming work groups or teams that have the appropriate skills to meet emerging demand. We used to do that, but we did it in an unenlightened way. Along the path toward enlightenment, we tried various alternatives that solved some problems and introduced others. We finally found a way to organize people around the work that appears to result in smooth, predictable delivery.

1980s-1990s: Matrixed organizations

In the 1980s, IT organizations tended to focus on maximizing individual utilization. They organized people around emerging demand by grouping individuals by skill set or specialization and creating a matrixed organizational structure. When work required particular skills, individuals were assigned part-time to that work.

Simultaneously, the same individuals were assigned part-time to other work that called for their particular skills. The various virtual work groups were called “teams,” but they lacked the characteristics of fully-functioning teams.

The approach led to problems that are obvious in hindsight: Local optimization, knowledge silos, cross-team dependencies, conflicting priorities of in-flight initiatives drawing from the same pool of people, and more.

IT organizations have learned to focus on maximizing throughput rather than utilization. It is still sensible to organize people around emerging demand, but the way we do it today is different from the old matrixed organization. The path from the 1980s to the present is marked by a couple of distinct approaches to organizing people around the work.

1990s-2000s: Stable, dedicated teams

Even in the heyday of matrixed organizations, people recognized that the approach was not producing satisfactory outcomes. The idea of forming a cross-functional team and keeping it intact across multiple initiatives, one initiative at a time, began to take hold.

Possibly the best-known attempt to form a stable team dedicated to a single initiative is the Chrysler Comprehensive Compensation (C3) project in the late 1990s. The software development method known as Extreme Programming (XP) emerged from that project. XP is at the core of many defined software development methods.

In roughly the same time period, the well-known process framework Scrum was defined (1993), based on seminal work published in 1986 by Takeuchi and Nonaka in the Harvard Business Review as “The New New Product Development Game”. Scrum also calls for a cross-functional team to remain stable and to be dedicated to a single initiative at a time.

Stable, cross-functional teams taking on multiple projects one by one became the norm by the first decade of the 21st century. Some companies built strong businesses on the model, including ThoughtWorks and Menlo Innovations. The founder of Menlo Innovations, Richard Sheridan, wrote a book about it entitled Joy, Inc. Some internal corporate IT shops also adopted the model, including several very large organizations.

2000s-2010s: Balanced product teams

Although Lean Thinking dates from the 1940s or 1950s, it did not penetrate management thinking in the IT space until the first decade of the 21st century. When it finally did penetrate, people began to evolve the stable team model to reduce dependencies and improve flow.

One of these evolved team models was developed at Pivotal. Their model for team structure extends the idea of a stable team to define balanced, product-focused teams. A team is balanced because it comprises all the skills needed to support a given product. Technical team members are “full stack” engineers who widen their skill set and domain knowledge through pairing and rotating through other teams in the organization periodically.

A team exists for the long term, but its composition changes in a managed way. Someone who knows the product well remains with the team, serving as an “anchor.” Part of his/her job is to mentor others to the level that they, too, can serve as anchors on that team. This frees the previous anchor engineer to rotate to another team, where he/she can continue to learn.

Most XP practices are applied. Work is done in pairs, and pairs can be lent to other teams to address fluctuations in demand. The model achieves focus for supporting products as well as providing a mechanism to organize people around emergent demand.

An issue with the previous “agile” approach of keeping teams intact for many months is that individual team members become bored and can burn out. The rotation scheme at Pivotal mitigates that effect and helps to keep people fresh and engaged.

Not quite there yet

Mura (irregular flow) is still present in this system. My observation (and I think others have noted this as well) is that mura seems to generate muda (non-value add activity) and muri (absurdity or overloading); that is, it’s often the root cause of other delivery problems, from a Lean point of view.

An approach that focuses on achieving continuous flow, without necessarily retaining aspects of previous models (however beloved those aspects may be), might further improve delivery practices.

2010s-Future: Fluid self-organization

Fluid self-organization around demand has not (yet?) become the norm in software delivery processes. Such a process would target mura explicitly. A number of organizations have experimented with this approach, although no fully-defined, published model has emerged.

One organization where fluid self-organization worked well for a time is Cengage Learning. LeadingAgile’s CTO, Chris Beale, and his team gradually evolved a traditional development organization to adopt XP, and then further evolved the process to the point that any individual was competent, and felt comfortable, pair programming with any other individual in the organization.

As demand varied, technical staff signed up for the work to which they wanted to contribute. For the less-interesting tasks, people had to work on things they might not have chosen, but they had first pick when the next set of work came in. The system worked well for about 150 engineers working in four locations.

An experiment along similar lines was carried out at a large credit card company in the mid-2010s. In that case, planners identified the skills needed for each piece of work and prepared a card for each. In a large planning session, a summary of each initiative was posted on a wall along with the cards representing the skills required. Staff members then took a card. That was their next work assignment.

For example, an initiative might call for two senior Java developers and two junior Java developers. Four cards, representing those four needs, would be posted on the wall along with the summary of that initiative. A person who had worked as a system administrator or software tester, and who was interested in learning Java, could choose one of the junior Java developer cards. It was expected that people who had senior-level skills would mentor junior colleagues.

The system worked well for about 60 people until middle management reasserted traditional methods of planning, organizing, and tracking work.

These systems, and others across the industry, worked well while they were in place, but as far as we know nothing like them has taken hold as a “standard” or “routine” way of working.

If fluid self-organization around emerging demand becomes the norm, we will have come full circle, back to the dynamic staffing approach we were looking for in the 1980s. The difference is that today we have the benefit of “enlightenment” through Lean Thinking.

Chop wood, carry water.