Toward a principles-based process

Agilists often tell us we should craft our own process based on first principles, rather than adopting a published framework as a fixed set of rules. When they say this, we settle back into our plush leather Chesterfields, crease our brows with a well-practiced furrow, swirl our Cognac, stroke our handlebar mustaches, and intone as Britishly as we can, “Ah, yes. Principles. Jolly good, wot?”

We then proceed to adopt a published framework as a fixed set of rules.

Because, you know, like, what principles?

Well, they turn out to be the same principles as were discovered by early practitioners of the Soft Arts way back in the nineteen-and-sixties, and that were subsequently exorcised in The Great Bureaucratic Inquisition of the early 1970s, which established the Holy Church of Waterfall that dominated the world until the end of the millennium (praise be slathered upon DoD-STD-2167A and the other saints and sub-paragraphs thereunto appertaining).

By the mid-1980s there were murmurs among the common folk that the Church of Waterfall may have been honoring false gods. After all, what loving gods would allow such misalignment between customer needs and software solutions as were common in that era, such massively corrupt code bases, or the systematic degradation of knowledge workers to the status of mere “resources?” A sort of reformation movement gained momentum in the 1990s. According to the Agile Pearls of Wisdom blog:

So-called lightweight agile software development methods evolved in the mid-1990s as a reaction against the heavyweight waterfall-oriented methods, which were characterized by their critics as being heavily regulated, regimented, micromanaged and overly incremental approaches to development.

When they advise us to base our process on principles, the Agilists are probably thinking of the four value statements expressed in the Agile Manifesto:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

And then there are 12 principles to illustrate those values (not reiterated here).

But that 2001 document wasn’t the first, and isn’t the only expression of similar values and principles. The Dynamic System Development Method (DSDM) was released in 1994, based on the following “building blocks:”

• End users actively involved
• Empowered teams
• High focus on team collaboration and cooperation
• Frequent releases and updates
• Development driven to meet business needs
• Incremental development is key
• Openness to change
• Keep requirements at a high level
• Efficient integration between development and testing

Another formulation of similar values, focusing on process and not on technical practices, calls out the guiding principles behind the Kanban Method. As David Anderson lists them:

• Visualize the workflow
• Limit WIP
• Manage flow
• Make process policies explicit
• Improve collaboratively (using models and the scientific method)

The popular Scrum framework defines a set of three “pillars” as well as several explicit values. These emphasize the human aspects of collaborative work quite strongly, as well as focusing more on process than on technical practices. According to the Scrum Guide, the pillars are:

• Transparency
• Inspection
• Adaptation

And the Scrum values are:

• Commitment
• Courage
• Focus
• Openness
• Respect

The word “commitment” is often misconstrued to refer to “meeting delivery promises.” It really refers to the team’s commitment to delivering customer-defined value and to applying the best professional practiecs they know of to their work; commitment to each other as team members.

A more recent formulation, known as the Lean-Agile Principles, has been adopted as-is by many organizations and even by some published frameworks. The following list comes from the website of the Scaled Agile Framework (SAFe), which has adopted the principles without modification:

• Take an economic view
• Apply systems thinking
• Assume variability; preserve options
• Build incrementally with fast, integrated learning cycles
• Base milestones on objective evaluation of working systems
• Visualize and limit WIP, reduce batch sizes, and manage queue lengths
• Apply cadence, synchronize with cross-domain planning
• Unlock the intrinsic motivation of knowledge workers
• Decentralize decision-making

It can be daunting to try and craft a custom process for your organization based on principles, when there are so many principles flying around. Fortunately, there are some common elements in these lists. Let’s see if we can break it down and make some sense of it all.

Categories

The various expressions of principles seem to address distinct areas:

• All development is driven by business needs (take an economic view, end users actively involved, customer collaboration over contract negotiation, and similar phrases)
• Alignment/connection between consumers of software (customers, users, stakeholders, and similar terms) and producers of software (developers, teams, “release trains,” and similar terms).
• The importance of clear and accurate communication (transparency, openness, and similar terms) and visibility of the state of work in progress (visual management, information radiators, and similar terms)
• The quality of the interactions between people (collaboration, cooperation, transparency, courage, and similar terms)
• The importance of feedback loops to guide solution design and to stay aligned with customer needs (iterative development, incremental delivery, fast and integrated learning cycles, and similar terms)
• An emphasis on empiricism rather than abstract planning and documentation (objective evaluation of working systems, inspection, adaptation, working software)
• Effective coordination of work across teams (synchronize with cross-domain planning, limit WIP, manage flow, and similar terms)
• Keep options open; embrace change (assume variability, preserve options, inspection and adaptation, responding to change over following a plan, and similar phrases)

You may arrive at a different list of categories than I, but you’ll probably agree with the general approach of identifying the key elements of these various lists of principles. When we get it down to a handful of key points, it doesn’t appear quite so complicated. All the variations beyond that amount to word choices.

To me, these common categories of principles distill into a short list of key elements a lightweight process must have if it is to help deliver value, especially in an environment that comprises more than one delivery team:

• Directly involve key stakeholders who have a clear understanding of needs and can make tactical decisions (without asking permission) about the direction, pace, and content of interim deliveries.
• Turn around incremental deliveries frequently to obtain feedback from key stakeholders to inform learning cycles.
• Visualize the overall process and the point-in-time state of work in progress for all involved parties.
• Employ effective collaboration methods among team members and across teams within the larger organization.
• Quantify the expected value and anticipated cost of planned work, including opportunity cost and cost of delay.
• Coordinate planning and execution of work items that span multiple teams or that have other organizational dependencies.
• Defer decisions until the last responsible moment, but also address knowable risks far enough in advance to prevent their becoming costly problems.
• Use metrics to detect potential delivery problems early enough to deal with them.
• Take full advantage of what others have learned over the years about managing flow and maximizing value-add time; avoid repeating mistakes others have already made.
• Rely more on forecasting based on empirical performance data than on estimation for short-term planning

You may have other elements in mind that I didn’t think of, as well. In any case, the exact terminology you use and the exact procedures you follow to accomplish these things aren’t as important as their substance. Try not to get tangled up in the “rules” and market-differentiating buzzwords that go along with published frameworks.

Finally, I think it’s useful to remember that you don’t have to get everything right on the first try. You can change your custom process over and over again until you’re satisfied with it. Use the feedback loops or learning cycles inherent in this approach to improve your own process incrementally.

LeadingAgile can help

LeadingAgile has a proven, practical approach to help organizations move in a direction that embraces these principles. We don’t have a rigid, canned methodology, or a big dripping-wet sack of rules for you to follow mindlessly. If you aren’t sure how to convert relatively abstract principles into practical reality, we can help with that. We can draw elements useful in your context from various methods and combine them in such a way that they don’t cancel each other out. So, don’t be afraid to do just what the Agile people say: Craft your own process based on first principles.