Embracing a Systems Engineering Approach to Agile at Scale

What if, instead of focusing on all the Agile practices, we started applying some of the things we know about systems engineering to help large companies become the kind of organization that can leverage those practices?

Instead of blindly performing the ceremonies of Scrum, what it we got super disciplined about how we decompose work and flow decisions through the organization? Craftsmanship and emergent design are great. But at some point, the ideation of features and requirements and the decisioning process looks a lot more like an assembly line in a factory than it does boutique delivery.

Video Transcript

Agile is a way of enabling work cells in that factory. But what is the factory? How does workflow across the Agile world? We’re really focused on the idea of empowering teams, the teams being the best people to understand how work is supposed to get done. There’s not a lot in the Agile community right now that basically says how do we apply systems engineering and systems thinking to how the assembly line of software happens?

What does the supply chain for software at scale look like?

(Musical Interlude)

There’s something that I’m exploring here a little bit that’s been on my mind lately that I’m going to share with you guys. So, I spend a lot of time talking about the differences between Agile practices and the nature of the organization that is necessary to deliver against those Agile practices. In my world, it’s really the difference between, let’s say, doing Scrum, right?

I’ve got Scrum Masters and Product Owners, and I do daily stand-ups, sprint planning, reviews, and retrospectives. But the idea is to have really well-formed, complete cross-functional teams, teams that operate off of a clear backlog and are able to produce a working, tested increment of software at the end of every sprint.

We go to client after client, up clients that are doing Agile really well. Agile is pretty well understood at this point, but they are unable to form the kinds of teams that we want them to form, unable to build the kinds of backlogs that we need them to build, unable to produce the working, tested software that we want them to produce.

And this is the interesting thing. So, I’m going to challenge you guys for a minute. If you go back to the early works on Agile, back in the early 2000s, the early Mary Poppendieck work, the early Kent Beck work, the early Ken Schwaber work, the early Alistair Cockburn work, the early Mike Cohn work, all that stuff, right?

Those guys were exploring the underlying theory of how you organize around value streams. They were exploring the idea of how to deal with variation in highly variable systems. They were exploring the idea of Lean applied to the software development process, how to operate in smaller batches, how to continuously focus on quality, how to continuously get feedback.

I talked a bit about the idea, maybe the differences between predictive process control and empirical process control. There is a deep understanding, a deep exploration of the theory of software over time. What started happening was that those practices became codified. And then you have Jim Cundiff and others at the Scrum Alliance who really popularized the CSM training.

Right. They took Agile mainstream. Even PME got on board with the PMI, ACP certification. In order to certify somebody, you really have to make it repeatable, right? You have to know the roles, the ceremonies, and the artifacts. What is Agile? What do you do to do Agile? At this point in our evolution as an industry, it seems like we are focusing on the practices of Agile over the underlying theory. I talk about this a lot, right? We talk about Leading Agile being a systems-first company. We have to get the structure of the organization right.

We have to get the governance and metrics right. We have to have an idea of what the end-to-end system of delivery looks like. Then we enable that with practices, and culture emerges over time. But what I’m really hunting down when I go down that exploration is that a lot of us have either forgotten or are not aware of the underlying physics that those practices were designed to work on top of.

It’s like the physics of team strategies, the physics of small batch, the physics of balancing capacity and demand, the physics of constant inspection and pulling risk forward, and all that kind of stuff. All of that stuff got codified into these practices of Scrum, these practices of SAFe and other methodologies that emerged. And I think one of the biggest challenges that we’re dealing with as an industry right now is that we’re thinking that if we do the agile things, we’re going to get the Agile results. If we show up and do a daily stand-up, that’s going to really help us get the results we want.

Another line of thinking that has been noodling around in my brain a lot… So back in the early nineties, I was in college, and I did a degree in computer science. My first job out of school was as a systems engineer for EDS. And one of the things that I think is interesting, you know, they put you through a fairly rigorous systems engineering development program. I think I probably undervalued the amount that the systems engineering perspective has really penetrated into how I think about things. We talk a lot about the idea in the Agile community of being a systems thinker, right.

But one of the things that I think is somewhat pervasive right now is that the idea of Agile and collaborative creation has somehow supplanted the idea of systems engineering. So what I believe we need to explicitly consider is: What is the system? How do we engineer the system that we are operating on top of?

And how do Agile practices enable that system? When we discuss the idea of transformation, when we think about implementing a new system of delivery, organization, or operating model, we are essentially discussing the systems engineering-focused design of that organization. How does work flow across the organization? What do interfaces between teams look like across the organization?

I have mentioned this in my videos before, but about five or six years ago, definitely pre-pandemic, I returned to my alma mater, the University of Florida, and conducted numerous talks on campus. I became involved in the Entrepreneurship and Innovation Group’s engineering leadership and participated in some of the industrial and systems engineering classes over time. During these interactions, when I discussed Agile with industrial and systems engineers, it prompted me to consider what the software factory looks like.

I started contemplating the application of industrial and systems engineering concepts to the software factory. The idea that has been occupying my thoughts is that Agile serves as a means of enabling the work cells within that factory. But what exactly is the factory? How does workflow function within it? In the Agile world, we are truly focused on empowering teams. Teams are the best people to understand how work is supposed to be done.

Recently, there was a commenter on one of our posts, and I responded with something along these lines: Imagine I am working on an assembly line responsible for installing doors on cars. In that specific task, I might be the most skilled person when it comes to understanding the mechanisms of putting doors on cars, right?

If there’s a problem with how we’re currently installing doors on cars, I might have valuable feedback on improving that specific task. However, am I necessarily the best person to engineer how the entire assembly line works or to reimagine the supply chain?

I believe there’s something in what we’re currently doing as an industry that assumes if we implement Scrum or SAFe, the assembly line and supply chain will naturally emerge. And to some extent, that may be true for smaller organizations that are ready to embrace Agile practices out of the box. Many things will naturally emerge from doing Scrum or SAFe, and impediments can be identified and addressed within the team’s operations.

However, a lot of these impediments are focused on optimizing specific aspects, such as making door assembly more efficient. What’s lacking in the Agile community right now is an emphasis on applying systems engineering and systems thinking to the overall process of software assembly. How does the software supply chain look at scale, even with the presence of scaled frameworks? I perceive these frameworks as operating models that assume the existence of a functional assembly line and supply chain.

But what if the assembly line is not effective? What if the supply chain is not effective? I’m not convinced that the individuals working on the assembly line are the best ones to provide advice in organizing the system or scaling the work. Furthermore, they may not be the most qualified to address the challenges of fixing a broken assembly line or supply chain and reimagining them.

In conclusion, I’d like to invite your feedback and input on this topic. Please feel free to comment and share your thoughts. I would appreciate guidance on which aspects of this discussion you would like me to delve deeper into. I believe there is much more to explore, but I want to ensure we are on the same page.

So, please provide your comments, and I look forward to your guidance. I’m not convinced that the individuals working on the assembly line are the best ones to organize the system, scale the work, or reimagine a broken assembly line or supply chain.

Okay, and there’s a book I’m reading, and I apologize as it just came to mind. I’ll ask my team to provide a reference for it in the comments or something. There are a couple of books on Agile systems engineering that I’ve been reading, by Dr. Bruce and others. I’ve just started on both of them, and they’re really fascinating explorations of what systems engineering looks like in relation to agile software delivery.

From the early stages of reading these books, it seems that they focus more on systems engineering for products. However, what we’re really discussing is systems engineering applied to organizations. When we talk about transformation, we’re not merely talking about improving collaboration in building doors. Transformation involves examining the aspects of the assembly line and the supply chain. What does that look like?

I understand that many individuals may dismiss what I’m exploring here because they don’t necessarily like the idea of an assembly line for software. Some prefer the notion of craftsmanship and almost a boutique-style delivery. It’s as if we’re suggesting that if we were Ford Motor Company, we shouldn’t be mass-producing cars but rather building unique Lamborghinis. However, the reality is that many software organizations strive to operate at scale and undertake significant endeavors. This often necessitates a certain level of systems engineering focus.

Over the years, we have observed that there are situations where agile delivery is attempted within non-agile systems. I have a talk called “Faster Food and a Better Place to Sleep” where I worked with a hotel chain and a fast-food restaurant, both based in Atlanta. We discussed agile delivery and non-agile systems. In these scenarios, the software factory is essentially an idea factory. How do we facilitate the flow of ideas and decision-making through the system? It’s an intriguing concept to explore.

So please don’t dismiss this idea outright, as at a certain level of scale, you will encounter software as an assembly line or a software supply chain. The supply chain may involve ideas, feedback, and validation. While the actual building of individual parts may have a unique and boutique nature at the work surface level, the decomposition of ideas into actionable work and the aggregation of those ideas into integrated deliverables becomes a supply chain problem.

Over time, we may attempt to delve deeper into these concepts. I’m unpacking ideas from various angles, but fundamentally, Agile, even in pure-play software, is an exercise in systems engineering. What does it look like to engineer a system of delivery at scale within a large enterprise? Once again, I would love to receive your feedback and understand which ideas resonate with you. Please feel free to leave a comment, and I eagerly await engaging with you there. Have a great day, everyone. See ya’