Agile Infrastructure Transformation and TBM

When I first thought of writing this article, I struggled with how best to talk to these perceived different approaches. Questions of how does Agile apply to infrastructure and what does TBM have to do with Agile were in my mind. A simple google search reveals just a slim number of articles and these are mainly based on the theory of how these concepts come together vs. real-life learnings.

Let’s start with some basics. What is TBM?  I have had exposure to TBM in a past organization and the driver and my perception was that it was about IT Towers to assess costs and consumption related to infrastructure spend so that the organization could leverage benchmarks and “defend” our costs. My experience is that in a TBM implementation that is driven by a CTO or even a CIO in a defense mode (“Why are your costs so high?”) is a more than common approach, but falls short of the value and intent of TBM. 

TBM is a model and taxonomy that at its most basic level does understand and map costs and consumption. But the intent is to elevate the discussion with the business related to value and trade-offs as you understand what is driving costs. So instead of having a defensive conversation with a CFO on why something costs so much, we are having a discussion related to what is needed and then can understand the costs related to supporting those business needs. TBM also seeks to link the long-term support costs beyond a project since once a project is done there are ongoing run costs that are part of the TCO of a product. 

The TBM Taxonomy

One example I experienced around the value of Total Cost of Ownership. A prior CTO gave an excellent explanation of what it looks like to truly understand TCO. At the time, I was leading a group that was responsible for Application Portfolio Management and assessing risk, value, and cost for all of our applications in the company. The CTO told me he wouldn’t be surprised if we found applications with low value and high costs that we needed to retire or replace. He related an extreme example where a company had an application that had been in place for several years and was put in place for parking lot reserved space management for the company. Over time it became used for only the executives in the company. After a detailed TCO analysis they found that they were spending over $1M per year on this application in TCO (costs to manage, support, and run the application). Having this information enabled a conversation on options – one of which was to hire a person to manually run the function. See where this is going? A shift to a product focus.

Product Focus

As we look at recent thinking around Agile, there are more and more articles on a shift from Project to Product, flow of work, and value-streams. This is where TBM and Agile align. It starts with the need to fully understand a value-stream for a service or product from a customer (internal or external) and how we encapsulate the people needed to deliver on that need. Understanding the costs and consumption of that service so that we make sure we work on the highest value work is part of the role of a traditional product manager and aligned to TBM. A TBM model and related tool will assist in that cost and consumption transparency so that those discussions can be informed with data. 

Agile in Infrastructure

As I work with more organizations, I realize that many have made great progress with Agile in software development areas. I spent quite a bit of time with part of a large infrastructure and ops group (over 2000 people and $1B of spend) in re-educating them in how the fundamentals of Agile applied to them as well. Many large, bureaucratic companies suffer from being too slow, perceived as too expensive, not transparent, and can’t make the changes needed to improve.

Challenges such as Conway’s Law get in the way of optimizing the end-to-end value stream. Conway’s Law showed through research and experiments that “Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure. “ — Melvin E. Conway. 

Applied to a Lean-Agile approach and internal processes, the results are seen in many organizations today. Take for example getting a group of people that all have a role in a process and get them in the room to optimize the process. If you have 10 different groups represented, you will likely have a ten-step process – where each of the 10 steps has been individually improved. This is because no one wants to opt-out, they risk feeling like they are not important, or that their job function related to that process is waste that can be eliminated. If you start with a value stream aligned structure with one leader of the end to end process it is much more likely you will have an optimized end-to-end process.

Visual of Conway’s Law

Many companies spend money to automate and create VMs or IaaC as an example, but the software teams are still frustrated, and the business areas don’t see the benefits of these investments. As an example, the company will have automated the build of a server, but the end-to-end process is still not optimized. When we apply Agile in Infrastructure, we start with a view of who the customer is, and then look at the flow through the system and what we need to encapsulate to reduce these silos and dependencies.

I was once asked by a CIO to explain why it took so long to get a server when we had automated the solution. We had spent money on software and people to be able to provision an environment and claimed credit for reducing the time from days to minutes.  However, the software team’s experience was related to the end-to-end results, not just this part of the process. The end-to-end piece included approvals, firewall changes, funding, financial management. and other steps. I told the CIO that my team needed a production environment and had value-stream mapped what happened. I told her that the end-to-end process involved us submitting 22 different request tickets which generated 30 tasks went to 12 different teams to get all of the pieces in place for a usable server environment. This process took over 90 days for something that took less than an hour of work because of all the hand-offs and wait states between the different teams. The managers of these teams had optimized their siloed part of the process, but no one in the organization owned this end to end process for the customer. 

End to End Processes

Where Agile for Infrastructure and TBM Align

Agile for Infrastructure and TBM align several ways. Agile is all about maximizing the value delivered for the organization and being able to adapt to change. TBM enables visibility to that value, costs, and consumptions so that portfolio-level decisions are enabled. It also aligns costs from IT Towers up to these services that are needed to support the organization. Agile provides the System of Delivery that supports the value decisions enabled by TBM and aligns the structure to reduce silos and dependencies to increase efficiencies by reducing waste.  

So where do we start, and how can we solve for these challenges?

• The key is that we leverage the principles that enable Agile to work, which we call The 3 Things: teams, backlogs, and working tested software/product.
• We design an organizational structure to reduce dependencies. We make sure that we better align to our customers to create a fast feedback loop. 
• We collapse silos and create cross-functional teams We shift more work to planned work as we get closer to the customer and get out from behind ticketing systems. 
• We truly understand costs, consumption, and performance so that we can make data-driven decisions to accelerate business outcomes.

Teams, Backlogs, and Working Tested Software

In one organization where we applied our approach, we first addressed the structure and consolidated some of the silos and enabled cross-functional teams. These new teams now had the key skills to perform the fulfillment of providing a server and were each aligned to a portion of the organization to create a better and closer feedback loop vs. trying the serve the entire organization.  We couldn’t completely collapse everyone into one team as there was a balance needed. But we were able to remove many of the silos. By collapsing these silos and streamlining the processes, we were able to reduce many of the hand-offs and wait states. Over time we can get more lead time on the customer needs and shift more of the work to planned work. The result of these changes was a more than 50% reduction in cycle time, simplification of the process to the customer (fewer steps, tickets, etc), and cost savings from reduction in the number of people needed to deliver the same volume of work.