Are These DevOps Obstacles Getting in the Way of Your Agile Transformation?

As more and more companies scramble to untangle the code of their monolithic legacy software and get it into the cloud, strategic priorities are shifting. 10 years ago, it seemed that a little Scrum here and some coaching there would suffice. But as companies scale, as the market shifts and more parts of the business become software-focused, there’s been an increase in demand for an end-to-end solution to help large organizations build infrastructure around sound technical practices.

Today, Agilists must have a plan for incorporating DevOps practices into the early stages of Agile Transformation. But what if we told you there’s a whole class of DevOps considerations that no one is talking about that’s secretly killing your ability to unleash the potential of Agile? Everything from testing, monitoring, and containerization to vendors and the skills of the IT people on your team.

In this talk, we’ll look at what’s left out of the conversation and discuss five non-obvious DevOps barriers to Agility in large organizations. Join us as we explore these obstacles and how you can overcome them. Leave with an actionable roadmap for aligning your technical practices with the work of the Transformation that you can begin implementing today.

Want the Deck

Text HIDDEN to 33777

Video Transcript

We will run into times where packing software makes it harder for us to transform the skills of our staff and external vendors. So how do we handle transforming our company when a lot of the work isn’t done by people who are directly employed by us? Career incentives may discourage people from being agile, and then we ask them to be agile. We’ll also discuss the lack of automation.

(Musical Interlude)

Alright, let’s get started. I’m Matt VanVleet, the Chief Technology Officer for Leading Agile. I also assist a couple of product startups. I used to have my own consulting company and have been involved in Agile practices since XP and before the Agile Manifesto. I used to run a company that developed unit testing software.

Then I joined Pillar Technology, where I spent 18 years and eventually became an owner of the company. Currently, I’m helping out various companies, including Leading Agile, in starting a software studio and other ventures. As a consultant, I’ve encountered many impediments or barriers to Agile transformation. How many of you here are consultants? How many of you work in the industry? And how many of you are in the midst of an Agile transformation?

I suppose to some degree, we’re always in the middle of it. Anyway, that’s the introduction. I’ll repeat this at the end, but if you want the presentation deck, it’s available. Let’s check if the volume is okay. Can you all hear me clearly? Okay, great. Now let’s move on to discussing the five things that can hinder an Agile transformation.

The 5 Hidden Agility Killers

Of course, the transformation itself is essential, but there are some factors that can slow it down. I’ll talk about these factors and provide techniques to mitigate or overcome them, and then we’ll move on to the next topic. But before I present my five things, what do you guys think are the five impediments to an Agile transformation? I don’t remember if they’re mentioned in the abstract, so you might be ahead of me. Share your thoughts.

Alright. Some of these aspects are already on my mind when it comes to Agile transformation. We need to obtain buy-in, address ambiguity, and navigate the process. Additionally, we may encounter challenges with packaged software that make the transformation harder. We can discuss why that is and the impact it has on the skills of our staff. External vendors are another aspect to consider. How do we handle transforming our company when a significant portion of the work isn’t done by our direct employees? Career incentives are also relevant. Sometimes, we inadvertently discourage people from being agile and then expect them to embrace agility. We can explore this topic further. Lastly, let’s talk about the lack of automation and address its implications.

Dealing with Packaged Software

The first one, it came up, right? Different change cadences. The package is changing at one cadence and maybe you’re trying to innovate and change at another cadence, right? Type of tightly coupled integrations. So you may have a system you want to change, but it’s integrated with the package or you may want to change the package, but you’re worried about all the downstream systems, you know, so you can’t upgrade without testing the integrations, you can’t do the integration without involving the team.

You know, one of the things about Agile is we try to do the simplest thing that’ll work, put it out in the market, test it, get feedback and change it. So that iterative cycle is hard when you get it’s hard to implement half or a third of a package or just these three screens, right? Because it’s set to do a set of things.

Now it also can accelerate you. So it’s not that you never should use it, but making good conscious choices of when to use it, when not in understanding that if it’s in your most innovative areas, it may not be accelerating it. Right? So, you know, standardize what you want to fossilize. So if we standardize on a package, those are areas of the business.

We just want to work consistently the same way all the time. Right? If we’re in the auto industry and we’re going from, you know, traditional vehicles, battery electric vehicles and from a hardware-based solution to a largely software-based solution, we may want not want to standardize that. Just there’s a lot of packages for how you would run a car.

But as an example, you know, I was at a customer and there you know, we ended up having a lot of requirements where we knew this was wrong, but we couldn’t change it because we’d have to go change the package. And it was a lot of work. So we just had to figure out how to work around or keep going with it.

So those are some of the constraints. What I want to do now is talk about what are some techniques because it’s not like I’m saying don’t use packages. We’re just saying be conscious of it. Right. And understand that when you’re trying to do a transformation, if you have a customer development team building a mobile app and you have someone installing a package, you’re going to have to have slightly different techniques for how you do things right.

So one is you really want to double down on testing. So at the edges of the package, any interfaces, integrations in and out, you want to be able to have automation around that. You want to be able to mark things that aren’t available because now you can move the package, something moving at different cadences. You need to be able to upgrade one without having to involve all the other teams.

The first time you put the package in place is something that’s, you know, you can get all the teams rallied together and figure out how to do it. But now one side is upgraded and the other side doesn’t. You know, you don’t want to have to the start, things change, right? You know, you want to make sure that your version updates can be tracked and version control, which seems obvious, but a lot of times where things are configuration based and maybe the last script that you’re going into a UI, you aren’t able to see all the changes you made and therefore upgrading and downgrading and changing becomes created.

And then, you know, understanding the business capabilities and which ones need to move and be innovative and which ones you want to standardize. So you select packages in the right circumstances. Okay, So how many people are dealing with a mixture of cost packages and custom code in their transformation? So get numbers. Yeah, hopefully if you if you can encapsulate them and isolate them and deal with the fact that they’re working at different cadences.

And then after that, you have to orchestrate and just manage and have, you know, good, good communication and all of that.

Lack of Skills on Staff

All right. The skills of staff. So what how do you see that as an impediment to your transformation? Right. Yeah, I’ve been in environments where I said we wanted to have a team and, you know, over the years we have outsourced all of development.

So to have a co-located team of ten people, they just didn’t exist that were available for the project. So you definitely can have that right. So the transformation is about learning. So you may not have the skills, you may have to get the team to learn, and that’s why we like to keep teams in place because they learn how to form and act as a team.

I was, you know, it was kind of reminds me of like there’s something like 10,000 chapters of Harry Potter on the Internet that are aren’t written by J.K. Rowling. And, you know, all those people who are writing those chapters are at work during the day. And they’re managers saying they’re not very creative. So one of the things is that we’ve gotten to the stance where we’ve put people into a role and said, do it this way, do this, and we haven’t given them, and then they go get their creative outlet at home.

And now we’re asking them to exercise different muscles during their work day and so forth. Another thing to do is like a lot of people will go through this transformation but not change how they’re hiring. So now you’ve got someone who’s gone through a six month transformation and someone leaves and someone new comes in and you have hired four people that have that experience and learned.

So you’re still now those people are coming in to a team that’s working differently and they haven’t actually had the opportunity to go through that journey right? So looking at that. So the skillset thing can be while bringing in new people. Right. And you know, I has to give people time to make the change, to learn, right? So there’s times where we just won’t have the right skills.

We have to hire, find them. But there’s also times that we’re teaching our skills and we need to make sure that we embrace and do that. You know, I, I was once have asked, you know, you know, the client was looking for someone that had 7 to 10 years of this Java testing framework, which had only existed for four years.

Right. And so a lot of times were hiring based on years of experience or knowledge of a tool. But we’re in a world where, you know, you know, if someone says, I want you to have ten years experience which had GPA or a language learning models like that’s not going to exist, right? So the speed at which you can learn, the speed at which you can do new things is going to be something that we should be measuring versus current knowledge.

Obviously, there’s going to be times, you know, you need some basis, need some knowledge, but we really need to kind of be bringing in people with the right skills that have that ability to learn and grow with the team. I may have just said all this stuff on the last night, but, you know, we’re learning over and over current knowledge upskill staff to take advantage of new technical practices.

They’re going to be teaching them things like TDD and refactoring and, you know, building things in isolation, you know, the ICD. So you’re going to be you know, you’re going to have to invest and teach these techniques, but they’ll let you move faster with safety. And, you know, ultimately the thing that slows software development down is fear of change.

So if I go change something, I’m going to break it.

Then I may be afraid of change, that I may decide that we should build two copies of this code so that I don’t break the stuff that used to work. So so building with these techniques are going to let us move faster. But you have to invest in the skillset of your people so that they know how to do those.

So they’re able to, to, to do that. You know, a real job this year talked about their new roles need to become more cross-functional. You know, ultimately, I think there’s kind of two roles on a team. There’s people who know how to automate things, especially like a software development, and there’s people who know how to automate things.

They can build business rules into code, they can make, you know, screens, but they can also make automated tasks. They can make automated things that set up your environment, just people who are good at automating things and whether they’re automating, setting up your server or your deployment or your business rule or what have you, you know, over time, people tend to be able there may be specialists in some of those, but they tend to be able to do those.

There are also individuals who deeply understand the business domain. They interact with the customer, comprehend the customer’s needs, and communicate these to the team. They can evaluate something we’ve built and determine whether it’s functioning as intended. Over time, your business and technical experts may even become cross-functional if they work in that domain long enough.

Often, we begin with very specific roles: a business analyst, a developer, a product owner, a product manager. However, as you progress through an Agile Transformation, these roles may become more fluid. It’s crucial to create a safe environment for people to grow in their careers without feeling confined to their initial roles.

External Vendors Slowing You Down

Let’s consider external vendors. As a consultant, one might wonder why consultants could potentially slow down your transformation. Ideally, they should accelerate it. But how many of you work with external consultants? If you do, why might a vendor slow down a transformation? They need to understand your goals to effectively contribute to the transformation.

Some consulting models may want you to become dependent on them. They might offer a service at a low price, but then you need their support indefinitely. This can be problematic if we’re trying to automate something, like testing, and we’ve hired a large team from an external vendor for manual testing.

The goal should be to help clients become independent, not dependent. A great consultant will solve a problem for you, and then you’ll hire them to solve other problems. Your companies won’t run out of problems to solve, so there will always be work.

The question is whether the consultants are engaged in a way that will accelerate your Agile Transformation. If you’re aiming for product-based, cross-functional teams working collaboratively, but your vendors are merely order takers who don’t participate in the process, then they’re not supporting your transformation.

If your vendors are writing code that doesn’t align with your desired practices, it can also hinder your transformation. For example, if you want to use microservices, but your vendor is free to code however they want, it can create conflict.

I had a client a couple of months ago who I advised to not just train their vendors but engage them as partners. If they’re not willing to adapt, then it might be time to find vendors who are.

I was asked, “Who could that be?” and provided a list of potential vendors, including us and several of our competitors. The question arose, “Why are you suggesting we work with your competitors?” The competition isn’t between companies, but rather about avoiding incorrect practices.

If you’re embarking on an Agile Transformation, it’s essential to communicate this to your vendors. Ensure they understand that their work methods may need to change and that this transformation is strategically important. Treat your vendors as partners, especially if a significant portion of your workforce comes from them.

Invite them to join you on this journey and invest in the relationship. If they’re unwilling, consider other options. Often, the issue lies in communication or lack of investment from management, which can set teams up for failure.

Consider incorporating these aspects into your vendor contracts. Focus on measuring output rather than input costs and prioritize results. If you want your vendors to be collaborative, innovate, fail fast, and deliver results, your contracts should reflect this.

Ensure your incentives align with your objectives. Ultimately, you want vendors who are accountable for your business outcomes, just as you want your teams to align with the business needs. If your business isn’t getting what it needs, despite having a product screen, that’s not an ideal situation.

Misaligned Career Incentives

Career incentives can also impact your Agile Transformation. Does anyone have examples or thoughts on how career incentives can slow down your transformation? I once worked with a group where we wanted developers to receive test cases early so they could address them before they entered the queue.

They were reluctant to do this because they feared it would eliminate bugs. However, the goal isn’t to have bugs. They were accustomed to a process where they would examine the requirements, build, and then test. This was more of a blind test rather than an open book test. But they were different departments, each with their own metrics.

I’ve also encountered a consulting company that believed moving to Agile meant eliminating QA. I’ve always argued that if we stop finding things, there will be plenty of other areas to explore. If there’s an incentive that encourages this, it’s problematic. We should encourage and value flexible roles.

You might not be a pure QA person; you might also work on requirements. Are your training, measurement, valuation, and promotion systems set up to accommodate this? If you’re transforming how your teams work, you may need to reconsider how you measure your teams and success.

Do you value people being subject matter experts or technical experts? Or do you need to move outside of the team and take charge of scorekeeping or orchestration to be valued? We’re trying to create teams that can align as closely to the customer as possible and produce results. We’re building teams with as few interdependencies as possible. However, our leadership structures and incentive systems might actually discourage team collaboration.

Early in my career, I was offered a promotion to a managerial role while I was a technical consultant. I wanted to remain technical for longer in my career. I was faced with a choice: stay in my role without a pay increase, leave, or become a manager. There was no technical career path available to me. So, I left and started a company that wrote software, which allowed me to stay technical. There are opportunities for those who wish to remain technical. For instance, I know of an automaker in town that has a technical career path in engineering for vehicle powertrains.

In some organizations, there are ten levels of engineers, with senior leaders having hands-on experience in building powertrains. On the software side, technical levels only go up four levels, and senior people are experts in orchestrating and navigating the system, rather than being expert technologists. We want to change this and encourage teams to grow and value leadership.

This has to translate into teams delivering more value and being more connected. It’s crucial to review the career path and ensure it doesn’t encourage people to leave your teams. Also, review performance measures. Are you measuring output or input? Are you counting defects or assessing the quality of your system? Can we measure how well the system is designed to meet customer needs? Are we measuring customer fit and satisfaction, rather than arbitrary count of velocity at the development team level, or the number of defects found, or tests run?

Lack of Automation

Why would lack of automation can impede your Agile Transformation. Automation can increase consistency, reducing mistakes, and can also increase speed. However, sometimes there isn’t enough time to invest in things that will make us faster. This could indicate that we’re moving so quickly that we’re not going to get significantly faster.

If you increase the frequency of releases and deployments, it’s going to cost more unless you automate. For instance, if you shift from releasing every six months to every two weeks or four weeks, the overhead of a release will increase. If you believe in weekly sprints and build every week, the cost will rise, especially if it takes three days to deploy. However, if you automate the most frequent tasks, the cost can be reduced. There’s a saying: “Take the hardest things and do them as often as possible, and they get easier.”

It’s hard for the organization to build it. It’s hard to deploy it. It’s hard to get to QA if it’s hard to get customer feedback, let’s do that more often. Let’s figure out how to get the cost out of there, because building it should only take as long as running the build strength. Deploying should only take as long as its place.

Yeah, but if there’s people, you know, if it’s a three-day process to get the QA know. So manual testing now there’s always a level of exploratory testing. Right. There are people what you want to do is I remember I did a project right past Y2K and we had 30 people whose job it was a test and they had you know, it took them two days and they had a set of manual scripts and they would go through and they would test it again and again and again.

And we still get in production and people would find the odd circumstance. Right. And we got to the point where we said, well, why? Let’s get that stuff automated. We got that stuff automated and we didn’t have the 30 people anymore, but we had ten or 12 and now we run the automated tests. And if there was a problem, they go tell us why, what the problem was.

But they just tested. And they would find the anomalies and the things you didn’t think about because they were able to use. You do that. We kind of got rid of the mundane stuff and they did. The more, more creative or more exploratory type testing. But the same thing with, you know, build even like you have security and compliance.

We had to do a penetration test. Well, again, let’s build into our build process checking for the known vulnerabilities so that our security people can look for the things that we don’t know. Right? We know that you should never write SQL code this way. Okay, then let’s just make it impossible to get that into the repository or into production and have them looking for the things we haven’t thought of.

Right. Or some of the more the things that are new are harder to automate. So, if things are hard, you know, do them often and automate them if you expect people to grow. So, another big thing we find is we’ll go on the projects. And we went on to a project just earlier this year and they assumed it would take us 21 days per developer to get up and running with our laptop and being able to build the system and run it, which seems like an enormous amount of time.

And it took our first person that long and but they write everything down in great detail by the third person coming on. They had most of that automated. So now if you needed to set up an environment, it takes about two days and there’s a set of manual steps you have to do. But it’s not like a million people join the team, so it’s probably not worth automating the last parts of it.

But just looking at that and having people whose mindset is, well, how do I get rid of all of this stuff? Because what you want is the value density of your software, the amount of time what you’re building in your software are things that customers are actually going to use. So it’ll go up, right? So that means the amount of time it takes to build, the amount of time it takes to deploy, the amount of time it takes to set up a laptop, the amount of time it takes to wait on another team that you’re depending on.

We’re striving to eliminate unnecessary tasks and focus on building what customers want. As customer needs change, we’ll continue to adapt. Automation plays a significant role in this process. I had a client who didn’t see the value in automated testing and viewed it as a cost. During a meeting, a department suggested changing the data handling process, but we weren’t sure if this would break the system’s rules. The client suggested using our automated tests to see what would break if we implemented the change. We ran the tests, identified four areas that would be affected, and were able to move forward. Automation provides the safety net that allows us to change more frequently and quickly without fear.

Alright, so that’s my deck. If you need to, you can get it by texting. And now I’ll open it up to thoughts or questions.

Q&A with the Audience

The promise of Agile is increased responsiveness to the market and faster software delivery. However, if it currently takes nine months to release and we want to do it monthly, but testing takes two or three weeks and deployment a few days, we won’t actually deliver more software in nine monthly releases than in a nine-month chunk unless we automate away much of the overhead. The goal is to spend as much time as possible on tasks that customers value.

The biggest impediment is often miscommunication. We’ve discussed communication with vendors, but it’s essential to communicate effectively with everyone. If a team is trying to transform and the surrounding teams are unaware, or if a team is told to transform without understanding why or what the new expectations are, problems arise. If leadership isn’t communicated with effectively, it can also lead to issues.

Communication is indeed a significant barrier. Trust is another crucial factor. We’re trying to build trust with the business by showing them progress regularly. If we consistently deliver what we promise in small increments, they’ll trust us with larger tasks. This principle applies to transformations as well. Demonstrating better results incrementally can build trust and create more opportunities.

Being able to measure and communicate progress is essential. If we can avoid building something the wrong way or if we become more efficient and need fewer developers, we can pay our team members more while still increasing output.

Leadership changes can also pose challenges. If new leaders don’t understand the journey we’ve been on or the way we’re developing software, it can cause issues. They need to understand the metrics for output. Dennis Stevens gave a talk about trust and how to communicate with executives about Agile. It often involves communicating in their language and continuously re-communicating as leadership changes.

There’s no shortage of constraints on your transformation. The key is to get feedback as often as possible, communicate effectively, and eliminate dependencies. Automation helps eliminate dependencies in software building, as does having a well-rounded skill set within your team. Most problems will likely be related to dependency, communication, or skill issues. Thank you.