Why Your Software Isn’t Soft and What You Can Do About It
One of the big benefits of software over hardware is that it’s supposed to be easy to change. But in today’s world, most companies have accidentally built large, monolithic software applications that have become harder than most hardware. They’re rigid and brittle. They take increasingly specialized knowledge, or an exponential amount of human-power to maintain, and it’s slowly killing company’s ability to operate with agility.
Today, we take a look at why your software isn’t soft, explore the challenges of writing code at scale, and discuss a modern approach to writing code that will maintain its safety and simplicity over time.
Video Transcript
Chris Beale
In traditional software development. We were taught object-oriented analysis, object-oriented design. There were use cases and all kinds of things we did to model software and it assumed we knew what the software needed to do and it assumed we know how we wanted the software to do it in big, large batches. And so we would go do all of that work and then we would start to implement all of that work, all of that design, not knowing and learning would happen along the way. It always does. So learning is happening. Our design is now under pressure from the learning and we have no way to know how to change it in flight, can’t go back to the beginning and how to change it in flight to accommodate really to be adaptive like that. But we’re writing all this code, and we don’t know if it’s going to work till we get it all together somewhere in the future, which is not in the near future.
So, there’s no safety. The testing is done after the software is written as a means of validating whether or not I understood the requirements, whether I actually understood, stood well enough to understand the design, how much learning happened, and we find out invariably everybody does that. Okay. We didn’t understand our design, we didn’t understand the learning, we didn’t talk to each other. We missed a bunch of things in terms of the different developers and teams that are writing the software together in these larger enterprise solutions. And we tend to then find that at the end. And that’s unfortunately too late for us to go do anything meaningful about it because now we have to live with all of that.
Introductions
Mike Cottmeyer
Okay. Hey everybody, thank you for joining us today. Today I have Chris Beal with me, Chris Beal’s, one of the guys on my executive team and leads a lot of our large scale accounts, and today we are going to talk about the idea of why your software is not soft. So Chris, explain to me what you mean by my software not being soft.
Why Your Software Isn’t Soft
Chris Beale
Yeah, Mike. So the whole idea of software was you’re supposed to be able to change it readily without really having to incur a lot of cost and headache, and that just hasn’t been the case. So
Mike Cottmeyer
Yeah, I mean it’s in the name, right? Soften the name
Chris Beale
Software. And so that was the whole idea from the very beginning is it was a lot easier to change that than it was the hardware. And in reality that’s probably even more today less true. That’s easier to change all that stuff than it is to change the software. The reality is the software, it’s brittle, it’s hazardous, you change it, you don’t know what’s going to happen. You have a lot of complexity, you have just a lot of headaches every time you want to change it, especially as software has grown up to the enterprise, it’s gotten more complex and scaled. And so the problem has become worse, the complexity, it doesn’t increase linearly with the scale, it actually increases exponentially. So we’re just, as an industry, we’re really struggling with large complex software systems that are really difficult to change.
Mike Cottmeyer
So, when you and I have talked about this in the past, as somebody who doesn’t write software every day, I go back to the Kent back and the Martin Fowler stuff that we were reading 20 years ago, and it seems like these principles are pretty well understood.
What Keeps Software Soft?
Mike Cottmeyer
So, the question I have for you is what are those principles that make software soft and why are they so difficult to implement at any kind of scale in most organizations?
Chris Beale
Yeah, so probably the most fundamental, the way I think about this, the most fundamental thing when I think Martin Fowlers, he used a definition of architecture that I really liked that which if I have to change, it kills me. So anything can be architecture. If it’s easy to change, I don’t care. So I don’t need to future proof it. I don’t need to have open-ended anything. I don’t need to get super complicated. If I change it and I know it works and does the right thing, then it’s not really an architectural concern to me. So even implementation can be architecture from that regard. What keeps software soft is simplicity. It’s the ability to have very simple things. So there’s a principle called single responsibility. Any little piece of software should do one thing and one thing only. It shouldn’t do five different things. So you take those things that do one thing only, you put them together in a loosely coupled way, and now you have software that if I, it’s at least I understand it. But the other one, and this is something I really like in my time at Leading Agile, I’ve been here as of this April, about eight years. Wow, that’s crazy. Is when I came here, I really learned from you and from the team here, this notion of safety.
So safety is a first class concern. When we’re doing transformations, we want to make sure that we’ve got the change management in place so that when we’re asking teams and organizations to change how they work, to let go of what they’ve been doing because what they’ve been doing is perfectly producing the results that they’re getting and that they have to redesign and do something else, we have to create safety for that and work them through it in a way that’s measurable and demonstrable and it builds on itself. Success builds on success and courage builds on courage and all of those things. Well, the same thing applies to if I don’t know what’s going to happen when I change it, then I can’t work in it safely. And there’s two things that’ll happen there. So simplicity now comes under pressure when I don’t know what’s going to happen, if I change it, if I’m thinking about the future, then I’m going to try to futureproof it.
I’m going to make it more complex upfront, so I will start to violate the simplicity by over architecting it with some open-ended thing I want to do in five years from now. So now everything today is harder because of the thing we want to do five years from now. The other end of it though is as a developer, and I still do write code, if I don’t know what’s going to happen when I change it and I’m under deadline pressure, I’m going to take shortcuts. I’ve got to get this done. I can’t go create a bunch of defects and blow it up, so I’m just going to create a copy of it, I’m going to change that copy, and then somebody else does the same thing and before, you know, got 13 copies of the same thing with slight moderation or slight alteration, and now you’ve got complexity. So anytime you need to go change it, you got to go change 13 things. So what keeps software soft is simplicity and safety. If I don’t have those things, I can’t have simplicity if I don’t have safety.
Technical Practices and Creating Safety in the Code Base
Mike Cottmeyer
So I’m going to go back to what I think is kind of ground level on this. When I was first learning about XP 20 years ago, again, not as somebody who’s writing code every day, there was this idea of unit tests. There’s this idea of test driven development. There’s this idea of continuous integration, continuous deployment. Talk to me a little bit about how those technical practices create safety in a code base.
Chris Beale
Yeah, I think to understand that, Mike, you have to understand how the industry’s been approaching this problem and why brittle monolithic software is pretty much what everybody deals with. In traditional software development, we were taught object-oriented analysis, object-oriented design. There were use cases and all kinds of things we did to model software, and it assumed we knew what the software needed to do and it assumed we know how we wanted the software to do it in big, large batches. And so we would go do all of that work and then we would start to implement all of that work, all of that design not knowing so, and learning what happened along the way. It always does. So learning’s happening, our design is now under pressure from the learning and we have no way to know how to change it in flight, can’t go back to the beginning and how to change it in flight to accommodate really to be adaptive like that.
But we’re writing all this code and we don’t know if it’s going to work till we get it all together somewhere in the future, which is not in the near future. So there’s no safety. The testing is done after the software has written as a means of validating whether or not I understood the requirements, whether I actually stood well enough to understand the design, how much learning happened, and we find out invariably everybody does that, okay. We didn’t understand our design, we didn’t understand the learning, we didn’t talk to each other, we missed a bunch of things in terms of the different developers and teams that are writing the software together in these larger enterprise solutions. And we tend to then find that at the end. And that’s unfortunately too late for us to go do anything meaningful about it because now we have to live with all of that and we have to just get it working. So the techniques you’re talking about, they established, actually you had an insight share it.
Mike Cottmeyer
What I was going to say is that what’s interesting is approaching it from more of a project management organization perspective. That idea of big upfront design and late testing kind of got codified into a lot of our process governance around there. When you think about the impact of the waterfall methodologies and stage gate governances and things like that, we kind of took that assumption and we actually codified it in terms of how we manage software projects and how we approve the funding and things like that. So I guess what I was just saying is that we’re making it worse a little bit by codifying some of those practices into our governance models.
Chris Beale
Yeah, absolutely. So now as a developer, even if I know the right thing to do is to build in small batches and test and integrate along the way, it’s really hard to do that. The governance model doesn’t support it, and the organization’s going to actually get angry at me if I try to do it. I have to be heroic at that point and stand up against the organizational design and the governance model in order to actually get learning out of the system and get all that stuff in smaller batches to work. But really with these practices you’re talking about with extreme programming, test driven development, continuous integration, everybody’s kind of munged them together in this dev stuck ops kind of bucket. But really what that was about was first of all, I’m going to write tests before I write my code and ostensibly not, it wasn’t really about testing, it’s about learning.
I can’t write the test if I don’t understand the requirement. So as a developer, it’s impossible for me to move forward if I don’t understand the requirements. So it’s really a conversation to get on the same page, just to write the test. But what those automated tests then gave me was safety. So now I can go implement that requirement. I can know that that requirement works and does the right thing and I can instantly know or near instantly know whether everything that used to work and do the right things, whether those things still do. And so I can work courageously and I can actually simplify the software. If I find copies of things I have safety, I can actually say, okay, it’s called the Boy Scout rule. Leave the campground better than you found it. I can say, you know what? That’s just complexity we don’t need, and I can collapse that into a single implementation and as long as the tests run, I’m good.
Mike Cottmeyer
Well, is there angle, I think I learned this one from Bob Martin, the idea that you write the simplest amount of code that will possibly satisfy the test and then over time through refactoring, you can actually make that code simpler and simpler and simpler and simpler. So it’s not only safety, it’s the simplicity factor that you mentioned as well. Is that right?
Chris Beale
Yeah, so the safety allows the simplicity. The first thing I want to do is to make sure that I understood it, it’s working, and then I look for opportunities to make it better. There might be situations where I’ve got maybe some larger modules, I can make them smaller. I have duplicate code, I can consolidate that. I have things that aren’t named very meaningfully for the other people. It’s great that I know it, but other developers are going to be in there too. So how do I make it meaningful for them when they come in that they know what’s going on? They can always look at the tests to understand the code, but the bottom line is the learning, the purpose of those tests was to get on the same page and understand, but the side effect you get is safety. And now I can work to clean up the code. I can work to keep it simple. I can keep our maintenance costs down. All of that stuff can happen, but I have a safety net. I know that it still works,
Mike Cottmeyer
Probably just pulling a little thread of interest to me. I think a lot of that’s critical for this idea of shared code ownership too as well. If I’m going to have lots of people working in the same code base, I have to have a way to know that I’m not breaking anything as well.
Chris Beale
And so you can find large organizations will create safety by, so some of the digital natives, they create safety by creating static teams that own one thing and own it for years. And so the four or six developers or eight developers on that team, they tend to grow. They all know they have tribal knowledge of that piece of code. They don’t understand the rest of the code, but they understand that piece of code deeply and tribally and natively, and so they can mentally understand the implications of any potential change. Mistakes still get made, they still make mistakes and they still break things, but that’s the attempt at creating safety. They
Mike Cottmeyer
Just get really good at rolling them back and load balancing and that kind of stuff?
Chris Beale
And so that’s the cost now. But the problem with that model is the challenge with that model is they don’t know all the interactions and how the code works across all of those teams. So code won’t just work within one team that does one thing. You need a bunch of teams and a bunch of their code to work together to do something interesting, and no one can see that. No one knows whether that still works or does the right thing, all of that integration. And so when you roll it out, you have to understand how to roll it out to very small slice of your customer base and then listen. And if it doesn’t break, roll it out to more, and if it doesn’t break, roll it out to more and finally maybe roll it out to everybody, but then you better be good at rolling it back because if it does break even that 1% of whatever market that is doesn’t want to use a broken system. So you have to be really good at rollbacks if that’s going to be your safety model.
The Challenge of Safe Code Practices at Scale
Mike Cottmeyer
So that actually surprised me when you made that comment that a lot of the digital natives aren’t even doing these safe code practices that we’re talking about. What makes it difficult as you start to scale this up? We go from eight developers to 800 developers.
Chris Beale
It’s not really a scale issue per se. The challenge with the digital natives that they have is they grew up small and tribal. Two pizza teams, great. That was all good stuff. They can move fast, but they also give a lot of autonomy to their teams. So they don’t generally tell their teams how to write software. So you are together, you are the team that owns this capability. So some teams will choose to use these techniques. They will test drive their code, they will write tests, they will make those tests pass, and then they’ll refactor for simplicity. But many of them, they believe they’re faster without the tests or they don’t create bugs or whatever it is. So what you get is kind of this just really preferential kind of approach that varies across teams, even the technologies they’re using. And some of these companies are very different based on what the teams, each individual team wants to use. And so it’s difficult to scale that because it’s kind of chaotic. So how do you get an organization of hundreds of those teams to work together when they’re working differently? And that can be difficult. Again, it goes back to rolling out, rolling back.
Mike Cottmeyer
Does the two pizza team strategy, does it at least lend itself to not building monoliths
Chris Beale
In a way, but it’s still possible to build monoliths within each two pizza team. So to still create dependencies in the code that make it difficult to test in isolation. So to know that something I’m going to add over here still works and not have to mess with the rest of it. So there are still monolithic things that grow up and sometimes even between teams in those companies where now you’ve got something that teams can’t release independently, they can’t test independently, they have to in effect the dependencies even though the code might not be compiled together, the effective strong dependencies requires them to test together. They can’t test in isolation. So different maybe styles of monoliths emerge, but at the end of the day it becomes monolithic.
How to Decompose a Legacy Monolith
Mike Cottmeyer
So, we see the effects of that a lot of times when we’re doing an early stage agile transformation because a lot of times our early stage transformation work is teaming strategies and how we’re feeding the system and governance and metrics and trying to get the organization in alignment, what we call Basecamp one, Basecamp two. By the time we start thinking about Basecamp three, we’ve hit some of those dependencies and we’re starting to think about pulling things apart. When you find yourself where your agility is limited because of the strength of the dependencies of the monoliths or the poor coding practices, lack of safety, too much complexity, what does an organization do to begin the process of creating safety, creating simplicity, understanding, I don’t know, decomposing the big things into small things? What does that start to look like?
Chris Beale
Yeah, so when we come in some of our engagements, some of the engagements I’ve been on, I’ve done early cycle efficiency analysis. That’s basically what percentage of the capacity the team has spent on value added activity, like adding features for the customer and creating value for the business versus non-value added. The whole ball of work they do, what percentage of that’s actually adding value for the customer? We have a customer up here in Michigan where I did that and the number was 5% cycle efficiency when we found them. So 5% of the team’s time was spent actually adding new things for the customer when we worked with them to actually, what I think is great about our Basecamp model is the killers of teams and of flow and of value, unmanaged or unclear priorities, unmanaged dependencies, and then really unnecessary complexity. Those are the big ones.
There’s others, but those are the big ones. What we’re doing in our base camp one base camp two thing is saying, okay, let’s get your priorities straight. It can’t be 10 number ones, what is number one and how do we actually decompose that to be something small enough to put in the system that’s still really valuable to do and the customer’s going to love and it’s going to move the needle. We get the priorities under control, we get the dependencies encapsulated so we’re not orchestrating dependencies all the way across the organization or across hierarchies of the org chart or whatever. How do we group those capabilities and teams so that the dependencies are minimal in terms of what they’re imposing? And what we saw in that area of this customer was when we did that, we improved their cycle efficiency from 5% to 25% just by getting their priorities straight, their dependencies in order, and then by breaking down things into smaller pieces, we moved even more, but it was really the introduction of these kind of safety driven engineering practices.
There’s still 40%, we’ve seen 40% improvements, 40 to 50% improvements in cycle efficiency. Once we get at those dependencies and create safety for these engineers to work courageously, not have to be forensic when they go make a change, not have to make the software slightly worse every time they touch it, they don’t know what’s going to happen to be able to get courage into their development cycle, I know what this did, I can fix that or I didn’t break anything so I can just keep moving. It really does unleash unlock a lot of the team’s capacity, not just in reducing or eliminating bug fixes that happens, but in not having to coordinate with each other all the time. I’ve got a stable API can make sure that contract is always stable. I have tests covering that. I know that it works and does the right thing even when I make changes to it, my customers aren’t going to get broken APIs. But also just the fact that working in simple code allows me to do things faster. I don’t have to go through some really complex analysis of how to make that change. So there’s so much productivity that frees up once you break those dependencies between teams in the code that now developers can work courageously.
Exploring App Modernization and Engineering Modernization
Mike Cottmeyer
So Chris, one of the things I was thinking about, there’s a lot of concepts that get floated around the industry, product extraction, app modernization, engineering modernization, maybe even into the idea of product driven organizations and composable enterprises. Talk to me a little bit, maybe just start with the app modernization, engineering modernization. What does that mean in industry relative to the kinds of topics that we’re talking about?
Chris Beale
Modernization is an interesting one for me because applications are, to me just natural places where monoliths occur. It a container, it’s a monolithic container of capabilities. So if you’re thinking about single responsibility principle that small things do one thing and then bring them together in some loosely coupled way, what tends to happen is our application oriented brains that we’ve grown up with in this industry promote the notion of monolithic multi responsibility software that’s hard to change. So what I think is cool about what we do and really looking at product driven organizations and figuring out how to help organizations move from monolithic software development organizations to product driven is we can take a lot of the things that Kent Beck and Martin Fowler talked about and actually understand, okay, but how does that map into how does it lay up against the organizational structure? What do we form teams around and how do we govern that and create clear priority and encapsulate or break dependencies?
So the notion that we have products and product teams, we can say, okay, we have this monolith application. Where do we break it up? Where does it make sense to cut it? And so when we’re looking at that, I think we’re looking at it differently than the industry does. The industry wants to re-platform, it’s got old cold fusion or something like that and it wants it on a modern technology. The problem is if you use the same practices that you use to write the thing in ColdFusion and get a big monolithic thing that’s complex and hazardous and hard to change, use the same practices and approaches on whatever technology you’re moving to, even in the cloud, you’re going to get the same thing. It’s an industry problem. Those practices naturally result. There’s the old saying, all systems are perfectly designed to produce the results they’re getting.
And so those developers, those practices that structure, that governance is going to produce the same monolithic thing. It might start out a little differently, but over time I guarantee it’s going to end up being the same. It’s going to be complex, hard to change. You’re going to spend 80% of your time keeping lights on and you’re going to have a hard time getting this stuff done. So moving from app to product, you’re already starting to modularize and create simplicity, really responsive or single responsibility in the organizational structure. How do I apply it not just to the software, but from you And I have talked a lot about domain-driven design. How do we start to think about the responsibilities of the teams and what they own and so that we don’t have duplicate responsibilities and duplicate code and duplicate systems. So it’s kind of cool to be a transformation company with technology capability like this and be able to bring those two things together in a way that informs maybe the first thing we need to do is just get the teams to work independently.
We don’t need to fix everything, but let’s just get the teams so they can release, they can change and release and test independently. Where do we have to make those cuts? That’s the domain driven design stuff, the capability modeling stuff we do. But then within that, so we’ve got the hard crunchy shell, we’ve got stable test covered contracts, what do we do to make the software more simple? Within each one of those things, you can make smaller decisions. Maybe we replatform only a little piece because that piece is changing a lot. That team software that we’ve broken apart has a lot of change pressure and it’s an old technology that we don’t want to stay on. So you can start to make economically informed decisions on smaller things versus an entire rewrite of everything, some big large amount.
Why Apps in the Cloud Become More Expensive
Mike Cottmeyer
So one the things, I’m going to back up a step and anchor it to a business problem that I see a lot with our clients and in the industry in general, when you do that kind of lift and shift without the app modernization or any of the product extraction work, what you end up doing is almost inevitably driving costs, moving to the cloud. Talk to me a little bit why a legacy app in the cloud is often more expensive to run.
Chris Beale
So, there can be a number of reasons for that. Generally the cloud business models rely on some sort of utilization. So I’m utilizing compute, I’m utilizing storage. I might not even get my data into the cloud and I’ve got the app in the cloud, but if I need it to scale, I’ve containerized it. I’ve done whatever I’ve done to move it to the cloud, but I haven’t really modernized or rationalized that monolith that application into smaller things. I’m scaling the entire thing. So if I want more of it, I have to create more of them. And so I’m using a bunch of stuff of capacity that I don’t need to maybe make a minor uplift in scale. And when you actually look at those costs for storage and for compute and all of that, it turns out that you get surprised, right? There are other reasons that maybe your contract, how you negotiated your contracts or whatever. But from a pure technology play perspective, yeah, it gets predicated on some model that, okay, now I can turn this thing off over here and I don’t have to spend money on that. But then when you do have to scale it up, you’re going to be creating capacity, utilizing a lot of capacity that isn’t incrementally related to or proportional to the demand you have.
Mike Cottmeyer
So, in the context of what we’ve been discussing, what’s the answer to recouping those costs?
Chris Beale
Well, I think that’s part of it, right? The other challenge you have is if you don’t, let’s just say you did the modernization, you broke it apart, you did the rationalization piece and you were able to cut out the pieces and now you can individually scale any of them up and not scale the rest of them. So you can really control the scale at you’re increasing your costs based on whatever your business is needing. If you were successful at doing that, but you gave those systems then to those capabilities, those products to the same teams that gave you the monolith in the first place, guess what they’re going to do? They’re going to rebuild it, so they’re going to just rebuild what you had. And so you have to move the technology and the capability and competency of the team together.
Mike Cottmeyer
Yeah, that’s an interesting point. So if I could recap, what you’re basically saying is if you have a legacy application in the cloud, and in order to scale the application, you have to scale the entire application that’s going to drive up storage and compute. If you pull it apart, then you have the ability to only scale the small components or whatever that actually need to scale. And so you get the economies of scale there, but then you took it a step further and said, if we don’t really bring the engineering organization along and the product organization along, they’re going to do the same behaviors, overloading the systems, creating lack of safety, the poor engineering practices, and they’re going to in effect, rebuild or instantiate the monolith in the cloud or at least not be able to maintain the investment that you made.
Chris Beale
It will become more complex and again, it’s an industry problem. The way we were taught to write software as an industry from the beginning naturally produces more and more complexity, monolithic systems, things that are hard to change.
Mike Cottmeyer
I’m glad you hit that. You said from the very beginning the way we’ve been taught, what is it about the way we’ve been taught that drives that?
Chris Beale
So the practices of thinking upfront and not working in safety and not writing code and then testing it will naturally produce. So without safety, I don’t know what’s going to happen when I change it. I’m going to make it more complicated when I put it on the drawing board to future proof it. I’m going to borrow risk from the future to make it more complicated today, because that’s how I think about safety now is through more complicated open-ended designs and architectures.
And then I’m going to, as I’m changing it, I’m going to make short sighted decisions. Software doesn’t get complex generally in one decision. It’s a thousand small ones, thousands, thousands small ones, and it just keeps getting worse. It’s the whole bathtub curve that we used to learn back in school, but it’s the way that people are taught. It’s not that you have stupid developers. It’s not that you have people who don’t know what they’re doing or whatever. It’s just the techniques we’re using naturally produce that. The other set of techniques that have been around now 30 years plus, those techniques create safety first, and now I can work safely and I can create simplicity. So it’s really just a matter of the tools I’m using, the techniques I’m using, whether they naturally promote complexity or simplicity because of safety. But the industry has never really reached a critical mass on those newer techniques. There are pockets of it that pop up in certain companies with forward thinking leaders, but you just have never seen the tipping point hit in the industry.
Escaping the Mindset of Monolithic Software Development
Mike Cottmeyer
So there’s an interesting thread. There’s almost want to ask you three questions simultaneously. We talked about this a little bit as we were kind of thinking about how we would run this conversation. Why is the industry not able to hit a tipping point with these practices? I mean, I’ve been aware of them for over 20 years. You probably even longer. I mean this isn’t new. Is it the university system? Is it our uptake? Help me understand a little bit.
Chris Beale
Yeah, so number one, yes, I have visited universities and talked to their faculty and the people who are leading these programs in these schools and very few of them, it’s mostly some of the smaller private colleges where it’s easier to change curriculum. So you can find little universities, little private universities that teach software development with these new sets of techniques and approaches. So it’s mainly there in large universities though the school that I went to, my son went to 20 something years later, was taking the same courses with the same materials and the same tests. And so it’s funny because I could actually remember when I looked at his stuff, I could remember taking the same exam. That’s probably something in computer science. So that’s kind of not a good thing. So it’s really hard for the universities I think to make that change because really stuck I think in that old kind of upfront mentality of how you write and think about and write software.
Mike Cottmeyer
Well, so I want to just add before you move on, I think I might know where you’re going. The university where I went to school, the University of Florida, which I don’t mind mentioning, and I do a little bit of classroom work there as well. I know these kids are getting exposed to Martin Fowler. A lot of them seem to know what unit testing is. Is it more the mindset of doing the upfront design? I mean, it seems like the kids are getting introduced to the ideas but maybe don’t know how to apply the ideas effectively or how to think through incremental and iterative development. I get a sense that that might be the thing. They don’t really know how to think about building a product incrementally, they think about building it in its entirety. Is there any merit to that?
Chris Beale
So, I think it’s an interesting angle to take. You can think about it this way. So writing a unit test, first of all, I’ve seen unit tests that are monoliths. So what does a unit test even mean? How do I know I’m actually writing a test that tests one thing in isolation by? And so there are still problems. There’s unit testing frameworks, but they can be used to write very large monoliths tests that try to validate a lot of different things. So first, if you get, let’s assume that they’re actually learning how to do what are called micro tests. So testing one behavior, you still have a situation where, think about it as learning. If you were going to build furniture and you’re learning how to do different joinery and you learn how to do a Morrison tenon joint, cool, when do I apply it? So being able to write a unit test is great. Knowing that you write that test, that micro test in a cycle of safety to say, I write that before I write my code. I write just enough code to make that test pass and then I go look for opportunities to make the code better and simpler.
That’s an application of that technique in a specific frame, a frame of practice that has a point of view just so anybody can write unit tests. If I’m writing my unit test after I write my code, I don’t have safety,
Chances are my code isn’t very testable in isolation. I wrote the test after. If I’m writing the test first, the code has to be testable. So it really isn’t just about the tool or necessarily about the technique, it’s about the practice that leverages the tool and technique in a way to create safety and simplicity, doing it the right time, the right way for a reason as opposed to just having a bunch of unit tests that I run at some point but aren’t very small, they’re just reinforced. I’ve seen actual unit test frameworks that when the unit test is written, it expresses a big upfront design. So the test itself actually mandated a design monolith that was monolithic. So you can write tests to do that.
Mike Cottmeyer
I don’t know if this is a thread to pull. I think I’m going to pivot here in a moment, but going down that path, I was actually at a university event last week talking with the leaders of the Entrepreneurship and Innovation Institute, and we were actually talking about that the problem might even be a little bit more pervasive. It’s like in our education system, often we’re rewarded for having the right answer rather than experimenting being wrong, learning, adapting, changing. And so what we were talking about in the context of innovation and in entrepreneurship, I mean so much of being an entrepreneur and innovator is trying things, being willing to fail, pivoting, learning new stuff. I wonder there’s a part of me that wonders if it’s even more fundamental than learning how to apply a technique or when a technique, there’s even a more innate lack of safety than what we’ve discussed so far. Any thoughts on that?
Chris Beale
Well, so where mind went to when you asked that question was, okay, imagine. So that’s the space that the university lives in. So experimentation, learning, they’re all about the learning. The thing is kind of cool, but it’s really about the learning and pivoting and the adapting and things like that. Imagine as they were doing that as universities, they were creating something and that something all of a sudden started to push against them in terms of their ability to pivot.
It started to build up brittleness and now they want to change, but they don’t know what’s going to happen to that thing every time they do. And so it’s going to stifle learning. It’s going to slow down, fail fast, it’s going to become the thing that we’re talking about. And so I just thought it was an interesting way to ask the question because that’s exactly the application to software is we don’t want the thing to slow us down from learning and from pivoting and from adapting. So I think with the university system, though I’m not a professor, I don’t teach courses. I think it’s challenging. There are sciences and curriculum in university systems that are very, you go get a researcher if you want to invent some new material or something like that. You got PhDs and colleges working on that stuff. Computer science evolves so quickly in industry. There are new frameworks, there are new techniques, and it’s not coming from the universities, it’s coming from people who have time on their hands and are really passionate about it, or it comes from Google or there’s all kinds of places that all this innovation is coming from and the university’s trying to keep up. They’re trying to stay relevant in an industry that’s moving really and adapting really, really fast.
Mike Cottmeyer
So, it’s almost like we have to get these kids out of college and then put them in programs inside companies that are really instantiating this way of thinking and this way of acting.
Chris Beale
Yeah, I mean the lifespan really, the lifespan of a new JavaScript framework is like nine months. It’s things just don’t sit that long. They don’t last that long. They’re just constantly coming up with improvements because it’s so democratized. So a university has a hard time maintaining central excellence and innovation around that because it is so federated and democratized.
Mike Cottmeyer
You had made an interesting point. I was asking a question about, again, why are we reinventing every 20 something years, how we’re thinking about this? And you said something about the rate at which we are growing the number of software engineers in industry. Tell me a little bit about that and the impact that you think that has on this problem.
Chris Beale
Yeah, so back to your question about why haven’t we reached a tipping point with these new practices? There are organizations that do it this way, it’s how they do it, and the developers learn like, oh my gosh, I can be so much more productive and I’m not fixing bugs. I’m not having to work in really hostile, hazardous software. The challenge you have though is generally the number of software development engineers in the world has been doubling about seven, eight years every seven or eight years. And then you’ve got people leaving. It’s also you got to keep up. It’s an industry where it’s constantly changing. And if you’re getting a few more rings around the tree, like me, hard to keep up with those kids. And so what you get is people moving out of the industry that may have this belief and may have this, they might be writing software anymore, they do something else. And so the fact that you have the systems that are teaching people to write software, teaching them the traditional ways that naturally produce monolithic, brittle software, they’re the ones that are actually filling the need for the most part. And so they’re pumping out new developers that use the traditional methods. There’s not really something that’s filling the world full of engineers that natively approach it with these kind of safety driven newer methods. And so you just never reached a tipping point because you’re always putting more of the other stuff into the system.
Mike Cottmeyer
Yeah. Okay. So as we wrap up, here’s the hard part of the question. So, we know why your software is not soft.
What Can You Do About Software That Isn’t Soft?
Mike Cottmeyer
The question is what do you do about it if your software is not soft?
Chris Beale
Yeah, so few different parts to that. So we recently helped a customer of ours very, very longstanding, and I think close partner of ours as a customer, large organization. They had this problem in a key system and very large set of teams. Hundreds of developers were on this, mostly outsourced, so staff fog type developers, some internal, but using these methods. And they had a big monolithic system that all of those teams had to figure out how to change, very, very difficult defects, things like that. So tough to release. So we came in and we helped ’em, and one of the things we helped them do, Mike, was to actually revamp their talent organization to bring in people who knew how to do this. So find the people who have these capabilities to write software this way in this kind of safety driven way, and basically start to replace those contractors with FTEs that knew how to work this way.
So that was part of it was let’s make sure that we’re not always having to train whenever we hire anybody, it’s somebody from the traditional camp and we have to retrain them or whatever. So we did that part. We also brought in some folks on the ground who helped to train the folks that they had so mentor and train them in terms of this approach and move the software forward. So we help them to figure out where to break it up, where are we going to cut it, make those cuts, where do we need to simplify? And the metrics were very compelling as a result of that. I was personally thanked at a lunch by the gentleman that runs the organization, and I found that to be very edifying for us to have somebody say, you made a huge difference. We were really kind of slow and difficult and now we’re really cranking.
And if you look at the quotes that they gave us from the people who were working that system, also very, very personally edifying for me, for people to say, we actually get to write software. We know it works. We’re adding value for the customer, we’re moving faster. All of the positive stuff around that, but you have to deal with the ecosystem. Leadership had to support it. If leadership doesn’t support it, you can’t have you write in those automated tests. I need you to just write and code can’t work in a capacity informed way. I just need to figure out how to get it done some heroic way. There’s just different things you have to do from set up the conditions. And once as you have the conditions in place, you have to make sure the system inside, so the people that you have, the leadership you have, the approaches you’re using, the tools, all of that have been transitioned from the traditional approach to software development to these new methods.
Mike Cottmeyer
Cool. Well, it’s a fascinating conversation, Chris. So thank you for joining me today and I look forward to continuing this and maybe unpacking this a little bit further.
Chris Beale
Yeah, sounds good. Thanks, Mike. Appreciate it.