Agile At Home: Should You Use Agile To Build Your Next Home?
Before we start our conversation about Agile and governance in the structure-governance-metrics framework we’re building out, I want to take a minute and see if I can finally tell you guys about the house my wife and I were planning to build this past summer. It’s an important conversation because we need to establish a shared understanding around what it means to plan on an agile at home project.
Scenario One: Team Agile at Home Builder
My wife and I decide we want to build a new home. We find a piece of property we like in a nice neighborhood. First, we find a builder we like and think we can trust. My wife and I have a really good idea of the type of house that will work for us and our family, and we decide that we want to spend somewhere around $500K to get the house completed. But, it needs to be finished sometime before our kids leave school for the summer.
Next step is to reach out to the builder and see what he can do.
The next day we show up at the builders office and tell him about our plan for our new home. The builder is really excited, seems competent, and is very confident he can get the house built within the time and cost constraints that we’ve established. We ask him what’s the next steps for planning construction on our new home. To our surprise, he tells us that he is an agile at home builder and doesn’t do any planning.
The builder proceeds to explain to us that as an agile at home builder, he doesn’t do any planning. He will meet with us every two weeks to determine our priorities and figure out the next most important thing to build. Every two weeks we can inspect what he’s built and change our mind about anything anytime. He tells me this is really in my best interest because I won’t know exactly what I want until I see it.
When I ask if I’ll have the house I want when I run out of time and money, he doesn’t know, but assures me I’ll have the most valuable parts of the house early. Furthermore, I could actually move into the house anytime during the build process, and if I decide I’ve built enough house at any time, I can cancel the contract early and pay him for only the weeks he worked plus 20% of the remaining contract.
So here is my question… would you spend your money this way?
The Paradox
Most people when they are spending their own money, want to have some idea of what they are going to get when the time and money runs out. They want some assurance that they’ll have enough bedrooms and enough bathrooms. They want to know the acreage of the lot and if they are going to get a one story or a two story house and how many cars they’ll be able to fit into the garage.
People understand they might have to make some tradeoffs later in the build process. They might go over on carpet and have to make some tradeoffs with the lighting, or they might decide they want fancy cabinets and go with a less expensive hardwood floor. Also, they may want to defer decisions about the colors, or the finish, or the bushes they want to put in the front yard. They aren’t willing to leave out bedrooms.
We wouldn’t spend our own money this way, but this is the way we are asking our customers to spend their money when they build software with us.
Scenario Two: Scaled Agile at Home Builder
Fortunately for us, this wasn’t the way that our builder asked us to build our house. My wife and I did meet with him and describe our dream home. We talked about the property we wanted to build on, the number of bedrooms, bathrooms, and the number of cars we’d be able to fit into the garage. And, we also talked about the quality of the finishes, the flooring and how we wanted the yard landscaped. That took about an hour.
After that, we spent a week or two having meetings with the architect. She and I got pretty specific about the room layout, the number of floors, the layout of the basement, and the placement of the house on the lot. We made high level decisions about floors and cabinets, but didn’t pick any specifics, decide any colors, or pick specific appliances, light fixtures, or door knobs. There was way more left open then decided.
What we did decide was the stuff that would drive the cost of the house. The stuff that would be expensive to change later. We went into the process understanding that a ton of stuff was going to change, but there was also stuff that would be really difficult to change, and those decisions had to be made early. We talked about potential risks once they broke ground and buffered in case anything went wrong.
The result was a blueprint and elevation for the house. We had a line item budget for construction costs, material costs and miscellaneous tools and supplies. Every line item factored in a profit for the agile at home builder. Every line item represented some feature or attribute of the house that I would probably care about when the house was complete. We did all that in a few weeks and made no detailed decisions about the home.
Planning Around a Budget
It was interesting, the first cut of the plan was about $200K more than we wanted to spend and we were aware that there was some risk and should probably buffer another $50K just in case. Even though I trusted the builder, I had my CPA and advisor review the plan line-by-line just to make sure all the costs were reasonable and we were making a sound financial decision before we decided to move forward.
As we were reviewing the plan… the builder said something to me that was pivotal for how we were going to manage this project. He told me that I had a 20K budget for hardwood floors. For that price, he assured me that I would be able to get a high quality floor, extra long and extra wide boards, and a very high quality wood with a very high quality finish. That said, my budget was 20K for the hardwood floors.
When it got time to build put in the hardwood floors, it was up to me to choose what kind of floors I wanted to put in. If I decided I wanted a less expensive floor, I could save money and use it for something else. If I wanted an exotic wood like Brazilian Cherry, that would cost more and I’d pay extra. It would ultimately be up to me, but he would work with me to understand my trade-offs and any associated costs.
At Enterprise Scale, Estimates Become Budgets
I instantly made the connection that this was exactly how I coached teams to define, estimate, and plan software projects. You have to have a high level plan, make key decisions early, and have time and cost estimates to present to the customer.
What you don’t have to do is define anything and everything before you even get started. You have to establish budgets to bound your uncertainty.
At the point the builder made the estimate, he was bound to manage the project to that estimate… the estimate became a budget. The agile at home builder had to work with me as the primary stakeholder to help me make tradeoffs to live within that budget. As we break down software requirements, focusing on minimally marketable features and maximizing value, we are effectively doing the exact same thing.
Okay… let me drive this one point home. Just like you wouldn’t build a house with an open ended scope, your stakeholders don’t want to commit to an open ended software project. We have to do enough work up front to put together a credible plan, mitigate risk, and understand costs. We have to estimate the work, set budgets, and make tradeoffs to ensure the product is done on time and on budget.
It’s important to be able to tell people what the project is going to cost and when it’s going to be done. Also important, is to tell people what they are going to get for their time and money. And, it’s important they know the risks and can buffer some contingency. They also need to know what’s locked and what’s open and the kinds of tradeoffs that they might have to make along to way to ensure success.
Why Is This Such a Difficult Problem to Solve?
The problem is that many software teams are trying to build products in less than ideal conditions and they are constantly realizing issues and risks they didn’t anticipate. They are working across too many projects at one time and have external dependencies that can’t be managed or controlled. People are not dedicated to the team. The result is that people are coming to the conclusion software can’t be estimated.
Going back to the house analogy, it’s as if we are building on rocky soil and are constantly finding boulders that have to be broken up and hauled away. It’s as if we have contractors that never show up when they say they will. It’s as if we are constantly changing our mind between a one story house and a two story house or deciding we want a basement after the foundation has already been poured.
Summing It All Up
The answer can’t be throwing our hands up and refusing to make a commitment. We might have to pay a few thousands dollars to determine if the build is feasible. We might have to pay another few thousand for soil tests to make sure the land can accommodate our new home. You can spend money to mitigate risk. Sometimes you need to do some work to learn what you don’t know.
No estimating, no planning, and not committing won’t be the answer for most organizations. It’s not a starting place anyone can live with.
You have to ask yourselves what it is going to take to stabilize the delivery process. Forming teams that stay together helps, that was my previous post on structure. Having a governance model to control the flow of work helps too, that’s my next post. Having a means to mitigate risk, invest in discovery, establish baseline metrics, and the ability to measure improvement over time all play a part. More posts to come!
#noestimates is a #nonstarter
Parting Thoughts
One final note… Kimi and I never ended up building this house. The neighborhood was beautiful. The plan and elevation were beautiful. We really liked our builder and trusted he could deliver our dream house within the time and cost constraints we agreed on. The problem was that housing prices were depressed in the county we planned to build in and the appraisal came in at 60% of the build cost of the home.
We could have moved forward anyway, but would have been instantly in a house that was worth less than we paid for it, with no indication it would recover in the foreseeable future. So sure, we spent a bunch of time and mental energy, and even some hard cash to find out we shouldn’t build the house. Had we built the house using mainstream agile thinking, we’d have found out too late it wasn’t worth what we’d actually paid for it.
We spent a few weeks of time and a few thousand dollars but avoided what would have been a very, very expensive mistake. That’s not waste in my opinion… that’s good risk management.
Check out the previous post, Shu Level Agile Isn’t The Same As By-The-Book Scrum.
Comments (13)
Lukasz
One tricky point – which is valid in enterprise software as well as in house building market:
who is going to pay those who are working on high level project?
More: what if ‘requests’ needing clarification are coming frequently enough to keep team busy – but stakeholder do not buy the planned project?
Peter Saddington (@agilescout)
Nice story.
I like it.
I have a story about this too. The difference would be that we’re teaching our builder and architect to be more agile in certain things…. and we’re still executing on building our home :)
Glen B. Alleman
Mike, nice discussion. In our “design, develop, deploy” world of enterprise ERP and large government software intensive systems, there is always a surprise. It’s just part of the domain. The myth of course – used improperly by many – is the big design upfront, committed budget, and “follow the plan” approach is how we work. That’s neither factually nor notionally true.
What does take place is a incremental, iterative process of discovery and containment. This is the basis of Systems Engineering process. The Vee approach. The Systems Engineering paradigm is rarely found in basic SW Development and not used enough in enterprise IT. It is mandated through contract flow downs. This SE approach integrates all the interdisciplinary roles on the project through an Integrated Master Plan where the “increasing maturity” of the deliverables http://goo.gl/QglVk is defined in a sequence that provides the best value to the stakeholders and allows for down stream chanegs when new capabilities are needed, while perserving the delivered value to date.
But one of the core myths is around the budgeting process. The “all in budget” is estimated as part of the architectural assessment of the project. This is not the technical architecture, but the programmatic architecture. “How are we going to get this thing built on budget, on time, to deliver the needed capabilities.” This is similar to the programmatic architecture of your house. “How is the house going to get built while providing you what you want and adhering to building codes, structural integrity and at the same time dealing with unforeseen events.”
While building our house here in Colorado, we discovered granite structures on the lot (sloping walk out basement) that were unforeseen. After blasting completed we consumed some of our Management Reserve and were back on schedule.
I’ve come to realize during the discussion of #NE, that the developers view of the world fails to consider several critical items
– budget and funding are a customer centric view. It’s their money. In our house examples, it’s our money not the builders. Having a partially built house or a partially built ERP system, or a partial delivery of avionics capabilities for a stealth fighter is not good. Oops, the partial avionics capabilities for F-35 is the case. Rats.
– Knowing in advance all the requirements is simply not possible in any domain. So the notion that traditional process do this is misinformed at best and naive at worst. This is the purpose of the Systems Engineering Vee and the Integrated Master Plan. Successive revealing of what is possible to meet the needed capabilities of customer.
– Every number on a project is a probabilistic number, driven by underlying statistical processes. Behave accordingly. The amount of misinformation conveyed in the #NE discussion is mind staggering. I don’t attribute this to the participants. I blame the public school system for not teaching probability and statistics well. For full disclosure our daughter is a teacher and agrees. Without a probabilistc model of the project, no estimate can be correct. Thsi lack of understanding results in bad management. This is not confined to “dilbert bosses”, but seen on large complex problems as well.
Much has been written about how bad estimates are used badly. But suggesting that we not estimate seems off at best. In the end it’s about the fundamental of all business
“Value is produced in exchange for cost” For your house, for my house, for an SAP installation, for Joint Strike Fighter, for the next generation GPS (my current gig as a program architect). Value can only be determined by knowing the cost. Only then can we assign units of measure to the value.
The value of of kitchen remodel was predicated on the cost. A really nice 6 burner stove with double self cleaning ovens is “highly prized” on new kitchens. “At what cost” You can spend $20K or you can spend $6K same value possible.
Possibly (probably not) those suggesting we no estimate could learn a bit from looking at mission critical projects like building your house or updating an insurance provider network, and producing a new product ready for market at the planned launch time for the planned cost so the ROI allows to stay in business. But I sense that’s not the world they work in.
Thanks for the comment Glen. I know I say this a lot, but I think I have more empathy for the situation that the #ne guys find themselves in, and why that segment of the agile community has given up on estimating. I’ve developed some language… not much I’ve written here yet… to help people understand where they are, where they are trying to go, and how to get there. As LeadingAgile has grown as a company, I find that we are acting more in a management consulting capacity, than a training or coaching or project delivery, capacity. There is so much that we can’t even talk about with customers regarding delivery, balancing capacity against demand, flow of value, metrics or estimating until some of the underlying organizational disfunction is fixed. Nearly every engagement of any size starts off with a recommendation for how to organize, how to govern, and what to measure. Once we have agreement on the product delivery mechanism, we can start to implement some discipline around estimation and delivery. We help companies that couldn’t estimate, and thought their backlog was unestimateable, get better at it in a relatively short period of time. Estimation isn’t hard, but getting an organization to set rational expectations around what can be delivered, often is nearly impossible. If developers don’t have the right inputs to estimate, a strategy to manage risk and uncertainty, and a leadership team that can deal with reality… they give up. I think that is what we are seeing.
Thanks again for the comment. I really appreciate and respect your point of view.
Henrik Ebbeskog
Great post!
One thing that comes to my mind is that building software isn’t the same thing as building a house. Building software is much more uncertain and contains more “unknowns”. Thus, it’s much harder to use previous “knowledge” when estimating cost/time.
An equivalent analogy would just as well be: “How much is it to raise a child? And when is it done?”
That’s why I think so many developers feel it’s so hard to estimate.
What’s your comment on this?
James O. Coplien
Much of the agile discipline emerged from the pattern discipline and was inspired by Christopher Alexander. Alexander espouses exactly the same principles for his buildings as one finds in agile. There are iterations: the cycles of the Fundamental Process. There is incremental delivery: Local Adaptation and Piecemeal Growth. There is estimation, which he discusses at great length in his writings. And above all there’s the engagement of those who live in the house: he puts his pillar of Architect as Master Designer aside and instead takes the humble mantle of Master Builder.
Many people make a caricature of agile to attack it. Agile does not mean to throw architecture aside: Jeff Sutherland, the inventor of Scrum, says that he never has and never would go into a software development without an up-front architecture. Many people confuse agile with its methodologies, which in one sense is fair enough, but I won’t defend them, and would advise critics to be more focused with their feedback if they have a particular method in mind.
In short, this post is kind of an interesting story, but it’s off the mark and, as such, irrelevant. Anything follows from a falsehood. We’re lured into the article with the title “Should you use agile to build your next home?” and later find incredible statements like: “No estimating, no planning, and not committing won’t be the answer for most organizations. It’s not a starting place anyone can live with.” Rhetorically we are to assume that agile implies no estimating, no planning, and no committing. In Scrum we estimate passionately. We plan passionately, both at the vision level and the tactical level. The team commits to the Sprint Goal every Sprint. I would like the author either to lay down the gauntlet and say that Scrum isn’t agile (which would only go to prove that the author doesn’t understand) or that he himself doesn’t understand agile.
Alexander has been pretty successful and building great works around the world. He has received numerous awards for his work on this process. I don’t know how many Mike has gotten, but I think that someone who understands both agile and its premises so poorly is unlikely to do so in a lifetime. Agile seems to be working pretty well, too. It’s just that it’s a lot different than some people think it is. It’s sad that someone wouldn’t take the time to at least do ten minutes of research before publishing something like this. And we know from his tone that it’s not incidental: oh, no, this column and its impact were carefully planned. Think of that as you weigh the facts and read the article again.
Hey Jim,
Thanks for your comment. I’ve heard you and Jeff speak on architecture and am aware of your point of view on this. Let me tell you a little about why I write and who I write for. I write to explore problems that I commonly see in the implementation of agile. I write to explore things that I think are wrong in it’s common explanation. I’m not attacking your work or Jeff’s work or anyone else’s work. I’m not setting up and knocking down a strawman. I’m not attacking the canon of Scrum. I’m simply speaking to the people that misunderstand. Unfortunately Scrum as envisioned is not the Scrum that is practiced. It’s not the Scrum that is often taught in CSM classes. It’s not the Scrum that is often applied. It’s not the definition of Scrum that has made itself into common practice. That is what I am speaking against and trying to explain.
Scrum as practiced often entails very little backlog, very little architecture, very little up front planning, very little awareness of how to make trade-offs to optimize the chances that defined outcomes happen. Unfortunately, the Scrum that you and Jeff teach is being bastardized my many of the day to day practitioners. So… In short, I am not writing to you and Jeff or seeking your approval for my interpretation of things. I am writing to the people that misunderstand Scrum and agile and may benefit from alternative perspective or a different way of looking at things than they may have encountered in their CSM classes. I am using simple metaphors I think will help people understand.
If you fundamentally disagree with my take on things, I’m pretty cool with that and open to discussion or debate… that said, your reply seemed to focus on a perceived attack on Scrum and not the merits of my argument. Given my intended audience… any thoughts on what I should add or take away? Knowing how you think about this, I doubt that you and I really disagree all that much.
David Asbury
This is a strangely aggressive comment compared with what was a balanced and well written piece.
Interesting to read the follow up comment and that there was no reply to it.
Andrew Remely
I’ve been a product owner a lot for larger government web site builds and I love agile. I’m also a building designer and want to use Agile with my clients to help them manage their extension and renos. So this is a great article.
I dislike the disconnect between the builder’s quote and the final price after unexpected variations!
Your points about builder’s estimating and budgeting is a good one. I think some of the agile sizing principles could be useful; ie small, medium, etc. Doing this kind of estimation with builders could help flush out wild *rse guesses and other unknowns. If they can’t give a reasonable sizing estimate, then this with would be an indication that more investigation is required and that larger vague tasks need to broken-down into small units (just like software team teams estimating software sprints). Never have an XXL sized tasks in a sprint – they always go off the rails!
Marc
Agile was created for software builds because seemingly, all software projects that were easily defined, no surprises, and customers views were right from the jump were over.
Hence, Agile is for where there is uncertainty so you can demo every 2 weeks (roughly) and get feedback and increment that cycle every 2 weeks. You would abhor technical debt in house building.
That said, when you build a house, you start with an architect (assuming you have land rights) who knows local ordinances, building codes, etc and you sit and get a feel for what you want: # beds, #baths, square feet, and ideally you would want to list your design preferences up front if you love modern, contemporary, Victorian, Craftsman, etc. The earlier you know the kitchen appliances, the sooner the plans would correctly show if dedicated outlets are needed. Or how many USB outlets you want and where. The architect should help with some images drawn out so you can get pros/cons of bedrooms in back, front, etc and placement of office (if wanting one) or do you need a pantry, etc.
Then, the build process doesn’t necessarily have to be Agile, because all details are known. You should, however, meet each AM for what would be a daily stand-up then do a house build review every 2 weeks. This would help in terms of when to order your appliances, flooring, paint, etc and when you can expect inspections and other time-based events.
Again, the majority of your work is defining your vision for the house and making sure you’re happy with what the Architect produces.
By the way, I did an Agile build of my house and picked everything out in advance: flooring, insulation, wall switches/outlets, appliances, water heaters, windows, and HVACs. Basically left nothing to the contractor to order or make margin on.
Prasad
Nice article, I like it! Some of the information captured my attention “The plan and elevation were beautiful. We really liked our builder and trusted he could deliver our dream house within the time and cost constraints we agreed on.”
joe
The information in the post you posted here is useful because it contains some of the best information available. Thanks for sharing it. Keep up the good work
Thank you! Very glad to hear you are finding our content useful.