Realizing that mastering Agile software development is more than just learning techniques, I am rebranding and changing the name of my company to (drum roll, please)…

To Be Agile

This new name (and website) reflects a new commitment to helping our community, not just do the Agile practices, but to be Agile with everything we do!

I hope you like the new website: www.ToBeAgile.com. I’ve moved all of my blog posts and other resources to it and will continue to update it with new resources and information. I will keep the old Techniques of Design website up for a while but I won’t be updating it. Please update your bookmark and contact information for me:

Website: www.ToBeAgile.com

Email: info@ToBeAgile.com

Thank you!

David Bernstein, http://ToBeAgile.com

I remember a poster at my college that said “All great minds don’t think alike but they do make connections.” I think this is especially true in software development where we are all largely self-taught.

When I ask my students what the most valuable thing they got from my training was, I get a lot of different answers such as the design techniques, the different yet highly valuable way of understanding patterns, the focus on quality, but one of the most valuable things developers tell me is the ability to form a common language for design so they can communication with their team mates in higher fidelity.

We desperately need ways to communicate better in software development. Language is vague and the software we write needs to be precise. It is often a challenge to express our design ideas to others and so they are not given the best opportunity.

I think we sometimes get caught in the fear that makes us worry about several things at once. When we have a conversation about multiple things at once it is easy to get lost. I see this sometimes, developers will jump from topic to topic, not really addressing any one of them so no real progress is made. By contrast, when we stay focused and work through each issue, one at a time, we can make a great deal more progress.

When we have disagreements on design approaches it is rarely about the current requirement and almost always about what could happen. But that is anyone’s guess. I try not to engage in what could happen because it is often just speculation. Instead, I draw on good development practices to help mitigate the risk of changing my design later and just focus on implementing the task at hand.

Of course, if I know a feature will be needed soon and it is easy to accommodate it now, I will go ahead and do it, or at least write my code in such a way so it is easy to add later. Remember, Kent Beck says, “Do the simplest thing possible, not the stupidest.”

When I do get into an argument with another developer I apply what I call the ten minute rule. If I can’t convince them of my approach in ten minutes I usually give in to their way. This is because I know it is usually easy to change a design from their way to my way if it becomes apparent that it is better to do so.

Changing a design to accommodate a new feature doesn’t always have to be a huge undertaking, especially if we have unit tests that cover our code, and we build software using Agile practices. I find building software in this way is far more successful and less stressful than trying to anticipate what the customer might want in the future. When we get good at refactoring and see that going from one design to another in many situations can be easy to do, a lot of the fear of having to get it right the first time goes away and development becomes much more fun!

Interestingly enough, I do not find many sources discussing problem-solving skills for software development. Most books are concerned with the mechanics of a language or framework and almost every software design book I read teaches a procedure rather than to pay attention to the clues in the problem itself. Certainly, universities do not teach these skills that we are just starting to recognize as highly valuable in our industry. As a result, there is a huge difference in the effectiveness of developers.

Software developers are among the smartest people in the world, no doubt. This is because we are constantly solving new problems. It takes an incredible attention to detail in order to build even the simplest of programs. But intelligence is not enough; we also need skill and strategy.

I’ve made it my professional mission in life to find effective techniques for building software and I’ve amassed a set of techniques that I’ve witnessed expert developers use to their great advantage. I have not only learned these skills but also how to teach them so they become top of mind and used. This is important because you probably already know some of the skills I teach in my classes but if you don’t consciously connect why those skills are important, they will probably not become habits for you and you won’t get the benefit of using them naturally.

Where to start? Start by believing that there are better ways to solve problems. If you don’t then you’ll have no motivation to try. Provide proof to your brain that others have these skills and that the skills they possess are learnable and worth learning. Then ask yourself different questions–questions you haven’t asked before. Focus initially on the reasons and benefits of your goals rather than the mechanistic of how to achieve them. This can help you see different perspectives that can open you up to finding better solutions.

How we think about a problem has a direct impact on the solutions that are available to us. What is or isn’t possible is oftentimes a fluid thing based on what we know and also what we believe. In software, virtually anything is possible but the level of difficulty often depends on our starting assumptions.

I’ve had the privilege of working with thousands of developers over the last quarter of a century in my career. The difference between a good developer and a super-star developer can be a factor of 100 to 1 (yes, really, I know developers who are one hundred times more productive than an average developer). I don’t think that is true for other professions. For example, I can’t imagine a carpenter being 100 times more productive than his fellow carpenter or a lawyer able to handle 100 times more cases than her colleagues but this can be true for software developers.

If you are a developer then you probably have the experience of thinking about a very difficult problem and then suddenly find a solution that makes the problem almost trivial. If you had this experience then you know there is the potential for huge productivity gains; we just need to have those epiphanies more often.

Flashes of insight may appear to be random but there’re not. There are things we can do to induce them. In fact, I would say that problem solving skills are really the thing that distinguishes great developers from average ones. Once we are able to understand a problem and see its solution the rest is merely typing.

Would you pay $10 to spin a roulette wheel with a potential payoff of a million dollars? Maybe so but you probably wouldn’t do it if you had to win 100 spins in a row, right?

Risk is part of life; we have to take them but we want our risks to be calculated. Yet when it comes to software development we often set ourselves up to play “all or nothing”.

On most projects where integration happens at the end of the project we accumulate risk and it continues to accumulate throughout the project. Each line of un-integrated code holds potential risk. We set ourselves up so that every feature has to work on order for any feature to work. One tiny bug can keep the entire system from working. Not even the most hard-core gambler would take a game with those kinds of odds yet we do it every time we embark on a waterfall project. No wonder that most software projects are not successful!

There is a better option.

Integrate as you go. As soon as you have a new bit of functionality that compiles, integrate it, first on your local machine where your core unit tests run, and then on a build server where all the automated tests run. Do this several times a day and you will always have a buildable system. This drops your risk of being able to ship something to close to zero. Maybe you won’t get every feature done by the ship date but you’ll have something and if you plan well then what you’ll have are the most important features.

Maybe this seems obvious to you but in my experience the biggest companies building the most complex systems do not do this. Waterfall projects are still the norm and “integration hell” as we’ve called it is the inevitable conclusion of a waterfall project.

If you still write code the old fashioned way where you have to completely implement a feature before you can even compile then you may as well come to work blindfolded with one hand tied behind your back. The power of your computer is idle, unable to check you work, help you with syntax or verify the code you just wrote will “play nice” with the rest of the code in the system.

We are software developers. Our job is to automate business processes and the first place we should look is to our own work! Don’t set yourself up to fail. Take little wins as frequently as possible by integrating as you go. It will give you confidence and momentum and you will always have something to show for your efforts.

A story, use case, or requirement is not done until it is integrated into the rest of the build system. “Well, it works on my machine,” is not a statement we want to hear on an Agile project. A story that is not integrated into the build is not complete and worth zero story points because it may hide significant risk.

When code is not integrated into the build it can hide the worst kind of bugs that only show up when it interacts with the rest of the system. On waterfall projects we do not get to integrate our code until the end of a project, right before we plan to ship—the worst possible time. These projects turn into an “all or nothing” situation and one tiny bug can keep us for shipping anything.

On Agile projects we eliminate risk every time we integrate our code into the build so we minimize the amount of risk we have as we go as we progress through a project. This is why we want a fast and uncomplicated build process, so developers will integrate often. How often? I usually say we should tell developers to integrate their code at least once a day but this is a bit of a trick.

Usually, the last person to integrate their code at the end of the day finds bugs in their code and because they can’t leave while the build is broken they have to stay late to fix it. After doing this a few times developers quickly realize that the sooner they integrate the fewer problems they encounter and so they end up integrating several times a day.

This means we find bugs almost immediately after we write them while the code is still fresh in our minds. Because the time between cause and effect is short we begin to change how we write our code and start adopting better practices. This is one of the main benefits of continuous integration, it shortening the time between writing a bug and finding it.

I worked on many projects in my 30 year career as a software developer. I’ve worked on embedded systems, operating systems, collaboration software, downloadable applications, and enterprise applications—the whole gambit. I’ve done coding, testing, managed developers, been a ScrumMaster, Product Owner and virtually every other role on a development team. The thing I enjoy the most is software design.

One thing that attracts me to design is how important it is to have the right design and how it has a huge impact on the success of a project. Over the last 10 years I’ve really focused my attention on how to design better and learned a variety of techniques and approaches for good design. So it may be a bit of a surprise for you to learn that the more I learn about design the less I suggest people do it—at least not up front.

Whenever I see people do up front design or I do up front design with groups, I find that we spend a lot of time thinking about what could happen rather than what is. Of course, we want our systems to last and we think a good up front design will help us, but in my experience doing a lot of design at the beginning of a project is often a wasted effort. We spend so much time debating about what could happen or what the client might need that we get ourselves lost. What could happen or what the client might need is anyone’s guess. If we have no certainty around requirements then we shouldn’t design for them.

I’ve been running some interesting experiments with developers. I’ll take a group of developers through a two-week training using test driven development. At first, I’ll ask them to develop some software that meets some basic requirements using the approach that is most comfortable for them except that they must write a test before writing implementation. But I don’t time-box them and I don’t tell them anything more than the requirements. Most of the time I see groups spent the first hour or more talking about what they want to build and how to design it. It’s typical for nearly an entire afternoon to go by with little or no code written. Then the next day I ask them to work on another set of requirements but to do it with no design. Very often I get asked, “Really? No design at all?”

I respond, “Do pants have pockets or do pockets have pants?” Of course, they answer that pants have pockets. Then I say “That’s the amount of design I want you to do.” In other words, just state the basic relationships between entities but go no further. Don’t figure it all out up front, in fact I ask them to start coding as soon as you see anything that they can code, first as a test and then make the test pass. If they know some piece of functionality they need then I have them write a test for it and then implement that test and do this over and over.

Developers are often quite hesitant to try this at first but when they do they often see that their velocity starts to increase significantly. I am often asked, “But what happens if we get it wrong?” I suggest that they change it when they find a better approach. What we learn from this is that changing our designs or our code is not that big a deal and often it’s far easier and less time-consuming than I trying to guess up front what the right design might be which in the end is just as likely to be wrong as starting with essentially no design.

With no design up front we are forced and encourage the design as we go and this is really the right way to design for implementation because it keeps us aware of the forces in issues that we’re dealing with. In my Software Development Essentials classes I covered six code qualities, three principles, three practices, three pieces of advice, and 12 patterns that I believe every developer show know. This is by no means is an exhaustive list of things to be aware of but it’s a good start and often it serves as a minimal set of things to get people to be aware of how to do emergent design. Combined with knowledge of refactoring and good discipline in test first development we can often be productive on a project immediately and emerge designs as we go with the minimal amount of reworking.

I’ve tried many development methodologies over the years and I have found none that support good quality software development more than the techniques I teach. The hard part is convincing developers that it really works and there’s really only one way to do that, have them experience it. Developers who take my coding class generate more code in three afternoons of lab time than they often do in a month!

These are things we were unfortunately not taught in school but they are critical for working in software development professionally. So if you’re curious I invite you to check out what I teach. I written a lot about it on my blog and offer lots of resources on my website (http://www.techniquesofdesign.com) but the best way to understand it is to experience it and if that’s of interest to you and I welcome you to attend one of my classes.

“We don’t need to do <blank>, we’re doing Scrum.”
I’ve heard some beginning Scrum teams say this. They think that doing Scrum is their get-out-of-jail-free card, freeing them from doing architecture, design, documentation or even thinking about what they are doing. Compared to waterfall, Scrum is a lightweight process but it is a process and there are essential pieces that cannot be skipped. In fact, the purpose of Scrum is to help us stay aware of these things throughout the development process, not just in the beginning.

“We don’t need to worry about architecture, we’re doing Scrum.”
In Scrum, we try to avoid a big upfront architecture but that doesn’t mean we do no architecture. There are certain things that we must figure out upfront, like what platform our code will run on. Our approach will be different if we are building a high availability online reservation system verses a point-of-sale cash register system for the local grocery store.

Our goal in Scrum is to figure out upfront what needs to be figured out at the beginning and defer on what we can figure out later. Why? Because we’ll know more later and we want to take advantage of that knowledge so we make the most informed decisions when we know the most.

“We don’t need to do any design, we’re doing Scrum.”
Again, I hear new Scrum teams say they don’t need to do design. In reality, we do more design in Scrum than we did in waterfall. In waterfall development, we do design upfront in a distinct phase and then we implement that design, sometimes blindly. In Scrum, we design as we go throughout the development process. There is never a point in Scrum development where we go on “automatic” and just execute the specification. We use the many feedback mechanisms in Scum to refine our designs as we go to build a better system for our customers.

It is easy to overdesign. If you have ever “painted yourself into a corner” when coding then you know how painful this can be and you probably want to avoid getting into that situation again. The remedy for most of us is to do a really good job at upfront design and be as complete as we can. But the problem is that we can’t predict the future and it is often a futile effort to try.

Like architecture, design is something best done as we go, for the most part, because we will know more and be more familiar with the problem when we get into it. There are some design decisions we must make up front but many design decisions, both big and small, can be done as we go if we know how to keep ourselves from getting stuck.

In reality, even waterfall development does a lot of design throughout the development process because even the best specifications cannot account for every little detail and many details are best worked out when we code. Scrum simply acknowledges this and provides a framework to support it.

Of course, if you see a design solution early in the process and you will need them later, there is nothing in Scrum that says you have to ignore it. Scrum just says that we should not be overly focused on design for work that may or may not be needed in the future.

The idea here is not to blindly follow a plan, the idea is to stay alert and focused on the problem as we are solving and not get too concerned with what might be needed later. If we build our software with good engineering practices then new requirements will be easy to add later.

“We don’t need to write any documentation, we’re doing Scrum.”
In XP we say “Barely sufficient documentation.” Software development projects are about developing software and creating artifacts can get in the way of that. Internal documentation can be useful but what tends to be even more useful is code that can be read and understood without having to reach for a separate document. If our code has unit tests then you can see exactly how the code is intended to be called by looking at the tests and there is less need for internal documentation.

I’ve heard these kinds of excuses from more than one development team but this is not what Scrum says! Scrum, XP and other Agile methodologies are lightweight compared to waterfall but we still have to think about what we are doing, not just upfront but throughout the entire development process. We don’t want to do all our planning in the beginning, we want to do it throughout the development process and be flexible to change our plans as needed.

Scrum projects do not need all the checks and balances of waterfall if we focus on building the right things and keeping our quality high. This is a key presupposition for Scrum projects that I don’t always see beginning teams fully get. We must follow development standards, keep code clean, and pay down on technical debt as we go in order to build software that can support a lightweight process.

If we know and follow good engineering practices then we don’t need as many of the checks and balances as in traditional waterfall development and we can put that effort into building more features and building them with higher quality because this is ultimately what will benefit our users the most.

One of the things that established professions like medicine and law have that we as software developers don’t have is some form of mandatory mentoring. For doctors it is residency, for lawyers it is internships, even carpenters need to apprentice with a master carpenter before they can become a master themselves. But in software development there is no formal apprenticeship that we all go though and as a result there is a great deal of inconsistency in skills between developers.

Don’t get me wrong, I love the diversity in our industry. Different perspectives can often lead to breakthroughs but I also see how just having some basic knowledge in common would help us communicate better and think more clearly about the challenges we face.

The apprenticeship model has existed for thousands of years and it is alive and well in the software industry. A lot of us owe a great deal to those who have taught us on the job. I learn tons from my students and colleagues and feel it is my duty to give something back and share what I know–that’s why teaching is a good field for me.

I know very well that there are perhaps even greater benefits in mentoring than in being mentored. Being able to perform a skill represents a level of proficiency that is far surpassed by being able to teach it. Teaching is one of the fastest roads to mastery.

As developers we all must be great teachers. Our ideas are only as good as our ability to articulate and engage our fellow developers in them. We must not only do the right thing but be able to explain why we did it. This is a skill that great developers have.

One of my students in a recent Scrum Developer Certification training who was particularly generous with his time and had helped everyone in the class improve their skills, confided in me that he was once very closed to sharing with other developers and saw knowing something that others didn’t as a competitive advantage.

“What changed for you,” I asked.

He said he had a few hours waiting for a flight with Ward Cunningham. In those few hours he said Ward shared a lot of valuable things with him. He said, if someone as super-smart as Ward could be so open, he wanted to be the same way.

This man was changed forever and he is now a highly valued thought-leader in his organization. Of course, he always had the potential but it was one chance encounter with someone like Ward Cunningham that showed him another way he could be and that changed his life forever.

For me, being able to spend a few days with developers is the greatest honor. My deepest wish is that I can help people the way Ward helped this person, not just with more and better development techniques, but by being someone that also embodies the qualities of agility and someday, maybe, someone will say about me what this student said about Ward.

Compromises are part of life. We must make tradeoffs if we are going to ship product but we want to make the right tradeoffs and that can only happen when we have information to back our decisions. Understanding good design principles and practices help us make informed technical decisions. The principles give us guidance in the large to help us keep sight of what we are striving for. The practices give us guidance in the small to help us form good work habits. Together the principles and practices we follow help us make the right tradeoffs so we can build the best system within the constraints we must work in.

Scrum and XP is not prescriptive, like waterfall. Their purpose is not to propose a process for developing software. Instead, they have distilled down a handful of principles and practices that allow us rapidly build high quality software that is more resilient to change.

The development practices that most developers have been taught are obsolete. They lead us to build code that cannot be changed as easily as our clients often need. The challenge is that Agile development is very different than traditional development and without changing the way we write code to accommodate developing in short sprints we can end up with poor code that is hard to change.

Traditional development shops tend to have several impediments to fully realizing the promise of agility. Changing the business to be more Agile and helping the stakeholders see how they can use the process to get what they need is something that doesn’t happen overnight.

We not only need to understand what the principles of Agile are but why they are there in the first place if we are to make the best use of them. You don’t have to do all the practices if you understand and can mitigate what the practices offer.

Far too often someone will read a book and try to transition their organization to Agile, unaware of the common pitfalls that transitioning to Agile can bring. They lose a lot of money and time trying to figure out best practices and how to approach their problem effectively.

More and more companies are saying what they do is Agile but many are not really doing any more than mini-waterfall, which is not Agile. Short iterations and stand up meetings alone are helpful but that not doing Agile, not by a long shot. To really take advantage of an agile development methodology the whole team must be well versed in Scrum and XP development practices. They must understand the “why” as well as the “how”. This helps assure building changeable code whose quality is so high that we do not need all the checks and balances of a traditional waterfall process.

Good software development is about making the right tradeoffs. To do this we must know tools and techniques in Agile for mitigating risk. When we do we can make the best choices for almost any situation.