Agile design is not fragile

A non-practitioner may be of the opinion that a software system should be better off after a change. But this is very seldom the case. The effort needed to implement a new feature is always less when hacked in than the effort required to redesign the affected part of the software.
The mark of a good software design is arguably the ability of the designed system to easily adapt to change. Yet, if every change weakens the design slightly, even the perfect system will gradually become inflexible over time. I cannot help but think of it as a law: any software design will naturally decay. This decay takes time; it could take months, years or even decades but decay happens.
In a paper written by Martin Fowler, he uses the term entropy to describe that property of a system that causes the decay. A system with low entropy has a slower rate of decay, and one with a high entropy has a faster rate of decay. A system developed with an up-front design starts with low entropy, and one that is slapped together haphazardly starts with a high entropy.
If up-front design is so great, why are we not doing it? Here are some issues that make this approach difficult in our industry:

1. The first is that the design can never contain enough detail to cover all the programming issues. This causes entropy to set in at the initial construction of the system.


2. The second problem is that the designer spends so much time designing that he gets no more time for coding. Gradually, he becomes unable to keep up with the changes in tools and technologies, and he could even loose respect for those who code all day long.


3. The third problem is that requirements change quickly. Every design is based on key requirements. A challenge of design is to decide which kind of changes you'd be able to accommodate easily. Entropy rises dramatically when requirements that introduce unforeseen changes are implemented.

There are surely situations that require up-front design, and there may be some success stories out there. But mostly, the realities of our work environments make this approach impractical. We can continue with (or start with) a haphazard approach, but I do not have the space on this blog to list all the problems associated with that! An alternative solution to face the design challenge is to start building software using practices and principles that aim to manage entropy.

Here are the key practices that relate to the system design:

1. Test Driven Development (TDD) means your software is designed for verification. The practice of test driven development is usually associated with unit tests, but must also be extended to the automated testing of integrated aspects, such as user interfaces. This means you do not have to pick the brain of the designers to make sure a change does not break something.


2. Continuous Integration (CI) refers to the practice of starting the system build and test process soon after changes are made to the source code. With this in place, a problem is identified minutes after a failure was introduced, making it possible to rectify the situation on the run.


3. Refactoring is the practise of making improvements on the design while coding new features. Slogans like "refactor mercilessly" can become reality only where TDD and CI is alive and well. The refactoring guru in your company is the guy that lowers the entropy in the system. He is worth his weight in gold!

The following principles have a high affinity with the design activity:

1. Keep it simple stupid (KISS) means that you should not complicate things because of reasons that are not 100% clear. Sometimes we implement things simply because it has worked in the past, or we want to try out a new cool technology. If there is a simpler plan that will do the job, do yourself a favour and follow it.


2. You aren't going to need it (YAGNI) is really saying that you should not waste effort on something that you "know" will be used tomorrow. Sounds easy, but it is not. As software engineers we tend to think that a system is better because it solves more problems than it requires. The reality is that the new feature is not yet well understood, and you are probably going to get the implementation wrong. It is not worth risking the entropy or the deadline on something like that.

It seems to me that Agile, as the proponent of these principles and practices, is a move in the right direction. It places more (not less) emphasis on design. Agile methods imply that everybody designs, and acknowledge that design happens all the time.

In short: Agile does not stifle design, it sets it free!