Nigel Bakker's SOA Soapbox

By Nigel Bakker and Chris Nel

5 October 1999

Software quality is one of those issues that gets most attention when it’s noticeably lacking. What is meant by the term “software quality”, and how can we ensure that it is present?

A well-known definition of software quality is “software that is delivered on time, within budget and to specification”. Well, in today’s fast moving software development environment, where the specification is out of date before the team has been put together, this will no longer fly as a working definition of software quality.

Early identification and rectification of errors, saves both time and money. In today’s business environment, systems are competitive tools and not just cost-savers. Systems that don’t perform leave you or your client either unable to play, or playing at a disadvantage. In the tight time-scales of today’s opportunities, it is then too late to seek the problem. Quality is either built-in, or it is not.

The forces of change

It’s often said that the only constant in the IT Industry is change. Predicting the direction of change is an industry in itself.

Gone is the old style of development, where the user’s requirements are documented in a detailed specification, which is then handed to the designers, who create a design which is coded by an army of programmers, and then tested by an army of testers. Today’s processes are shorter and sharper; naturally adaptive and encouraging of change. Today’s computers are business tools – no longer just cost-saving, but actively participating to create significant competitive advantage in a rapidly changing business landscape. Today’s software development processes are adapting accordingly.

How do we manage our software development in an ever-changing environment, in a way that is not only tolerant of change, but encourages it? The simple answer is an iterative, incremental development process, which delivers real business advantage in small, fast increments.

How then do we control quality in this environment? Clearly, just doing user testing at the end is not enough: By the time user’s have found and reported bugs, the team is already well into the next increment… The cost of tracking, tracing and fixing such bugs is excessive. To draw a parallel with the construction industry – it’s easier to make changes while the mortar is still wet, than to chip away after it’s already set. Software developers are no different, they must find the bugs as early in the process as possible – while the mortar is still wet, if you will…

Quality FIRST

Quality cannot be an after-the-fact issue: we can certainly test after the fact, but true quality is built-in: it’s planned for, designed, and an integral part of the entire development process; It ought to be a natural part of what developers do all the time. Two major factors indicate this:

•development cycles are too short to allow for long testing periods at the end of each increment.
•The cost of change during development is a small fraction of the cost of change once the bugs have been deployed.
A factor that mitigates against deeper quality sentiment however, is the perceived lack of tangible benefit: ‘I wrote 1000 lines of code today’, versus, ‘I wrote 300 lines of tested code today’. In many environments, more is better and 1000 wins hands-down against 300 (no matter how many times that code has to be redone.)

So, what tools are there that will help to bring a quality focus into software, earlier in the process? Sadly, there are very few. You need largely to depend on in-house built test harnesses and test code developed by the programmers themselves. You also need a leadership team that cracks the quality whip with monotonous regularity – frequently the same leadership team has to balance the cost/time/scope equation. Quality suffers.

One of the major drawbacks of writing test code is the fact that the test code itself can have bugs. This leads to debugging the tests! Who quality controls the test code?

The vast majority of software testing and quality tools focus on verifying that the software works at a functional level. Very few tools focus on testing before and during coding. Contrast this with the fact that, long before software becomes ‘functional’, it exists first as a design; then as logically grouped ‘units’ of software. These units are only later tied together so that they collaborate to produce the chunks of functionality that users see.

If software quality is to be built-in, testing must focus first on guaranteeing the quality of these ‘units’. The units themselves perform micro-tasks in fulfilment of the design; an arrangement you might view as being contractual. In verifying unit quality, testing will verify performance against a ‘contract’. Only later in the process, will it also need to verify the chunks of functionality.

What is needed is a tool that will underpin quality through the development process, that enables you to start testing the software, almost before it exists!

eXtreme Testing

A radical new approach to software development that has been gaining popularity, is eXtreme Programming (often referred to as XP). XP was first showcased on the Chrysler C3 Payroll project and was the brainchild of Kent Beck, Ron Jeffries and others. One of the interesting aspects of eXtreme Programming is the notion of eXtreme Testing.

This notion embraces the idea that before you write a line of code, you should have defined and written your unit tests. How, you might ask, can you write the tests before you write the code?

In Smalltalk this is simplicity itself, and XP was in fact first successfully applied in a Smalltalk environment. In most other languages, Java and C++ included, this is somewhat more challenging – you cannot write code that depends on other code that does not yet exist. The answer lies therefore in a means to define your tests independently of your code, whether or not that code has already been written!

Tests, then code

To encourage both managers and developers alike to include aggressive unit testing as part of the life cycle, it needs to add value beyond just finding bugs. Of course, “just finding bugs” should be enough reason in itself, but somehow the perception and reality seems poles apart. What value can defining tests early yield then?

Clearer specifications

For a start, defining the tests first, forces developers to understand exactly what they will be coding – and to think about how their code will be used by other code in the system, rather than on how they are going write the code. This emphasises the external interface of the unit of code, rather than its internal workings. An emphasis on this “contract” focuses developers on developing just what is needed to satisfy requirements, which results in less code, and ultimately fewer bugs.

Progress Indicators

Ever ask a programmer how far they are and get the answer 80%, then discover that the last 20% takes 80% of the time? Having tests in place allows developers and management alike to track progress at a very fine-grained level. There are two measures here: The first is the number of tests that pass, gives an indication of actual progress for specific units. The second is the total number of tests defined, provides a metric of how much of the system is in place. Add this to the number and percentage of tests that pass and you have some pretty powerful progress indicators.

Safe Changes

How many times have you seen a “small change” that “just takes a minute” bring a system to its knees? Having a suite of unit tests in place allows errors in these small changes to be caught immediately they are introduced. A simple rule that states that no unit is released unless all the tests pass, works wonders. Of course, you must have the tests in the first place and be able to monitor their relevance and accuracy.

Refactoring

The growing “refactoring” movement, another discipline practised by the proponents of XP where code is cleaned up before functional changes are made. For refactoring to work, you must have tests that ensure that none of your “clean-up” changes break anything in the existing. Here the tests protect your code and thus the investment.

All of this suggests that testing early will save more time and money than testing later. It also contributes to getting the required functionality delivered without the expectation of comeback once the team has moved to the next increment.

XML, Java and Testing

XML is a meta-language – that is, a language that is used to define languages. Using XML we can specify sets of rules and definitions. A set of rules and definitions in XML is known as a DTD. A DTD can be used to define the format and rules for test data. Freely available tools for editing XML content make it possible to define test data using a DTD. A test framework to load and execute tests can use this data to execute tests. XML also becomes the natural medium for writing out test logs since it is inherently structured and can easily be imported into applications ranging from relational databases to spreadsheets and word processors.

This particular combination of Java and XML, makes it quite feasible to define your tests independently of the code under test and run them whenever and as often as you like. Even non-programmers and juniors can help define some of the tests, and anybody, including the project manager can run them.