When you notice that the quality of your product isn’t a high as you’d like, what can you do?
For example, if your indicator is the number of “escaped defects” (i.e. bugs found in the customer-accessible environment) you probably want a very low number; zero is ideal but probably isn’t realistic – no matter how tight the controls are, some defects will escape. In fact, in a safe (non-life-threatening) environment, a few escapes may actually be desirable as an indicator that the controls are not excessive (i.e. wasteful).
But if you only have one metric, then you could end up focusing on that at the expense of everything else. If you only focus on preventing bugs you could spend a lot of time “gold-plating”, building far more than the customer really wants.
There needs to be one or two other metrics for balance, e.g. delivering on time (or improving the cycle time), automating as much as possible, reducing technical debt – again, it needs to make sense in your environment.
Once you have those complementary metrics in place, the question may arise: who is responsible for achieving them? In “traditional” software development there used to be a Quality Assurance team that were meant to catch all the bugs before the product was released.
The obvious problems with this are that it puts too much onus on the QA team and it’s too late in the dev process … oh, and anyone who has worked on a waterfall project will have seen that QA always gets squeezed by deadlines, so the all-important “final line of defence” has to make compromises.
It’s unfair to make the QA team solely responsible for quality when they only see the product when it’s finished – at that point, it’s very expensive to rectify a defect. Sometimes the escalating cost of fixes can lead to their prioritisation, delaying “cosmetic” defects, resulting in a bug backlog.
A better approach is to identify and tackle quality problems sooner in the development process. For an agile team, that means continuous testing, frequent feedback from the intended users, as well as code review, pair programming and other feedback loops. Quality is something the team “builds in” throughout their process; they seek to constantly improve the product they create as well as how they create it.
As far as bugs are concerned, I like the approach recommended by Yassal & Daniel Sundman: Fix It Now Or Delete It! If the bug has to be fixed now, then fix it; if not, then delete it.
I tend to believe there’s room for a little leeway: if it needs to be fixed ASAP but without disturbing the current sprint, then bring it to the next sprint planning… but if it doesn’t make it into that sprint then it should be deleted.
But quality is more than just removing (or better yet, avoiding) defects. If the team get feedback earlier then any changes needed in the design can be incorporated sooner changing direction to meet the users’ needs. The team should also be constantly looking for ways to improve how they work, i.e. retrospectives and/or kaizen. If the team can identify bottlenecks or waste in their process, then they can increase their throughput and maybe make their work more enjoyable.
Quality can mean many things; once you’ve defined it for your circumstances and identified how you’ll measure it, then ensure that everyone understands not only what quality is but also how everyone is involved in contributing it. (Ideally, the whole team have been part of those previous steps!) That could include writing automated tests for more than just the happy path, involving real users in providing feedback early and often, or participating in retrospectives to improve the way in which the team work.
But you’re not done yet – quality is a moving target. Once hitting the current target consistently has become a habit, then it’s time to expand your definition. If you start simply with just counting escaped bugs, then consider inspecting how long it takes to go from idea to released feature (“concept-to-cash”) – how can you go faster without the bug count creeping up? Hopefully, the answer will involve removing impediments, bureaucracy and other wastes but that’s a topic for another day.