The third, and (currently) final Pillar of Testing is Traceability. I lifted this term from the CMMi world, who likely lifted it from somewhere else. Of the 3 pillars, this is the most advanced one, though at first it may not appear so. It can also be the most time intensive one to retro-fit into a process, so it is best to implement this early in a product’s lifecycle.

There are two types of traceability. While both are important, reverse test traceability is warrants more attention as it has the most direct effect on overall Quality.

Reverse Test Traceability
You have achieved reverse test traceability when you can directly link a test case to a requirement (or back downstream in the overall development process). While this may sound conceptually easy to achieve, there are a few gotchas hiding in the woods to trip you up.

  • As soon as you start talking about requirements, you open a whole other can of worms. The root cause of many a project failure is requirements management. To do test tracability it is best to have as specific as possible requirements to give you smaller buckets to put tests into.
  • There are two parts to requirement management; the part that comes from product management and the developed/flushed-out part after they have been in development awhile. The second part is a red herring in the context of test traceability. Test cases need to trace back to what product management wants as they are the little-c customer. If development produces something that is not what product management wanted in the first place and you test the heck out of it, both groups have wasted their time. There is no need to have both team leads lose their head when the development lead’s is sufficient. 🙂
  • What if your product is 6 years old and has had 9 releases and has historically had poor requirements? I would suggest that for now you work on making ‘right now and forever more’ traceable. The next lull in the cycle that arrives work on the previous version etc.

There is hope though for this. It seems that a lot of testing vendors have realized this is an area ripe for exploitation and are starting to include this in their products. I know for instance Mercury had the ability to handle reverse test traceability in Test Director starting in version 7.

Forward Test Traceability
The other type of test traceability is (logically) forward test traceability. This is where test cases are mapped further along the product lifecycle, more specifically, the support phase. As mentioned previously, nothing bugs a customer more than when something that was reported as broken is repaired only to become broken again. Every issue brought to the attention of your support organization should have a matching test case (or complete set of) to ensure it never comes back to bit you again.

Note that what is ‘forward’ and what is ‘reverse’ is context dependent. This being a pillar of testing, the context is for test. If this was a pillar of support or development, the directions and targets would be different.

Benefits of traceability?

  • Structure – every tester is always working against a specific area of the product and that area is easily identified
  • Estimation – by looking at the number of test cases associated with a feature and knowing your test velocity (stealing this term from the Agile kids) you can more accurately estimate your test duration
  • Coverage – you can easily see which requirement(s) are getting disproportionate amount of testing (or lack) and can reallocate resources to improve overall product coverage
  • Reporting – some organization’s release criteria include such things as ‘number of test cases executed’ which can result in a pretty useless number when aggregated. It is much nicer to say ‘For requirement X, we did Y number of test cases’

Oh, and you would make me happy. 🙂