A hiring manager approached me the other day about a position they had at their company. It sounded interesting so I threw my hat in the ring to see where things went (not far). As part of that hat tossing it was requested that I send a brief write-up on your experience on the process side, root cause analysis and defect prevention. I like how it turned out so here is the (somewhat) redacted version that more-or-less sums up my (current) feelings on those things. It is horn-blowy at points, but it was in the ‘you really want to pay me lots of money’ context.

Quality process and defect prevention are essentially one in the same. First, quality has to be established in the culture of the organization including not just the testers, but developers, project managers and product managers. This means both identifying and engaging key influencers to use modern development and testing practices.

Second, defect prevention, while a nice term is somewhat of a myth. There will always be defects, but changes to how the development process can make discovering the problems earlier:

  • pragmatic use of unit tests
  • a buddy system for checking in code (a less stringent form for XP)
  • constant refactoring
  • continuous integration
  • static analysis (source code, byte code, coverage)
  • creation of development standards

Once the code gets out development, the testers can use a number of different techniques towards testing. I am experienced in, and advocate the use of exploratory testing which is simultaneous test design and execution. This has the advantage over traditional scripted tests in that it does not constantly recycle the same test data over and over (in most applications, a single set of test data will find a bug only ever once), lets the tester utilize his/her experience in determining problems, and counteracts a large part of a testers intentional blindness. (An example floating around right now of intentional blindness is Do The Test).

Root cause analysis not really the domain of a tester in my opinion in the strict sense. Since the cause of bugs is ultimately in the code, it is up to those with the most knowledge of the code, the development team, to identify the root cause of problems. However, it is possible to determine some aspects of the root cause externally by:

  • Analyzing the code changes made for a series of fixes for things like:
    • common modules – time for a refactor?
    • common developer origin – time for some additional training? (using this in a punitive manner is suicide from a corporate morale perspective)
    • common defect types
  • Creating custom static analysis rules (most, if not all tools can have additional rules created) to find common organizational issues

Once you get a quality infrastructure in place, the net gets tightened earlier around problems and root cause analysis becomes less of a priority and/or need. When the space shuttle was originally being created, every bug went through root cause analysis and that pattern was then searched for through the whole code base. With more agile techniques and today’s tools those glory days are (largely, thankfully) behind us.