All About Artifacts (Part Three) – Test Cases: Styles, Authors and Inspiration
Test Case Styles
Much like with Test Plans, there is an infinite number of ways you can style or structure a Test Case. I’m going to do a whole bunch of aggregation and put things into four buckets.
- Classical – This style of test case has a number of steps to it, and in each one there is an observation or claim that needs to be proven either at each step. Done poorly, these are so explicit and constrained that testing becomes a brain-off activity. However, there can be wiggle room in them as well which allows for situational modification. Some systems, like Quality Center or TestLink are built around the creation and modification of Classical test cases; whether you use it in a brain-involved manner or not is entirely up to you.
- SBTM – Session Based Test Management (SBTM) is a technique for managing testing sessions and described in this paper by the Bach brothers. As they describe, sessions are tracked in text files stored in version control. Recently, Johnathan Kohl has been working on Session Tester which as they describe it is an exploratory testing tool for managing and recording Session-Based Testing. It has a timer so you limit your test sessions to a particular length, and it provides an easy way to record session notes. This tooling layer is something I think could give SBTM an additional boost.
- Checklists – At CAST2008, Cem Kaner, a lawyer in addition to tester, gave a keynote on types of checklists and how testers can leverage them to improve their testing. He has put the slides online. Here is a quote for slide 2 which outlines his position well I think: I relied heavily on detailed checklists and task outlines and templates for forms, but we were trained to use them as aids to critical thinking in the moment, rather than as directives to be followed.
- Mind maps – Mind maps are originally a brainstorming technique and in simplistic terms are visual checklists. Mind maps are the secret weapon of advanced testers. If you get to a conference with a lot of the leading lights of the craft you will see them taking notes not on paper, or in a word processor, but in a mind mapping tool. The big drawback of mind maps is that they are either binary files (which prevents diff’ing) or they are crazy xml documents, and event if you do diff them, the result is nonsensical.
Of course, there is no right or wrong style to use. Well, perhaps with brain-off classical tests, but in general a testing effort will have a mix of styles. The proportion of each type will vary with the style of development being used as well as the skill level of the testers. Me personally, I tend to be a mix of checklists and mind maps (often turning my checklists into mind map nodes).
Who Writes Them?
A common question I get when talking about Test Cases is ‘who writes them?’. Not to be weaselly, but it depends on what type of test case you are talking about.
In rigid, hierarchical organizations, the entry level testers rarely get to write test cases. Their job is primarily test execution. But at the same time, there is no reason why they cannot create a checklist of interesting ideas they have thought of while doing the execution.
I’ve had success developing mind maps individually and then as a team flushing them out. But that was in an environment that was accepting of change and novel ways of approaching tasks.
It could in fact be argued that if you turn your brain on, even to low, while doing testing then you are writing test cases. You might just be missing the step of the process where you record the test case.
Test Case Inspiration
The final thing to discuss on test cases is where to get ideas from. As you get further into your career you will build your own repertoire of ideas, but until then, here are some. (But certainly not all.)
- Explicit Requirements – Things that are in the requirements
- Implicit Requirements – Not in the requirements directly, but implied. Corporate colours for instance
- Industry Standards – If there is a Working Group or other interoperability committee there is likely a document which spells out part of the behavior of your application. SAML and ID-FF are two very boring standards that I spent far too long in.
- Corporate Standards – Coding style and the ilk
- Usability – Can it be used by the target audience for the intended task?
- Accessibility – Can persons without sight or with motor control issues use your application?
- Risk – Either of product failure or component change
- Data Storage – If the application is backed by a database or by a file will largely determine which direction your testing takes
- Logs – Figure out what triggered any stack track in the logs. Ideally you want the production logs as those will be actual usage data and fixes are guaranteed to affect users since a user is the one who created the entry through their actions.
- Source Code – You know how to read code right? TODO or FIXME are fantastic sources of things to test through the front end
- Past bugs – Bugs cluster, and ones that got logged from production point to holes in your testing. Is there any other bugs hiding around the ones you have already found? Have you patch the hole in your observations?
- Heuristics – Rules of thumbs. Here are two lists of heuristics to consider: James Bach, Elisabeth Hendrickson