According to wikipedia, a test case is a set of conditions or variables under which a tester will determine whether an application or software system is working correctly or not. I like to just say it is a question to be asked; much simpler.

And while we’re dealing with definitions, a test set and a test suite are pretty much synonyms in my book. If I needed to make some distinction it would like be that a set is subset of all tests available, and a suite is a collection of similarly targeted tests. For example, a sanity (or smoke) set vs. the login suite.

Why write Test Cases
Writing test cases can be pretty boring at times, but there is certainly some value in writing them down ahead of time. Here are the main reasons why I do it.

  • The big reason for doing test cases in advance is to give you more time to think about the feature. The creative juices take a bit of time to get flowing and having the application in front of you with the clock running towards the release is not conducive to that.
  • Human beings don’t function at peak performance when under stress and being up against a release date it can certainly be stressful. By writing down our test cases in advance, we lessen the chance we will miss something because we are under stress. We might miss them for other reasons, but not that one.
  • Without having a set of tests which everyone references, there is no definition of Tested for the team. There is a semblance of Tested, but that is not what the CEO wants to hear when they ask you ‘Was the tested?’. (Trust me on this.)
  • This is almost Part 2 of the previous one, as having test cases that people execute means you can record who did what. In some environments, this level of test execution traceability is important. But of course, in others it is used as a big stick to beat people with.

And of course, if you think of an interesting testing idea while actually doing your testing you should write it down as well. Just because you didn’t think of it ahead of time does not make it not worth recording.

Test Case Rules
See Test Case “Rules” post.

Test Case Spectrum

In this presentation, Jon Bach gave a scale which measures the ‘freedom’ a tester has when executing test cases. Starting with the the least amount of freedom to greatest, they are (with my definitions):

  • Pure Scripted – Either scripted in terms of automation, or in terms of actors reading their lines from a script
  • Vague Scripts – Automation which has some intelligence or scripts that have some room for the actors to deviate or improvise
  • Fragmentary (Scenarios) – A scenario is a story that describes a hypothetical situation. In testing, you check how the program copes with this hypothetical situation. The ideal scenario test is credible, motivating, easy to evaluate, and complex. – Cem Kaner
  • Charters – The tester is given a mission and is then able to use their skill and judgement about how to accomplish it.
  • Roles – Similar to Charters, but with less constraints
  • Freestyle (Ad-hoc) – No constraints; complete freedom