I’ve come to realize that I have a tonne of content that is in powerpoint and not in the blog as entries which can be a pain if I can’t get to the slides (that are hosted on sourceforge), so I’m going to start recapping some of it. Here is the first one…

First, let me start by clarifying why rules in in quotes. These rules are very much heuristic in nature and should not be taken as gospel; it is quite easy to think of scenarios when they would be inappropriate to follow. They are merely guidelines that have I have successfully applied when thinking up test cases ahead of time. They also help when thinking about whether something will be automated or not.

  1. One, and only One thing – This rule is states that a test case should only be measuring one, and only one thing. Imagine you have a test case that does two things: A and B. If A passes, but B fails, does the test case pass? Or does it fail? It is more like ‘sorta’ which is not really a state you want to be in. Similarly, if A fails, but B passes, does the ends justify the means and it is a pass? Likely not. Likely, sorta, maybe, kinda are not things I like having to report.
  2. Self-sufficient – Test cases should not be dependent on other test cases. Much like the previous rule, if you need 5 tests to pass before yours can even run, you are setting yourself up for a tangled web of test case dependencies. This doesn’t mean that your test case can’t have a number of preconditions as part of the test setup, but those are included in the test case. And these setup steps do not have to be overly prescriptive. ‘Create a valid user’ is enough detail for a lot of tests. An extra benefit of this is that tests can be run in a random order which is a good check for whether they can be run in parallel if/when they are automated.
  3. Black or White – Black and white is a nice place to be. The success (or fail) criteria of a test should be very clear. Again, fuzzy results are not fun. (Of course, the definition of what is a pass or a fail is contextual and should be specified somewhere.)
  4. Valid test data – All data has rules around it, though they might not be obvious at first. Explicit rules are things like the column definition in the table and things like password strength rules. This only half the problem, and the less important half at that. They other half deals with the implicit rules that you might not be able to get without questioning the actual users. I learned this one first hand while in the Bahamas on my first business trip for my first job. I was testing central bank reporting software for the offshore Trust subsidiary of a major Canadian bank and using numbers like 5, 10, 25, 100 as inputs. Technically they met the explicit rules and were really easy to do math on without a calculator but when the head of IT for the company saw what I was doing he hit the roof. To open an account there, clients were plunking in most cases over $1 000 000. My $5 test was not exercising the application in any sort of valid way. I added a bunch of zeros to my input numbers and redid all the tests. Lesson learned.
  5. Repeatable – Tests should be constructed in such a way as to be repeatable, both immediately and at some point in the future. This often means that you reset the environment to its prior state whether through removing entries in the database or simply clearing your cookies.
  6. Write for the the correct audience – Do not write your tests for 3rd graders unless the people executing your tests are 3rd graders. Odds are testers doing the tests know the system, so you don’t have to explain how to use the system. ‘Setup a password server’, ‘Create a new question’ or ‘Login as a user with administrative rights’ might not mean anything to you right now, but you are not in the situation for it to. If you were, then it would.
  7. Brain on? – The best testing is done with the tester’s brain firmly locked in the ‘on’ position. This means not boxing them in with things like ‘Start the application. In the firstname field input ‘Adam’ and in the lastname field input ‘Goucher’…’. This was the old school way of designing test cases and too often employed by teams who thing poorly of their testers. The only excuse for tests of the brain ‘off’ sort is when there was a specific triggering condition, then you should have the test case specify that specific condition.