Automating Test Design in Agile Development Environments
Antti Huima loves algorithms and his talk at Star West 2010 reflected this. And that is actually a good thing.
- If V-Model is constructing a skyscraper, Agile is working on an oil painting (since oil never really dries and is constantly reworked). This is the best metaphor I think I’ve seen on the old vs. new way of doing things.
- He mentioned crunch time though a couple times when describing agile. If you have a crunch time, you’re not agile.
- Automated Test Design (ATD) in this context means producing test cases by running some sort of algorithm or program. Such as Pairwise, Finite State Machines or System Model Driven.
- Its automated since a machine is doing it
- BUT humans are involved in every step
- When choosing your ATD strategy, think about
- How quickly can you update your model and generate new tests?
- How small an increment can it handle
- You can use agile for implementation and v-model for test design
- Rules of thumb around requirements
- Small changes in requirements must lead to small changes in models
- Must be able to start with a small model rather than the whole thing immediately
- No significant delays caused by tooling
- Support for generating directly executable tests
- Only model what you need right now
- Helps TDD since the tests are created for you. I don’t think I believe this one. TDD’s big win is not the suite of regression tests (which are nice) but the evolving design and testability that result of building the tests by hand. Having a machine do them misses the point.
- You can’t as Deep Blue why it did a [chess] move. The same applied for your generated tests. Which is the big weakness. Go ahead and ask your algorithm ‘why’
- Models don’t have to be monolithic