I’m going to start this review by pointing out some irony. As a tester, it is my job to critique applications that were often created by people I would consider friends. And I’m perfectly happy to do that. But when I am critiquing a creative output of someone I know, be it a book, article, song, performance, etc., I am much less comfortable doing that. With Beautiful Testing coming out in just over a month though, I have started to ask people to read and critique it. So fair-is-fair, and I’m going to write up a review of How We Test Software At Microsoft by Alan Page, BJ Rollison and Ken Johnston.

HWTSAM is about, well, how testing happens at Microsoft. Which shouldn’t be all that surprising actually. And who better to describe it to the outside world than the current Director of Test Excellence and two past ones as well.

Like most books of this size (just over 400 pages) it is divided into sections; each which had, to me, differing amounts of value.

This first three chapters are groups together a ‘About Microsoft’ and describe just that. Now, not being a Microsoft employee, nor can I see myself being one (more for geographic reasons than political), I really don’t care. But I understand why they are there and agree that they are necessary as they set the environment in which the rest of the book operates. I can see this being required reading for all new Microsoft testers.

The next four chapters, the ‘About Testing’ ones, are what truly make the book and have the most markups in my copy.

In Functional Testing Techniques (chapter 5), BJ Rollison nicely describes Equivalence Class Partitioning, Boundary Value Analysis and Combinatorial Analysis (sometimes call All Pairs). Sure, there are decent Wikipedia entries on each of these, but sometimes that isn’t enough. The ECP and BVA are fundamental techniques to the craft of testing and all testers need to know them. Combinatorials is certainly useful, but I consider that one of those double-edged swords which when wielded incorrectly can cause more damage than good.

BJ’s contributions continue into the next chapter where he discusses Structural Testing Techniques. This of this as the White Box chapter; using the structure of the application (the code) to derive tests from. It is certainly not an exhaustive exploration of the topic, but a nice toe-dipping-in-the-water that will let testers be dangerous from the start.

Alan is back with the next two chapters. The first on Complexity, which is a metric I’m not too hot on so I just skimmed it, but the second was on Model-Based Testing and was pretty good. Unfortunately (or fortunately depending on perspective), I’m not in at C# shop so can’t use the tool he describes. But again, in the context of a Microsoft book it makes sense that it would be Microsoft platforms discussed.

The next section, Test Tools and Systems, didn’t really bring me too many great ah-ha moments, but did confirm a number of things that I have preached or independently ‘discovered’ or ‘invented’ which is nice. But then again, I’ve been doing this a while now. Of the chapters in here, ‘Customer Feedback Systems’ (chapter 13) was cool from a ‘so that is what / how that works’ perspective and ‘Testing Software Plus Services’ (chapter 14) had lots of little tidbits on taking your application to ‘internet scale’. The best, and perhaps most ‘duh’ inducing one was the importance of the deployment process in the success of projects and should be treated as a first-class artifact of development.

The final section, About the Future, includes two chapters. The first deals with with Automatic Failure Analysis which is certainly something Microsoft cares about given the number of (automated) tests they run daily as well as Virtualization which is another thing today’s testers needs to understand. The last chapter, Building the Future, is another Microsoft specific one but also includes the best page in the whole book. On page 392 is ‘Who Owns Quality?’ and it speaks about the need for a Culture of Quality. (I’ve started to pester Alan is making this page available online; once he does, I’ll put the link here.)

I don’t claim to know everything about testing, but I’ve certainly got a pretty good base by this point. When I’m reading testing books, I am looking not for (completely) new techniques, but clarifications of, new insights on and new ways of explaining techniques. HWTSAM ably met all that criteria. Is it the most important book for testers that came out in 2009? Probably not, but testers (inside and outside Microsoft), especially ones that are new to the field, should still find a lot of value in it.