I have maintained for awhile now that in general I think that Agile methodologies may improve the speed in which code is developed, they will have a long-term negative effect on overall software quality. I’m starting to soften my stance a bit on this, but I still have some core concerns about it. The Mary Poppins reference is of course how she got the kids to take their horrible tasting medicine; while the whole Agile thing doesn’t sit well with me, lets call it not as pungent as it was even a couple months ago. What follows is my ‘State of Mind on Agile concepts’ post (something more than one person has asked for).

Requirements
If you haven’t figured it out yet, I firmly believe that strong requirements from product management are required in order for your project to be successful (on time, all features, adequate quality). In waterfall, spiral and their kin, requirements are specified directly some form of a Product Requirements Document (PRD). In Agile methods, requirements generally come in the form of Use Cases/Stories. Now, I don’t pretend to be an expert on use cases, but the ones I have seen online have tended to leave too much wiggle room for developers to interpret things on their own. Interpretation is bad in my books. How is Test supposed to verify that the use cases was successfully implemented as product management envisioned without constantly dragging them away from the other things they have on their plate. So if Use Cases are of sufficiently narrow definition, I can go along with them. (And yes, learning more about Use Case creation is on my to-do list).

Emergent Design
Okay, I know that often the Customer doesn’t always know what they want which is the classic argument against Big Design Up Front (BDUF) and strict requirements regimes in general, but the notion of Emergent Design just scares the socks off of me. I can see this working in smaller project, especially pet ones, but to be making the design up as you go along in a large, enterprise application seems massively risky — and a key role of QA/Test is to remove risk. This especially falls on its head when you have many silos within your application that must talk to each other and the design of each continues to evolve. A large portion of your time is spent making version x of silo a work once more with version y of silo b. Perhaps if a hybrid approach was conceived where the large, over-all design was worked through before coding began, and the component modules were emergent internally (their profile externally would not change) I might buy into some version of this.

Documentation
I don’t have a copy of the Agile Manifesto with me as I’m on the train with no network, but one thing that gets mentioned constantly in Agile discussions is how the Agilistas hate documentation. Documentation is not only a Good Thing, it is a Must Thing. The argument I have seen against it in the context of Test is that the Use Cases are all the test documentation that is necessary. I’m not buying this one regardless of the spin. Use Cases are the requirements to ensure the functionality of the product, but are by no means an exhaustive set of Test Cases. These need to be to be, you guessed it, documented. Another area the Agile group dispatches with documentation is in design documents; generally with the argument that the code is self-documenting and doesn’t lie. While that may be true, the design needs to be written down somewhere anyways to allow others who do not want to wade through thousands of lines of code to understand the design. Like, oh, say the Test group. How a product is designed is one of the inputs into the ‘How do I test it?’ equation.

Continuous Integration / Iterations
Love it. I wish we did more of it. We do nightly builds, but that catches whether the build broke, not whether the product when assembled together does what it is supposed to. Taking 2 days to get the parts working in a manner that Test can proceed each and every iteration is in total violation of the concept of continuous integration. If an iteration is scheduled to be delivered to Test Monday morning at 1035, it should be available then.

Unit Tests
I’m not hot on the notion of ‘create a failing test then write code to make it pass’ unit testing (test driven development). This would seem to encourage ignoring the larger picture and cause micro management of features which have to work together in the macro environment. What I am in favor of is the extensive use of unit testing. If your organization does not have a culture of unit testing, in my mind you have already lost the Quality battle. Oh, and just because you are using *Unit, you are not necessarily doing unit testing. If you are using it to create very specific tests aimed at the code with mock objects and the rest, then you are doing things correctly. If you are using *Unit as a scripting framework from which you run end-to-end functionality test, then you are doing something wrong. Well, maybe not wrong as there is certainly value in it, but don’t call it unit test as that is not what it is.

Geographic Dispersion
With more and more companies having wider dispersed workforces and flexible work schedules, the face-to-face interaction which drives a lot of the Agile methodologies is decreasing. Pair-programming over thousands of km can be done with the right technology (I believe Oracle does this) but this is a huge impediment to implementation. Likewise, Scrum (which to me seems to be the best of the Agile processes these days) requires the 15 minute standup meeting. If the team trickles in over a 4 hour window, when does that meeting get scheduled? How about when they are on opposite sides of the continent? I guarantee someone has thought of this, but I can’t think of a way that would get the same results as having everyone working from some grand master plan somewhere.

Velocity
What a great term. I’m officially working it into my testing vocabulary. The test velocity is the number of tests / number of iterations(or drops — whatever you want to call them). This velocity can tell you an estimation of how much more time you have remaining in your test efforts. And you do have your test cases all recorded so you can do this sort of math, right?

So there you have it. My pros, cons and indifferents surrounding Agile processes as they relate to Quality and Testing. Let the debate begin over how on crack I am today. 🙂