A friend of mine pinged me during his commute this morning about my thoughts on weening a team off of thinking they need ‘a priest (QA) to bless (test) all our work’. ‘Free’ consulting means it gets to be content. :D

Obviously, this is a ‘people problem’. So the approach will vary place-to-place and even within the place. Regardless though, need to start by expunging ‘QA as Quality Assurance’ from the organization. They don’t actually ‘Assure’ anything. You, or a half dozen other people could override. So ‘Quality Assistance’ is a nicer reframing. Or better still, ‘Testing’.

Then you need to play detective and find out what the inciting event was that caused a) the first ‘QA’ person to be hired, and b) how they got anoited as priests. Smooth transisition away from that requires you know those two things.

Organizationally, I would be interested in;

  • how many things are found by the testers
  • what the categorization is (because those are developer blind spots)
  • how many things that are found actually hold up the build until fixed
  • and of those, how many could have shipped

From a purely technical perspective, some practices that address this;

  • dark launches via feature flags and have new stuff rolled out slowly to user slices
  • acknowledge that production is different than any other environment and is the only environment that matters. To quote Charity; ‘I test in production, and so do you.’
  • the only metric that matters in today’s world is ‘mean time to production’. Something isn’t ‘done’ unless it is in production being used by the target customer. Everything you want to do hinges on that. Put on your wall a whiteboard with ‘number of deploys today’, ‘number of deploys this week’, ‘number of deploys this month’ which you increment each time it goes to production
  • if you think your feature stories are small enough, you need to slice it more
  • not to overload the term, but increase the observability of the application in the more traditional way not honeycomb way. If you are pushing to production fast and often, you need to know its behaving or not fast and often. Number of logins per 5 minutes, number of registrations per 5 minutes, number of searches per 5 minutes, etc. Every new feature / fix needs to have a measure to know if it is working. (It will take a long time to get to here.)
  • move to trunk based development. Everyone should be pushing code at least once every 2 days. Feature branches allow people to get sloppy.
  • Obviously, TDD is huge in this. (or TAD — I don’t care, just slow down and write some damn tests before committing)
  • Steal from Etsy’s playbook and have your pipeline such that day 1 at <redacted> is pushing to production, and day 2 is paperwork / onboarding. Forces you to get your development environment in shape so you can onboard someone from bare machine to productive in an hour and also it breaks the feeling of sanctity around production and creates shared ownership. I believe everyone at etsy did this, not just developers. (Though obviously non-developers had a borrowed environment and were hand-held)

MTTP reduction is the whole thing purpose of building out a Continuous Delivery pipeline. ‘QA Priest’ doesn’t fit time wise for that. (Its also why Scrum is a Continuous Delivery anti-pattern.)

But again, this is a Culture thing. To quote Jerry; ‘things are the way they are because that is the way they got there’ — figure that out and you can change the culture.