If you are a causal observer of the automation space or are ‘just’ a ‘user’ of automation this post might seem a bit pedantic, but as someone who lives full time in automation it in the ‘pet peeve’ category.

There are a lot of moving pieces involved in [successful] automation and it is important to know what they are and what their role is. Opinion can then get thrown into the mix when defining their boundaries.

  • Driver – There needs to be some sort of library to actually interface and ‘drive’ the thing you are automating. That’s the entire purpose of a Driver. The four ‘core’ Selenium language bindings are examples of a Driver.
  • Runner – A Runner collects, executes and reports on scripts. They often also provide an assert framework. JUnit, RSpec and Py.Test are examples of pure Runners. Runners typically enable an xUnit style of script creation.
  • Framework – Frameworks wrap a Runner with helpers and opinions that allow it to be more than the Runner’s author’s had either originally envisioned or consider beyond scope of the project. These are often site-specific, but occasionally are made somewhat generic as is the case with Saunter.
  • Oracle – First rule of [browser] automation is never, ever, ever, ever, ever trust [the browser]. You really do need to go to the source system to ensure that the result you are seeing is accurate. It is not enough to pass a script because something appeared in the browser — check the various points in the database that would have been affected by whatever you did in the browser.

The reason this is at the fore of my mind right now is that I am working on the PHP version of Saunter which meant I had to re-evaluate the stack that I am using. And to my (re)dismay the boundaries between these things blur in that language more than in any other it seems. As a framework developer, I like the boundaries to be crisp allowing me to puzzle-piece things together.

Take for instance the WebDriver-PHP project. They are at first glance a Driver, but they also integrate with Sauce Labs which is a Framework role. PHPUnit also confuses things a bit. Its a pretty darn good Runner, but includes a Selenium RC Driver as well. To muddle things further, it wraps the Runner in opinion making it impossible (or at least really, really hard) to use it outside of the PHPUnit way.

Open Source automation has really started to mature in the last year or so where it is absolutely a viable option for teams. But if the various components involved start to blur the boundaries between roles too much, I fear that there will be an ever increasing pile of re-implementation. So my plea to the community is to operate with those boundaries in mind.