Whenever I am ‘code reviewing’ a company’s set of Se scripts, one of the heuristics [metrics] I apply is the number lines of lines of script code to the number of lines of logging code. The ‘right’ answer to me has been that any logging aside from what the framework you are using provides is too much.

This minimalism is based on the idea that scripts should be run from an CI server of some sort even if the team isn’t actually doing CI. The server will keep track of execution history, produce pretty graphs and let you drill down into results. And that is pretty much all people need when looking at the scripts.

I’m starting to revise my stance on reporting for two reasons.

  1. I can now predict fairly accurately from talking to the people who wrote the scripts how much extraneous logging is in scripts. If script creation is outsourced (either near-shore or off-shore) it will be huge. If it is an in-house effort then it is minimal.
  2. Tracking Selenium Commands Using C# and SQL Server 2008

Why do I hate reporting code? Simple. It clutters the script. In extreme cases I have seen ratios of 1:4 which means that in a 100 line test method there is 75 lines of logging code! When I press the issue and try to find out why they have this it is inevitably so someone, yes, one, thinks they need it and so everyone suffers.

The blog article mentioned embeds logging to a database right into their framework by wrapping the commands. Which has the effect of having your cake and being able to eat it too; you get the logging but you don’t have it cluttering the script. (Or at least with very minimal changes.) You can also change what / how things get tracked without changing the script.

There are other tricks / techniques to accomplish this without creating a completely new class to inherit scripts from like using JavaDocs (or similar) or amending the output format for later consumption but I think I like the class modification idea best though as that opens doors for further customization.

All this brings me back to the revised stance on reporting:

  • The primary reporting mechanism for reporting should be the CI server that runs the scripts consuming the framework’s output.
  • If you insist on needing additional reporting in your scripts, hide it in the framework code and don’t put it in the scripts themselves.