One of the things that I do for organizations is write BrowserMob scripts for doing load testing of their applications. The BrowserMob API is a JavaScript implementation of the Selenium APIs with some extra bits that are important for instrumentation of the run.

Too often though there is a tonne of back-and-forth to get all the correct expectations set and information gathered. This is expensive both in terms of money and time. In most situations, there is some new marketing initiative or event that provides a hard deadline for the performance issues to be identified and resolved so anything that impairs this is very bad.

To that end, here is a mini checklist of sorts to speed things up.

  • Flash – Personally, if I had a[n obnoxious] Flash based site, I wouldn’t try to do much in terms of load testing since it is primarily a client-side technology. From a functionality perspective, there isn’t much that can be [easily] done as the Stage is a big black box. What can be done is a tonne of requests to the server providing the SWF resources which BrowserMob can waitForNetworkTrafficToStop.
  • Steps – BrowserMob scripts are typically a single Transaction with multiple Steps in them. As the outsource company, I need to have detailed instructions about what constitutes a Transaction and its Steps. By default I create a new Step per page change, but sometimes people care about the timing of a complete flow (or flow portion) rather than individual components. If this is the case I need to know about it.
  • Changes – Yes, it is a good idea to incorporate performance testing early in the development cycle, but if you change things at the last moment, without informing the script developer, your runs won’t be very useful and cost you more while the broken scripts are debugged.
  • Authentication – A lot of times the sites being tested are not supposed to be seen by the public yet. But for both the load run and the script development it does need to be somewhat public. There are two solutions to this. Security-by-obscurity is generally not a good idea, but a gobbley-gook url does the trick for this. The other option is to put a Basic Authentication password on the site. Not NTLM or other scheme. Just plain, basic authentication. White-listing networks doesn’t scale; on one day I wrote scripts from my home network, the airport network, in the air, and a hotel.
  • Are you ready? – Way too often I can tell while creating the script that there is no way a site is going withstand a real load run. And it is quite evident that no one asked ‘are we ready to do a run? the schedule says that we are, but are we really?’. Use the site at the same time with a couple people in the office. If it isn’t blazingly fast then, do you expect it to withstand 50 concurrent users?
  • Users (1) – BrowserMob scripts can be fed unique information (often usernames) via CSV file. Having those ready right at the beginning is fantastic.
  • Users (2) – If specific users need to be used, make sure they are configured correctly in the system before providing their information.
  • Rules – Most systems have rules around usernames, how many incorrect passwords, etc.. The script writer needs to know all these for creating random string generators for fields.

I think these are the main ones. If [When] I remember or discover new ones, I’ll update the post.