A couple weeks ago a former student of mine pinged me to let me know she had recently got a new job as a tester and her first task was testing the product’s installer across a number of platforms. This couples with the Dr. Project code sprint this weekend (where I will be looking at installation stuff it seems) indicate that it is time I get my thoughts on installation testing recorded (as they occur to me, for the duration of the ride into work)

  • There are basically two types of installer; native and wizard. Native installers are things like RPM for linux and packages for Solaris. Wizards are native to windows, but I have worked with them on unixes as well (at one point the linux java distribution was a shell script with the entire binary uuencoded in the body). I prefer that products use the native installer type since
    • it means all the system admin tools can access and control the installed resources
    • the necessary libraries are already installed. Nothing is sillier than having a machine that will be running headless and only running server processes but needs to have the entire X Windows libraries (and its myriad of bugs / exploits) installed just to load your product
  • Refresh your OS often. This typically means you want to invest in something like ghost if you are on windows and not using virtualization or if you are, then just copy over a clean virtual image.
  • Wizard-style installers for unix are generally java based. This means it is often okay to extensively test on one platform and then quickly skim through the other platforms making sure to check any platform specific things (placement of statup scripts for instance)
  • Perhaps more important than the installation process is the uninstallation process. Essentially, everything that was put on the disk by the installer should be removed. This does not include things your application created through. Things like config files made after installation or log files should be left for manual removal.
  • If there are things left for manual removal, don’t give the user an error message that a directory could not be removed. Instead provide a nice message saying that you removed as much as possible, but there are some files that need to be manually removed.
  • Follow the system conventions for installation. On unix platforms this typically means that things are installed under a single directory and anything else in the filesystem are sym links. It drives me bonkers when non-system products install things all over the place. On windows this means putting your application under Program Files/Your Company Name/Your Product which means you better be able to handle spaces in path names
  • If you include a necessary library, don’t pollute my environment with it. I can consistently get XP Pro to bsod on me by running a particular java application. I’m pretty sure the problem is actually that the Oracle 9i installer puts java 1.1.8 (yes, 1.1.8) into the path making it the default system java. If you are providing something like a specific version of the jre, spend the time to teach the rest of your products where to find it (like, say, the through the registry; that is what it is there for).
  • Make it easy to install dependencies. The ethereal installer prompts you at the end if you want to install the 3rd party library it depends upon, and if you say yes it launches its installer. This is nice. What is not nice is often the unix installation paradigm of digging through a README and then having to hunt the internet for a source you (almost) trust to provide some bit of functionality. If you ship your software on CD this likely means having a ‘3rd party’ directory (or something similar) which contains everything that is needed to get your app running. If it is distributed via the web then links right off of the download page are always nice.
  • pretty much all the installation methods can be automated. With wizard installers this is often achievable through the ‘console’ or ‘silent’ mode. These modes need to be specifically enabled but are a great time saver. Bribe your build person to turn these on for you.
  • Learn about how your platform loads its libraries. DLL Hell could have been largely been avoided by including the version of the DLL your product needs in the directory with the rest of your application as the current directory is first in the list of places checked. Breaking other things on a machine to get your product working is generally considered bad form.
  • Use an absolutely fresh installation of your OS every couple builds. We were once burned by a dependency on a MS Office sneaking into our product at the last moment (which resulted in a bsod of all things)
  • Test with the current security patches installed on the system for most of your testing, but at some point you have to draw a line in the sand and say there will be no more system configuration changes. This is actually quite the management challenge. At HP we were dependent on the ISAPI functionality of IIS having certain abilities but they changed some of the things we were using in a service pack after we had drawn the line. We redrew it.
  • Use an appropriate permission model. If I want to install something in my home directory that does not use a privileged port and does not pollute the filesystem then I should not have to be root. This means I don’t like seeing permissions checked based solely upon the username (Administrator on windows, root on unix) or by group name. If you need to do something, check if the user has that specific permission. Who knows, perhaps that user has a funky set of permissions which lets it write to /etc without being root.
  • Use your IT person (or friends in similar roles) as oracles. If explaining what your installer does raises their hackles, there is likely a bug
  • Don’t assume users will read the instructions, but make sure they are accurate just in case you have the oddball who reads them trying to follow them.
  • Write your installation documentation at the correct level for your audience. If they are DBAs, you don’t need to explain what a tablespace is for instance.
  • Be consistent with paths if part of an application family. I consider it a bug that I have in Program Files both an HP and an Hewlett-Packard directory. Similarly there is an IBM and ‘IBM fingerprint software’ one.