So far I have shown a bit about making Basic Job Chains and Job Chain Starting Points. Now its time to talk about Parallel Job chains. Parallel chains are how you reduce the duration of the automated portion of your pipeline (recall that we’re talking Continuous Delivery, not Deployment here).

They are also how I recommend people they distribute their browser tests instead of implementing Selenium Grid — but that is the next blog post.

Recall that last time I suggested that there are multiple phases that a pipeline consists of. The first is going to be triggered off of the commit and is going to be fast. Even the Selenium part of it will be since we’re only doing the smallest sampling of Selenium scripts possible to determine whether it is even worth out time proceeding with this commit. The next phase which I’ll call the Regression and/or Browser Security Blanket phase however is much longer to do. Here we run all the scripts, against all the browser combinations that we care about. It is insane to try and run these sequentially as by the time things are ‘done’ they would be dramatically out of date.

Jenkins has always had the ability to create downstream builds, but by default they are fire-and-forget which we don’t want. What we need to be able to do is run things in parallel and when they all succeed, tag the build we’re working with as good.

Enter the Join Plugin.

When configuring a job with the Join Plugin installed you can specify not only the downstream job(s) to run, but a job to run when they are complete (and are successful — though that is also configurable). The blanket phase starts to look like this then.

  1. Traffic Cop job – scheduled
    • IE6
    • IE7
    • IE8
    • IE9
    • FF3.6
    • FF4
    • etc.
  3. Vandal job – that does the revision control tag

Looking at these individually, the Traffic Cop job does not need to actually be hooked up to the revision control system which speeds things up rather dramatically as its whole purpose is to just control which jobs need to be run and when. (Either in parallel or at the end of everything.)

  • Build other projects – A comma separated list of all the jobs to do in parallel
  • Join Trigger – The job that will be run after all the parallel jobs succeed.

One side effect of not hooking the Traffic Cop up to the version control system is it somewhat breaks the Parameterized Build Trigger. Instead, we need a bit of shell magic. I’m doing this all on a Mac with Mercurial, so my Build – Execute shell step is

<pre lang="bash">TAGGED_REVISION=`hg tags | grep POST_COMMIT | awk '{sub(/^.*:/, ""); print}'`
hg up -r $TAGGED_REVISION
./run.sh tags=puddle

which says to get the list of tags from hg, find the one labeled POST_COMMIT then get the specific changeset associated with it. And then run the scripts. (In this case limiting it to just the ones that are tagged with ‘puddle’).

The job that ends this chain, as the final one of each chain is responsible for tagging this revision with being ‘good’. Unlike the Traffic Cop job, this one does need to be wired up to the revision control since we’re applying a tag. Since this chain isn’t actually doing anything other than tagging the revision, the Build – Execute shell is

<pre lang="bash">TAGGED_REVISION=`hg tags | grep POST_COMMIT | awk '{sub(/^.*:/, ""); print}'`
hg tag -f -r $TAGGED_REVISION CLEAN_BLANKET
hg push

And this chain is done. The next chain to be run would be a manual one that deploys (without human intervention aside from starting it!) the revision tagged with CLEAN_BLANKET to the [manual] test environment.