Twitter is excellent for planting troll-bait one-offs, but the blog is better for planting long-tail ones. So this afternoon I tweeted something along the lines of this.

Unless your are interviewing someone for a job that requires [stupid] logic problems, then they don’t have any place in an interview.

And I stand by that. Though a lot of twitter seems to disagree, to one degree or another.

Context. I was at a client today and the two people I am working with came back from an interview…

Them: Do you like logic problems?
Me: Not really.
Them: Want to try the one we just gave in the interview?
Me: Sure…
Them: How long should we give him? 8 seconds? 5?
Me: Them: Design an algorithm that solves Me: (immediately) Dunno. Them: Give up? Me: Nope. I didn’t come up with something immediately and don’t care enough to focus any thought on it Them: …

This of course led to a much larger conversation about the value using these sorts of things in an interview has. My mentioned above, I don’t think they have any. (Or at least very tiny minuscule amounts to avoid the use of an absolute…).

More context. The interview is for a position that will primarily be webdriver automation using python with some manual testing thrown in there as well (and will be taking over the stuff I put in place.)

So. How does the ability to answer a logic problem help determine the candidate’s ability to do the job? Very. Little. Here is a secret; web automation is not a ‘logic’ problem. It’s not even that hard. It is way easier to be an automator than a tester.

What should you spend the time in an interview for a web automator instead of wasting everyone’s time with the logic problems? Ummmm, how about you have them automate your app. Right there. For real. Projected on the screen.
</the></something>

  • What oracles do they establish?
  • Which patterns do they use?
  • How ‘robust’ are their locators?
  • How is their synchronization?
  • etc.

These are all critical things to understand about an automator. Especially the one that will take the lead on the project. Notice a distinct lack of explanation why manholes are round or how to determine the odd billiard ball out in a minimum number of steps.

(Oh, and looking up syntax is absolutely ok. It shows they know where to look. We have this internet thing — we don’t need to memorize everything anymore. Now, reliance on an IDE to program for you is not ok.)

I see logic problems in an interview almost as a crutch that the interviewer is using to hide that they don’t really know what they are looking for. And as an interviewee, if a logic puzzle rears its head in an interview the bozo bit gets partially flipped on the whole situation. Do I want to work at an organization that values people who can solve logic problems? Or that are awesome automators and/or testers. (This isn’t to say there isn’t a possible intersection between the two groups, but it is not guaranteed.)

What are a few arguments I have received today in favour of logic puzzles; none of which sway me…

  • It shows a high IQ and high IQ is needed for a testing role – People play lawyers on the internet, so I’m going to play neuroscientist. So first, all being good on logic problems means is … you are good at logic problems. A lot of people spend their free time doing these things so they train their brain to spot patterns and build their own solving heuristics for them. Same for crossword-ers or word search-ers. Heck. Look at the whole ‘tester game’ thing. (Of course, the tester game phenomenon has a large degree of arrogance associated with. And no, I won’t play.) Back on topic. No, you also do not need a high IQ to be an excellent tester. Again, I could be conflating IQ with other things (only playing neuroscientist remember…) but there are more firms hiring Autistic people as testers (article).
  • It lets us see how they work through a problem(1) – Sure, but is it a class of problem they will experience at the job you are hiring for? Those are the problems you are hiring for.
  • It lets us see how they work through a problem(2) – As usual, Michael Bolton provided a Feynman reference (even if it is fictional); Round Manhole Covers, or: If Richard Feynman applied for a job at Microsoft. Its a fun read and on target in capturing what I have seen of Feynman. But he gets a job offer for marketing. Not in test. Not in web automation. His answer would tell me nothing about his ability to do those jobs, though would tell me not to engage him in an argument about manhole covers.

A slight variation on this is of course, FizzBuzz. FizzBuzz is loved online and is somehow seen as the bar that should be met to be called a programmer. I have not once had to use anything FizzBuzz like in automating apps since I started in early 2000. (And that is of course the only experience I can have.) If you are hiring someone whose responsibility includes programming, absolutely you should have them write code. But have it be relevant to the job! As mentioned above, for a web automation job, have them write web automation code! And if you must use FizzBuzz, then you should write a web app that does it and have them automate it.

Way at the beginning I mentioned that the job that originated this discussion is mainly automation and partly testing. The automation side I think I’ve beat to death. Interviewing for testing is, perhaps unsurprisingly, very simple. Have them test! Logic problems will not help you determine if they have a set of heuristics and philosophy for mobile, or payment gateways, or video, or, or, or. Unless your product is vapourware (or super crazy top secret — at which point, have them test something else), have them test your real product! Again, live! And not just silently do it pointing out [possible] bugs but out loud explaining their thoughts. (I wouldn’t have them write bug reports as an interview is a fake scenario and you’re not going to get a ‘real’ bug report.) Flashing back to Feynman, this is the useful part of the article. The thinking and rationality behind the thoughts. As an interviewer, that’s what I care about. Does it clash with mine, does it create overlaps, does it fill capability holes, etc.

But you can’t get that from the creative process solving a logic problem! As a community we claim to care about context, but interviewing outside of context seems like a waste of everyone’s time.