A question on Twitter about interviewing a "QA" candidate spawned off a few Tweets from me, and I realized it had been quite a few years since I've written about hiring. Things have changed enough in the testing world that I think it's worth getting some of my new-ish thoughts down in a post.
First off, I have two older posts on the workflow I used when growing my testing team at Telligent from zero to nine.
- Managing Your Hiring Effort discusses a workflow I used--we didn't really have a suitable flow at the company, so I had to manage my own.
- Initial Screening Questionnaire talks to a screening step I used to weed out low-value, low-potential candidates.
I can't emphasize this enough: Approach hiring and interviews through a tough, disciplined Lean Value mindset. You don't have a lot of time, and you need to figure out a funnel process that only moves high-value candidates with great potential through to the next step. Do *NOT* spend your time phone screening 100% of the candidates. Do *NOT* even think about bringing in 100% of the candidates for a face to face. Have a gated flow that will help you find a great new team member.
PREPARING FOR THE INTERVIEW
OK, so on to having a conversation with the candidate. First, a few things for you to think of, hopefully before you advertise the job opening:
- What problem(s) are you hoping the tester will help solve?
- Are you expecting the tester to be Quality Assurance? If so, stop right there and step back. Testers don't assure anything. They inform the team and stakeholder of the state of the software. That's it. There's one single role that assures quality: The Stakeholder.
- Do you have other testers in your organization? How will this new position interface with them?
What does a day-in-the-life of a tester in this position look like? Who are they talking with? What are a few of the conversations one might have? What particular tasks might they be doing?
- What technologies does the tester need to be familiar with? Proficient with? Expert with?
Also, ensure each candidate knows how your process flows. Make sure they understand there will be a screening exercise, and that on-site interviews will include some actual pairing to do real-world testing. Never, EVER surprise your candidates--you may lose a good candidate if you spring surprises on them or don't keep them well-informed.
I highly recommend creating some form of problem or exercise to help screen your candidates. The exercise is meant to be done offline and should give some insight into how the candidate approaches problems. You're shooting to give the candidate a small enough exercise that can be done in a day or two, yet provide some meaningful insight to them. Ergo, you might use a code or testing kata.
HAVING THE CONVERSATION
I can't emphasize this enough: an "Interview" to me is a *conversation.* It's not you asking questions of the tester, it's having a chat to find out how they view their work, and the interactions they feel are critical.
Here are a few things I like to find out:
- Of course, I want to hear about their experience. I key in for a few things I have learned to find concerning
- QA is the final authority on whether to ship or not
- Fixated on bad metrics like the number of test cases, defect rates, etc.
- Preference for isolated QA versus integrated testing
- Things I find encouraging:
- Prefers immediate fixing of bugs versus filing bug reports
- Likes to pair with devs and other team roles
- Has worked on building communication with biz side
- Understands the importance of testing being an information discovery and sharing effort
During the conversation, I like to explore the issues above to learn their mindset. I do like to spend time understanding how they build out their test cases/scripts/exploratory charters. I want to gain an understanding if the candidate is starting from a risk assessment/value proposition, or just diving off into the weeds without thinking. (Erm, that sort of goes when I interview devs, too...)
Here's some other things I like to cover in the conversation:
- How well do they read and understand code? Can they read automated tests (unit, integration, UI, other) well enough to use them as a starting point for diving into exploratory testing?
- What sorts of conversations do they strive for with the business to understand the business's fears and concerns? Do they know how to dig into business-level risk and value?
- What sort of things do they look to cover in Three Amigos conversations? Do they even know about Three Amigos, even under a different guise?
- Readability, correctness, performance. How would they rank those? (There's no correct answer. It's a great conversation starter, but if you interview with me and you rank anything other than "readability" as #1 then you're wrong... :) )
- When are we done testing?
- What level of technical details are they comfortable with? Can they walk a data flow through a system, understanding what's done at the UI layer, web services, biz logic, data, etc.?
- How are they learning?
- How do they view their role as a tester in helping the team and organization to succeed?
DO SOME TESTING
Just like dev interviews should do some actual coding (not only asinine, stupid stump-the-dummy whiteboard exercises), tester interviews should do some testing. Make sure the candidate knows ahead of time that you will be doing some real work.
Find what the candidate is comfortable with doing--say exploratory API testing with Postman, or writing WebDriver tests, smashing web forms with BugMagnet, or figuring out combinatorial values for a complex matrix. Pick something *they* will succeed at and work through it.
Along that route, you should push their comfort zone to find out how they deal with situations outside their normal experience. Allow for discomfort, but look at whether they're getting defensive or viewing it as a learning experience.
LOSE THE OLD SCHOOL MINDSET
"Quality Assurance" is a term that needs to die a long-overdue death. Modern testing doesn't take on the mantle of "assuring" anything. Testers simply can't do that. Testers help inform the team and organization of whether a system has enough value to make its users happy and better at their jobs.
Your interviews of testers should align to that.
Yes, I've left out roughly 9,462 things I cover in my own interviews, but at some point, I need to hit the "Publish" button and get some value shipped to this blog!
What sorts of things do you value in your own testing interviews? What have you found useful? I'd love to hear about them in the comments!