In this article I interview Maya Middlemiss, MD of Saros Research, who talks about the ins and outs of recruitment for user research, and among other things, how to avoid getting people like the dude in the photo from turning up to your research sessions.

INTRODUCTION

One of the most common questions I get asked by research newbies is “Where do you get your users from?”. My response is simple – I outsource my participant recruitment.

If you choose to do-it-yourself, recruiting participants for your user research can turn into a nightmare. You’ve got to track down the right people, screen them, interview them, give them appointments, make sure they turn up in the right place on time, cope with cancellations and misrecruits, arrange payment, and that’s before the research has even started. Aside from it being a massive time drain, if anything goes wrong (like having the dude in the photo turning up to one of your sessions), you’re the one accountable. This is why I work with recruitment partners who I can rely on to get the job done properly.

INTERVIEW WITH MAYA MIDDLEMISS

HB: Maya, I need to run a research project in exactly two weeks, and I need 10 niche users turn up at my office on a specific timetable. Can you help me?

MM: The short answer is yes! That’s one of the primary services that we provide at Saros. All you have to do is email me the specification of what kind of participants you need and what your schedule is. We then devise a screening questionnaire with you. Following that, Saros will pre-screen a demographically eligible sample from our database (circa 160k UK consumers) and invite a selection them to complete your screener online. Those who pass will be telephone interviewed, and finally, we’ll arrange appointments to fit into your timetable. This means you can relax and rest assured that on the day, the right kind of people will turn up at the right time. We deal with all the admin and logistics so you don’t have to.

HB: How much of your business involves sourcing participants for user experience & usability research? What other areas do you service?

MM: Our clients these days are probably about 50/50 usability & user experience, vs . ‘traditional’ market research – (e.g. advertising focus groups). We also have a number of design and innovation clients who sit somewhere in between the two and use a variety of methodologies.

HB: What’s a typical price when recruiting users for a run-of-the-mill usability study?

MM: Well it’s hard to name a ‘typical’ project, but entry level for usability is probably around £75 per head – it all depends on how hard the people are to find / how niche it’s going to get. “Must be a Vodafone user” = easy; “Must have a Blackberry Bold on Vodafone business plan taken out in past 3 months and not reject the i-phone, born in a month with an ‘r’ in it…” let’s just say, samples can suffer from overdesign sometimes!

Of course respondents also need compensating, and a 90 minute one-on-one usability session is actually very demanding to take part in – makes a group discussion about shampoo look like a walk in the park! For anything over an hour we advise an incentive of £50, and for B2B it can obviously get a lot higher.

HB: Some companies aren’t set up to pay respondents in cash, and prefer to pay by giving freebies like mouse mats and t-shirts. What’s your stance on this?

MM: Anything branded by the end client is not acceptable to be given to respondents as incentives, under Market Research Society regulations re the problem of selling under the guise of research – this regulation was tightened up in 2008 and a lot of people still aren’t aware of it. It really has got to be cash – you do get what you pay for, and we know you have a great deal more invested in the session than that £50 in any case, far more to lose if someone drops out last minute or fails to show up.

If you don’t want to handle/prepare the cash yourselves we can can arrange the cash and have it couriered it to your venue. Some researchers like to pay their participants electronically after the event, but good old fashioned cash is the best possible motivator and guarantor of good turnout! The only exception is under 16s, if they are taking part their adult chaperone should receive a small cash payment for travel expenses but we feel it’s better to give kids non-branded vouchers rather than a ‘brown envelope’.

HB: The most annoying thing when you are doing depth research is to realise half-way through a session that you’ve got a dud user on your hands who is giving you bogus data. It’s unavoidable that duds will slip through the net occasionally – But what does Saros do to minimise this kind of problem?

MM: These people are small in number but disproportionately problematic in our industry. Our database currently numbers over 160,000 members, and I would guess a couple hundred are maybe in this category –we advertise for members and they’d be crazy not to register with us. Of course any research activity they take part in with us we control and track rigidly, but we have to rely on their declarations regarding ‘other’ research… Many of them we have tagged – we always ask for feedback from our clients, and log every comment received. We also run regular de-duplication routines to catch the multiple applicants – at one point we had 14 women in Leeds who apparently shared a set of initials and a mobile phone number, she had set up different email accounts, profiles etc. – unbelievable! It’s actually very hard to track down the *very* small minority who lie constantly and consistently – we continue to root them out, but we believe our best possible defence against their damage to our business is to continually recruit fresh new genuine people to replace them, and there are thousands of them for every liar/dud.

We try very hard to design screening processes that are robust and watertight, but occasionally someone successfully lies their way into research for which they are blatantly ineligible. We word our confirmation and joining letter to respondents very carefully to indicate that our client has NO obligation to pay them anything, if it emerges that their responses to screening questions was ‘inconsistent’, though we completely appreciate that it’s very challenging on a number of levels to say to someone ‘you’re clearly a lying little ****** aren’t you, get outta here!’. Obviously you are going to try and limp onwards and salvage some kind of insight from the session however bad they are… Whenever anything like this happens, we will work with you to try and put it right – and if we have misunderstood or misapplied a screening criteria, or sent you an outright con artist, that will be at our expense!

A particular problem for one-on-one research is when your participants don’t turn up for the session. At least with group research, the session can still go on – with usability testing, you have to cancel the session. There are various safeguards you can use, depending on your schedule and your needs on the day., For example, we can recruit an additional session at the end of the day, and if you don’t need them, we can cancel and pay them off by phone. Alternatively, you can pay a participant to be a “floater” and sit in the lobby all day, to take the place of a no-show if needed. as If you have mission critical project, you can even double book all your sessions – but this is pretty expensive. If you tell us your situation, we can work out the best way of addressing your needs.

HB: What can I do to avoid getting users who are monosyllabic and basically unhelpful?

I would hate to reinforce any unfortunate stereotypes about early adopters and techies being shy inarticulate and geeky, so we won’t even go there! Let’s just say that EVERY research participant, whatever the project, needs to check out on a number of criteria beyond meeting the spec, and our interviewers are looking for a number of softer qualities that we hope add up to a ‘good’ respondent… Do they clearly have good spoken English? A strong but understandable accent is fine, but can they comprehend your conversation clearly and respond articulately and helpfully? Are their answers consistent and unhesitant? Are they genuinely enthusiastic about taking part and wanting to pass the selection process?

We also like to ask one or two completely unrelated ‘imagination’ questions at the end of the phone interview, to test that they do have some conversation and ability to free associate etc. If you can share details of the methodologies you will be using, we can tailor this appropriately. For example, in a project looking at packaging prototypes and designs, we might ask people about things you could do with a brick rather than build with it, or if it’s all about using a visual interface we can ask about associations with colours etc… or sheer fantasy stuff about who from fiction/history they’d like to meet or superpowers they’d choose!

Maya Middlemiss is Managing Director at Saros Research Ltd. Feel free to get in touch with Maya regarding your participant recruitment needs in the UK.