With our service you can setup tests in minutes without the hassle of recruitment which can take a lot of time (and energy).

How do you recruit participants? How long does it take?

As such, you can only hope to uncover baseline usability issues – “Can a human carry out task X without confusion”. Don’t get me wrong, these insights are very useful – it’s vastly better to carry out this kind of research than none at all!

However, once you’ve got these insights under your belt, your research questions will mature. Your questions go from “Are key tasks completable by humans in general?” to “Does our webapp cater to the needs, goals and expectations of our target user group / user community?”. Remember, usable is not the same as useful. It’s entirely possible to create a webapp that is very easy to use, yet fulfils no real needs in target user group. You’re never going to find out if your app is useful by testing it on random people outside your target user group.

As Lee from WhatUserDo pointed out in the first comment, they are now offering the ability to create your own panel, and to screen testers from their predefined panel. This sounds like a good step forward to me.

@Nathan – yes, it’s good to use different types of research during a product life-cycle, for sure! However, consider this. With a tool like whatusersdo or usertesting.com (remote, qualitative, unmoderated) users carry out a task for 20 minutes remotely. No measurements are taken, a screen recording is simply captured. Afterwards you have to watch that video. You are going to spend 20 minutes one way or another. If you were to use a tool like Webex, you could take the exact same amount of time but run the session as an interview (remote qualitative, moderated). This means you can steer users away from rambling on about irrelevant stuff, and you can question them when something interesting happens. Obviously this is not for everyone, but the benefits are clear to see.

Although maybe one would ideally like to perform a moderated session at all stages of a project it may be acceptable to use a remote test if one is just trying to gain some quick insights or test out relatively minor changes to already tested interactions.

A company that I work with has repeatedly executed research plans that blend both approaches together throughout a project life cycle:

* Early: usertesting.com
* Early, again: usertesting.com
* Next: Moderated usability test
* Another: usertesting.com
* Multiple rounds (various international versions with language-specific participants): Moderated usability tests
* Final, for confirmation of a few details: usertesting.com
* Woops, another final confirmation: usertesting.com

A small thing to add – we’ve recently started sending custom screeners for those clients who want better profile matching of panelists.

Email support@whatusersdo.com to find out more.