One/Two way mirrored facilities are not worth the hassle/expense. Apart from making the participants hyper aware that they are being observed, the people observing need to be quiet and can’t engage with the research team. How many of us have seen their participants turn to the mirror and wave at the observers?

Eye tracking (non-headmounted) is the most natural way to test users. Modern systems are far from being ‘awkward’ as you put it. Participants are not forced to think out aloud (which is highly un-natural), they just get on with tasks on their own and then the moderator plays back the eye tracking at the end of the test to discuss the session. Research has shown that users find this style of testing least stressful.

Rob’s right about the importance of good facilitation. I find that a well prepared meeter-and-greeter can be a big help too; checking that the participant’s OK, answering questions, getting them a drink, hanging their coat up etc. Little things, but it all helps to create a more relaxed feel and makes establishing that crucial rapport much easier.
+1 for the formal labs thing too (though you know that I prefer to do usability tests out in the field).

Perhaps it’s something to do with the name or labs being seen as more ‘scientific’, but they often tend to be very sterile places. One of the best examples of a lab done right that I’ve seen is at Nokia in Huntingdon. It’s set up to mimic a sitting room (think of an Ikea display) for testing their consumer products. The lab in your story sounds great too, though I imagine the Director had a bit of a hard time explaining re-imbursement for personal belongings on the company expenses. He didn’t go on to become an MP did he?

I do a fair bit of remote usability testing, I’ve also done a lot of lab-based stuff, and research in situ. Frankly I think the ‘user’s real environment’ thing is a little over-don (usually by people who don’t have labs) whilst the usefulness of labs has also been over-egged (probably by the people who DO have them!)

Your 9 points sound like the argument for decent facilitators above all else ;-) Feeling like you’re speaking to a real person who is sympathetic and above all transparent about what they’re up to is paramount, surely?

What you mention above is one of the reasons we developed our remote usability testing tool, Webnographer. The participant is in their natural environment, and they don’t have to travel to get to any lab.

By the way where did you get the picture from? It is brilliant.

I can really see that one backfiring!