According to this article on Yahoo! news, the OLPC “Sugar” UI has had absolutely no user testing carried out on it yet. Shameful stuff! I’ve also heard rumours that the UI design has taken place predominantly behind closed doors at Redhat. In other words, apparently no prototypes, mock-ups or even sketches have been put in front of any of the target user groups to ascertain whether the proposed features are either useful or usable.
I guess the folks on the design team haven’t heard the axiom “test early, test often”. This is surprising since it’s on pretty much the first page of every undergrad HCI / CS textbook under the sun.
The OLPC is a phenomenal piece of hardware, and has such worthy goals. It would be such a shame to find that it falls down on usability just because of lack of planning.
I’m increasingly drawn to the conclusion that almost nothing actually shows any indication that it has had the benefit of any user testing. I see glaring issues with so many websites and so much software and product design that I can only conclude that it’s either never undergone a user test or that company politics intervened to prevent the findings actually being implemented. :(
It’s one thing when a website or software product is done badly, because the customers will eventually vote with their money and choose better alternatives. But it’s not the same when you have a captive audience.
You’d think that this product would go through some testing with at least one elementary school before being unleashed in such a large scale to children of the world’s emerging nations.
I’m hoping there’s more to this than we might have heard about.
I disagree: I don’t think it should be user tested at all until after it is released. Feedback, especially from people who may already be familiar with the current UIs and desktop metaphor, would screw it up. Sometimes it is best to go with a vision, behind closed doors, and do what you think best. It can always be updated after release if needed.
I’ve seen too many UIs be user tested only to become worse for it. Sure, maybe they did the testing improperly — doesn’t matter — it’s still worse.
I also think it’s retarded to prejudge because they aren’t doing something that most people would say is necessary to a good product. There are many products that don’t go through much if any user testing that are awesome, and there are many more that do and that suck. Making software is just too random to claim that some particular “way” of doing it is right, and all others are wrong. Sounds like religion to me.
Scott, I understand the point you are trying to make. As Henry Ford said “If I had asked people what they wanted, they would have said faster horses.†(instead of building cars). However I disagree. User testing isn’t *asking* people what they want- it is *watching* people use your product / prototype, and seeing what issues they have with it, in use. The other important thing is to test on your target audience, not just any old person.
I’d be really interested to know what UIs in your experience have got worse as a result of user testing. They would make really interesting case studies.
In my experience, user interfaces most commonly get worse after user testing when management thinks the customer is always right — literally — and meddles with the development team’s effort. This does matter because management meddling transforms the activity from user testing to user design. It’s not user testing’s fault when people fail to perform it any more than it’s a hammer’s fault for hitting your thumb instead of the nail. Making software is random only if you do your best to ignore the available tools.
Pete. This happens when user testing is done badly. It’s quite easy to do user testing well, but without giving it due thought, you can mess it up, particularly in the interpretation of the results.