Testing – common in advice; rare in practice

It’s common to be advised to test everything with users, sometimes with a hint of derision for those who might not.

But it’s not so easy in practice, I’ve found over a long career. At one point our design agency bought eye tracking kit, and employed a researcher. But we found it almost impossible to sell this service to clients. This was in the early 2000s, and we were possibly too early to market. These days there are many more specialist agencies you can work with, and the digital industry has normalised testing.

So here are some caveats about testing:

• User-testing is no substitute for good design. It will hopefully reveal flaws in your work, but it will not give you the solution. And if you test poor design, it does not suddenly become good design because it has been tested.
• Don’t confuse testing with science. Unless you have done a proper, controlled study addressing a well-thought out research question, it’s not safe to draw universal conclusions from product testing.
• It can be very hard to find users to test, who are representative of your actual users. For example, I have worked on information intended for senior procurement managers, high net worth investors, recently bereaved people, and defendants in court cases.
• It is impossible to test every variation of personalised communications. Many customer service documents (such as letters, bills and statements) are assembled dynamically using customer data.
• To test medical or health information with actual patients, you have to go through arduous and sometimes overwhelming ethical approval.

• It can be hard to persuade some clients to pay for it, when they can already see how much better your new design is than the old one.
• Some clients see testing as a judgement on your work, and want an independent agency to do it so you won’t be biased. Persuade them that testing isn’t about proving you are right, but getting close to users. When your mechanic fixes your car and takes it for a road test, they don’t get an independent agency to do that. They need to hear the sound of the engine themselves.

Having said all that, you absolutely must involve users at some point in your work. Good design rarely happens without considerable experience of meeting users. You need a huge degree of empathy with a wide range of real people, and you won’t meet them in your imagination sitting in a cool modernist studio in the city (or your spare bedroom, for that matter, if freelancing).