Summary: Hornbæk & Frøkjær, Making Use of Business Goals in Usability Evaluation: An Experiment with Novice Evaluators

The paper conducted an experiment aiming at examining the effect for using business goals in usability evaluation. The results indicate that, if the evaluators are asked to consider business goals in usability evaluation, less usability problems are found than people who are not required to do so. However, these usability problems are deemed as of “higher business relevance and overall utility to the company that commissioned the evaluation.”

This experiment is a between group experimental design. Two groups of evaluators are asked to conduct “think-aloud tests” to a website. Before the test, both of the groups are given a background material to read regarding the business goals of the company who owns the website. The only difference is that one group is required to consider the relevance of the usability problems they found to the business goals, but the other is not. After the test, both of the groups’ evaluation reports are given back to the company’s representatives for assessment. The assessment is based on a four-dimension criterion: clarity, interaction, business, and overall usefulness.

The implication of this experiment is that, besides usability knowledge, other kinds of knowledge, such as business goals, could play an important role in usability evaluation. They could help evaluators eliminate the problems that are not relevant to the business goals and focus on the problems which are more useful for the company.