In this first PEP (Practical Exercises for UX/HF Practitioners) talk, our Sage President and Sr. UX/HF Consultant, Shannon Halgren, PhD taught our Medical Technology UX/HF Meetup group how to conduct a Heuristic Evaluation of Medical Technology. See our PEP Talk Introduction post which describes the PEP Talk format and our Medical Technology UX/HF meetup. We hope you can come join us at our next meeting!
A summary of the talk is provided below, but we also encourage the reader to red the presentation slides which provides more detail.
Click here to view
Conducting a Heuristic Evaluation of Healthcare Technology
Heuristic Evaluation (HE) is a part of a group of activities that identify use-related risk factors and improve product usability. It is also a proven budget-sensitive usability inspection technique. The results of a Heuristic Evaluation serve as inputs to the product design and the use error analysis (uFMEA or similar).
HE is a great technique to use at multiple points in the product development cycle. Check out this table below to see when conducting a HE might be right for you!
When to Use HE |
WHen Not to USe HE |
|---|---|
|
|
Heuristic Evaluation Step-by-Step
There are three steps to the Heuristic Evaluation process: 1) Define, 2) Evaluate, and 3) Report.
THE FIRST STEP IS TO DEFINE.
As Shannon discussed in her talk, before starting the HE review users, tasks, heuristics and evaluators all need to be defined. The table below discusses these categories to determine for each of these items in more detail.
CATEGORY |
ITEMS TO DETERMINE |
|---|---|
| Users |
|
| Tasks |
|
| Heuristics | Choose your set of heuristics. If you are evaluating a medical device, Shannon recommends using the set developed by Zhang et. al [2] specifically for medial devices:
|
| Evaluators |
|
THE SECOND STEP IS TO EVALUATE.
In the evaluation step, evaluators will evaluate the device and task workflow from the point of view of both a user performing tasks using the device and as a UI expert. In both cases, the evaluators use the chosen set of heuristics and take notes as they work. Evaluators typically perform independent reviews and spend one or two hours on their review. Longer evaluation sessions may be necessary for larger or complicated interfaces with a substantial number of dialogue elements [1].
THE THIRD STEP IS TO REPORT.
The Heuristic Evaluation report will include an overview of the HE process and present the findings. Report formats may vary, but typically each heuristic is scored independently and specific violations are listed and rated in terms of usability and risk severity. This report (which can be in either word doc or excel format) can then be used to support quality discussions between the evaluators and the product team.
IN SUMMARY
Heuristic Evaluation is a usability engineering inspection technique for finding usability problems in a user interface design. It is an excellent method to use to order to catch the “low-hanging fruit” prior to investing in usability testing with end users. HE involves a small set of expert evaluators who examine the interface and assess its compliance with “heuristics,” or recognized usability principles. HE results are an excellent input to the product design and use error analysis (e.g., uFMEA).
References
[1] Nielsen, J. 1992. Finding usability problems through heuristic evaluations. Proceedings ACM CHI’92 Conference (Monterey, CA, May 3-7); 373- 380.
[2] ZHANG, J., et al. (2003) Using usability heuristics to evaluate patient safety of medical devices, J of Biom Info (36)1:23-30
Click here to view
CONDUCTING A HEURISTIC EVALUATION OF HEALTHCARE TECHNOLOGY
Shannon has extensive experience in the area of product user experience and human factors, with an expertise in user research and healthcare technology. Through Sage Research & Design, Shannon and her team have helped start-ups as well as established Fortune 500 companies like Dell, Apple, Facebook, Intel, Netflix and eBay create products with excellent design and usability and have helped healthcare technology clients such as Abbott, Ascom and Hach design safe and FDA-compliant products. Prior to founding and growing Sage for over 10 years, Shannon co-managed the usability group at Macromedia and founded the usability group at Apple/Claris. Over the years, Shannon has presented her work at several professional conferences (HFES Healthcare Symposium, UXPA, CHI) and in professional journals and books. She holds a M.A. and Ph.D. in Experimental Psychology from Rice University in Houston, Texas where she specialized in Human-Computer Interaction.