Think Aloud Methods with Eye Tracking in Usability Testing: A comparison study with different task types

Abstract

The concurrent think aloud (CTA) method is perhaps the single most valuable usability engineering method. The method has certain issues that may be avoided by using another method: Retrospective think aloud (RTA). RTA can be conducted un-cued, or cued by different stimuli. Several studies have been done to compare the different methods, but none of these have examined the methods under different circumstances. In this study three different methods, CTA, RTA and RTAg (RTA cued by video and gaze data) are compared when applied on three different task types: Easy interaction task, reading task and complex problem solving task. The comparison was done in terms of four measurements: Task performance, usability problems identified, amount of verbalization and the participants’ subjective assessments. It was found that with the two retrospective methods there were elicited more usability problems by means of verbalization as compared to CTA. It was also found that during reading tasks and complex problem solving tasks the participants found it easier to verbalize using RTAg than using RTA. Some observations were done that invite to further research.