A quality of experience evaluation of text and 3D instruction formats in augmented reality applications

Abstract

Augmented reality (AR) is an emerging technology that has significant potential as a solution for novel procedure assistance and repeatable procedure training. Instructions are a method to communicate how to perform a procedure for different reasons and pedagogical goals. This can range from assistance with once-off product assembly to long term learning. The main barrier to mass adoption of optical see-through AR headsets for these roles arises when AR instruction fails to fulfil the user’s pragmatic and hedonic needs and expectations due to human, system and context influencing factors. User quality of experience (QoE) considers this fulfilment to be reflected in the user’s degree of delight or annoyance. The ability to directly measure emotional response using modern psychophysiological instruments is shifting the focus of quality assessment towards evaluation of fulfilment of user needs and expectations. In this context, the work presented in this thesis focuses on understanding the influence of instruction formats considering AR as a potential platform for procedure assistance and training. Instruction format was evaluated over two distinct studies specific to the procedure assistance and training roles. In Study 1, the influence of paper-based and AR-based text instruction formats on user QoE for procedure assistance was evaluated using a Rubik’s Cube® proof of concept. In Study 2, a combined text and interactive animated 3D model instruction format was compared against a text-only instruction format within AR using a GoCube™ proof of concept for training. Two separate AR applications were developed. Physiological ratings, facial expressions and eye gaze metrics were recorded. Subjective experience was reported using Likert scale, self-assessment manikin and NASA task load questionnaires. Statistical analysis was employed to identify statistically significant differences between usage of the different instruction formats. Correlation and regression analysis were undertaken to identify novel implicit metrics of QoE. The results from Study 1 show that the AR instruction format yielded objective performance benefits over the paper-based instruction format for procedure assistance while participants reported higher acceptability of AR. Heart rate features indicated increasing stress in both test groups, which corelated to mental load in both groups. Study 2 results show that the text-only instruction format yielded faster instruction response times in procedure training compared to a combined text and model instruction format. Female trainees using the combined instruction format were significantly slower in training and recall than females that used the text-only instruction format but reported requiring less cognitive effort than male participants during training and recall. An absence of statistically significant correlations between physiological ratings, facial expression and emotion terms used by the participants, calls into question the utility of such emotion terms as measures of emotional state. Facial expressions of action unit 20 correlated to task duration in both studies.