A quality of experience evaluation of augmented reality and haptic feedback in a low-cost gait analysis system

Abstract

Gait analysis is a technique that is used to understand movement patterns and, in some cases, to inform the development of rehabilitation protocols. Traditional rehabilitation approaches have relied on expert guided feedback in clinical settings. Such efforts require the presence of an expert to guide the re-training (to evaluate performance and provide feedback), the user to attend a clinic and is based on subjectivity of the clinician. Nowadays, potential opportunities exist to employ the use of digitized “feedback” modalities to help a user to “understand” improved gait technique. This is important as clear and concise feedback can enhance the quality of rehabilitation, recovery, and prevent injury. A critical requirement emerges to consider the quality of feedback from the user perspective i.e. how they process, understand and react to the feedback. In this context, this PhD thesis reports on the design, development, and evaluation of a gait feedback system with two feedback modalities: haptic and augmented reality (AR). The initial part of this PhD work focused on evaluating different motion capture systems as part of an overall gait analysis system. The objective was to develop an alternative, cheaper and more accessible system. The proposed gait system (which included integrated camera and inertial sensors) was compared with the gold standard in motion capture. This was important to determine the most accurate capturing system to use in a feedback application. The next and major contributions of the PhD project focused on the design of a gait feedback system and evaluating the user Quality of Experience (QoE) of the two gait feedback modalities for knee alignment. The aim of the feedback is to reduce knee varus and valgus misalignments, which can cause serious orthopaedics problems. The QoE analysis aimed to understand how users perceived the proposed Haptic & AR systems in terms of utility, usability, interaction, and immersion. This involved assessing the easiness to adjust to feedback (utility), how easy the feedback was to understand (usability), how users interact with the feedback (interaction), and the awareness of body while moving (immersion). This analysis considered objective (improvement in knee alignment), subjective (questionnaire responses) user metrics, and implicit user metrics (e.g. physiological responses such as heart rate, electrodermal activity and eye information) from users. The findings show statistically significant higher QoE ratings for AR feedback. AR feedback also significantly reduces the number of varus iv misalignment (by 31%) when compared to baseline readings. Gender analysis showed significant differences in performance for the number of misalignments and time to correct valgus misalignment for AR feedback for males. The male AR group, the level of reduction for varus was 45% and 18% for valgus misalignments (p<0.05). Consistent with the male group, although to a lesser extent, AR feedback reduced the number of varus misalignments by 35% for the female subgroup (not significant when compared to the baseline). Physiological responses of participants to feedback stimuli are also presented. Event-based comparisons of heart rate showed higher variability for participants during haptic feedback, which could be an indicator of stress and was also reported in the male subgroup. The haptic and AR groups also showed significant differences in electrodermal activity (EDA) for varus, partial alignment (only one leg aligned) and complete aligned (both legs aligned), which could also indicate increased task demand. EDA signals were also filtered through frequency analysis. Pupil analysis reported that when a participant receives AR or haptic feedback and both legs were misaligned (varus or valgus), the pupil diameter was significantly greater for the haptic group, which could indicate an increased task demand. The final analysis dealt with bivariate correlations of physiological measures to check whether involuntary responses after feedback can be correlated and linked to stressful situations. The analysis reported that physiological measures of pupil, skin conductance, and heart rate are correlated to some extent.