Enhancing mHealth App Onboarding Using a Multimodal Mixed Reality Avatar

Abstract

Navigating novel smartphone applications is often complex and unintuitive during the initial onboarding phase. Missing or ineffective guidance can cause misuse, errors, and abandonment, which is particularly critical in domains such as mobile health (mHealth) apps. While previous research has shown potential for onboarding with conversational avatars in mixed reality (MR), the impact of avatars combining gestures and voice for app tutorial guidance remains unknown. To address this gap, we conducted a within-subject study with 24 participants using a smart insole mHealth app as a case example. Participants completed tasks in MR under four conditions varying the presence of avatar voice and animated pointing gestures. Our findings show that tutorial guidance improved with voice and gesture compared to no avatar support, enhancing usability and learning experience while also highlighting remaining challenges. We contribute design implications for avatar-supported guidance in mHealth and broader human-computer interaction (HCI) contexts.