login
Home / Papers / Visual-gestural Interface for Auslan Virtual Assistant

Visual-gestural Interface for Auslan Virtual Assistant

3 Citations•2023•
Maria Zelenskaya, Scott Whittington, Julie Lyons
SIGGRAPH Asia 2023 Emerging Technologies

A prototype smart home assistant that utilises gesture-based controls, improving accessibility and convenience for Deaf users and features Zelda, an interactive signing avatar that provides responses in Auslan (Australian Sign Language), enabling effective two-way communication.

Abstract

For Deaf people to use natural language interfaces, technologies must understand sign language input and respond in the same language. We developed a prototype smart home assistant that utilises gesture-based controls, improving accessibility and convenience for Deaf users. The prototype features Zelda, an interactive signing avatar that provides responses in Auslan (Australian Sign Language), enabling effective two-way communication. Our live demonstration includes gesture recognition and sign production.