SeEar Project Image

SeEar

Enhancing Educational Experience of Deaf and Hard-of-Hearing Students using interactive Augmented Reality (AR) captions

Overview


Low-Cost Smart Augmented Reality Glasses: Provides localized and tailored captions to enhance the educational experience for hearing-impaired students.

Role


  • Led ideation, research, design, and development of software and prototype.
  • Followed a User-Centered Design (UCD) process with Co-Design methods to develop 30 AR prototypes with interactive, localized real-time captions.
  • Addressed key research questions:
    • Presentation and placement of captions that enhance verbal memory, comprehension, and engagement.
    • Visualizations for low-confidence transcriptions.
    • Strategies for wearer text visualizations.

User Studies and Findings


  • Conducted 5 user studies with 24 participants.
  • Quantitative and qualitative analysis showed significant improvements in:
    • Verbal memory, language comprehension, engagement, motivation, and language learning.

Recognition


  • First-author papers published at CHI 2024 and under review at IJHCI.
  • First Sri Lankan undergraduate to publish at CHI.