Research Projects
Reframing Isolated Sign Recognition
Sign languages are used by 70 million people worldwide, but most information and communication resources are meant for written and spoken languages. Towards goals of language access and justice, ASL Citizen is a community curated isolated sign dataset of 2731 signs from American Sign Language. We propose this dataset (which contains over 83000 videos) be used for the task of video-based dictionary retrieval for ASL. We offer baseline models using state-of-the-art techniques, and define corresponding metrics for the task.
This work was part of my summer 2022 internship at MSR!
ASL Citizen: A Community-Sourced Dataset for Advancing Isolated Sign Language Recognition [arXiv]Supporting Speechreading
Speechreading is the art of using visual and contextual cues in the environment to support listening. Often used by d/Deaf and Hard-of-Hearing (d/DHH) individuals, it highlights nuances of rich communication. However, lived experiences of speechreaders are underdocumented in the literature, and the impact of online environment and interaction of captioning with speechreading has not been explored. To bridge these gaps, we conducted a three-part study consisting of formative interviews, design probes and design sessions with 12 d/DHH individuals who speechread.
Innovators with Insight
This work is a qualitative study of the design activities of persons with disabilities who have innovated accessibility technology. Through semi-structured interviews with innovators and a review of the historical record, we explore how life and learning experiences of these individuals have influenced their design choices. We aim to promote the articulation of these lived experiences and explore their perspectives as innovators and disabled people, as makers and users of assistive tech.
Exploring Embroidered Tactile Graphics
Tactile graphics convey rich visual information through a sense of touch, and are of accessible to blind or low vision (BLV) individuals. However, traditional techniques are expensive and require specialized knowledge. Here we explore the use of machine embroidery, which is affordable and readily available, to create tactile graphics. We have evaluated different embroidery stitches, patterns and techniques for tactile contrast.
Using Fiber Arts and Sonification to Improve Accessibility of Maker Spaces [CHI '22 Workshop]
Enhancing Access to High Quality Tangible Information [SCF Poster]