TowardstheAutomaticTranslation创新.PDFVIP

Towards the Automatic Translation of American Sign Language Brandon Taylor Ph.D. Thesis Proposal November 29th, 2016 Human-Computer Interaction Institute, Carnegie Mellon University Committee: Dan Siewiorek (Chair) Anind Dey Carolyn Rose Roberta Klatzky Asim Smailagic Abstract There are estimated to be more than a million Deaf and severely hard of hearing individuals living in the United States. For many of these individuals, American Sign Language (ASL) is their primary means of communication. However, for most day- to-day interactions, native-ASL users must either get by with a mixture of gestures and written communication in a non-native language or seek the assistance of an interpreter. Whereas advances towards automated translation between many other languages have benefitted greatly from decades of research into speech recognition and Statistical Machine Translation, ASL’s lack of aural and written components have limited exploration into automated translation of ASL. Previous research efforts into sign language detection have met with limited success primarily due to inaccurately tracking handshapes. Without this vital component, research into ASL detection has been limited to focusing on isolated components of ASL or restricted vocabulary sets that reduce the need for accurate handtracking. However, improvements in 3D cameras and advances in handtracking techniques provide reasons to believe some of the technical sensing limitations may no longer exist. By combining state of the art handtracking techniques with ASL language modeling, there is an unexplored opportunity to develop a system capable of fully capturing ASL. In this work, I propose to develop the first ASL translation system capable of detecting all five necessary parameters of ASL (Handshape, Hand Location, Palm Orientation, Movement, and Non-Manual Featur