基于Kinect三维手语识别-计算机应用技术专业论文.docxVIP

Abstract Sign language is the main way to communicate with the outside for the deaf. Doing research on sign language recognition will help hearing-impaired people to communicate with others, and it is also an access to services provided by the society. We firstly investigated the status of sign language recognition, and then studied the hand tracking and gesture segmentation which were relevant with sign language. As for the problem what to be solved is Kinect-based data for sign language recognition, we studied the work of hand tracking and gesture segmentation. And Unlike the existing sign language recognition under the HMM framework, we propose a model based on the trajectory and key gesture fragment.. And the specific works are as follows: Firstly, we proposes a new hand tracking method by integrating appearance, depth and location. Combine with the face detection, we firstly automatically create a color model to achieve the automatic hand detection to determine the tracking target, and then achieve the hand tracking work by multi-cues fusion based the target estimation function and energy function to get the maximization value. In 44810 frames of the 300 videos, our correct tracking rate is 91 %. Secondly, we divided gestures into three types: the hands apart, hands coincidence and hand face coincidence. For the first type, we use color and depth to segment; and the second, we treated hands as a whole part; for the last one, it’s already unable to distinguish hands and face by color and depth, so we use clear facial information to remove the interference of face. Lastly, we think that it’s the key gesture fragment to distinguish the sign language, so we propose to use the overall trajectory and key gesture fragments to do sign language recognition. Using parallel HMM framework, our recognition rate on 370 Chinese sign language vocabularies reached 92.64%, it’s as well as the traditional HMM method, and we greatly reduced the time consumption. Ke