Hand gesture recognition with depth images_ A review.pdf

? Abstract— This paper presents a literature review on the use of depth for hand tracking and gesture recognition. The survey examines 37 papers describing depth-based gesture recognition systems in terms of (1) the hand localization and gesture classification methods developed and used, (2) the applications where gesture recognition has been tested, and (3) the effects of the low-cost Kinect and OpenNI software libraries on gesture recognition research. The survey is organized around a novel model of the hand gesture recognition process. In the reviewed literature, 13 methods were found for hand localization and 11 were found for gesture classification. 24 of the papers included real-world applications to test a gesture recognition system, but only 8 application categories were found (and three applications accounted for 18 of the papers). The papers that use the Kinect and the OpenNI libraries for hand tracking tend to focus more on applications than on localization and classification methods, and show that the OpenNI hand tracking method is good enough for the applications tested thus far. However, the limitations of the Kinect and other depth sensors for gesture recognition have yet to be tested in challenging applications and environments. I. INTRODUCTION Hand gesture recognition is used in human-robot interaction (HRI) to create user interfaces that are natural to use and easy to learn [1]. Sensors used for hand gesture recognition include wearable sensors such as data gloves and external sensors such as video cameras. Data gloves can provide accurate measurements of hand pose and movement, but they require extensive calibration, restrict natural hand movement, and are often very expensive. Video-based gesture recognition addresses these issues, but presents a new problem: locating the hands and segmenting them from the background in an image sequence is a non-trivial task, in particular when there are occlusions, lighting