Hand gesture recognition using depth information and DTW

Abstract

Human computer interaction has introduced a new paradigm for the construction of computer interfaces. This requires the construction of easy to use, natural and intuitive computer interfaces. Various methodologies have been considered to build such interfaces. Gestures and especially hand gesture represent the most easy to use, natural, intuitive and memorable way to communicate with computers. However, gesture recognition systems are difficult to design. These systems are often used in complex environments with cluttered background and different lighting conditions. These factors act as major constraints in the design of gesture recognition systems. However, the introduction of depth sensing devices such as the Microsoft Kinect makes it easy to capture hand gestures in complex backgrounds. The accuracy and efficiency of gesture recognition systems are often affected by the choice of features to be used to represent the hand shape and the algorithm to be used for classifying the hand gestures. In this thesis, we try to implement a hand gesture recognition system that makes use of the time-series representation of the contour points of the hand shape and uses Dynamic Time Warping (DTW) for classification of hand gestures. DTW is well known for its accuracy and effectiveness in matching time-series representations. Our proposed system makes use of the depth information of the scene provided by the Microsoft Kinect. This allows our system to be used in challenging backgrounds. We evaluate our system and compare it to some contemporary systems. The results of evaluation shows that our hand gesture recognition system is accurate and efficient with a mean accuracy of 94.6% and mean running time of 0.5179s. Our system is also invariant to scale, rotation and translation and runs effectively in complex background settings. We also survey some of the methods and algorithms used for recognising hand gestures. Finally we explore some of the application areas of gesture base interfaces. Interaction using gestures has proven to be useful in many domains such as in medicine, in classroom for teaching and presentations, in sign language recognition, interactive emotion recognition, simulations and in the gaming industry.