Alphabet recognition in unconstrained Air Writing using Depth Information

Abstract

In this thesis, we present a machine learning approach to recognize on-air writing of English Capital Alphabets (ECAs) using di erent feature is introduced include depth information. The hand nger's motion while writing the alphabet in the air was captured as depth images with the help of a depth camera. The depth images were then processed to track nger movements and after that smoothing procedure was applied to generate hand trajectory data. 11 point-wise features including depth value were calculated from the hand trajectory data which are also time series. Each air written alphabet is then compared with 26 alphabet templates using Dynamic Time Warping (DTW). The DTW distance features are normalized between 0 to 1 and used as features. So, a feature vector of 11x26 =286 normalized features and the appropriate class label was fed to Support Vector Machine for training and testing. 15 fold cross veri cation classi cation result provided an average accuracy of 55.4% with 15 users. We also explored feature removal method based on a gain ratio. We removed the features that have the worst gain ratio. Iteratively 60 features were removed and the accuracies were compared. However, the best accuracy of 57.17% was found by removing eight features.