PointLSTM and Depth-CRNN based Hand Gesture Recognition

Haque, Amira; Rahman, Mirza Zamiur; Sayera, Reeshoon

dc.contributor.author	Haque, Amira
dc.contributor.author	Rahman, Mirza Zamiur
dc.contributor.author	Sayera, Reeshoon
dc.date.accessioned	2023-04-28T03:30:28Z
dc.date.available	2023-04-28T03:30:28Z
dc.date.issued	2022-05-30
dc.identifier.citation	[1] R. Q. Charles, H. Su, M. Kaichun, and L. J. Guibas, “PointNet: Deep Learning on point sets for 3D classification and segmentation,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. [2] K. Lai and S. N. Yanushkevich, “CNN+RNN depth and skeleton based dynamic hand gesture recognition,” arXiv [cs.CV], 2020. [3] Y. Min, Y. Zhang, X. Chai, and X. Chen, “An efficient PointLSTM for point clouds based gesture recognition,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. [4] M. Oudah, A. Al-Naji, and J. Chahl, “Hand gesture recognition based on computer vision: A review of techniques,” J. Imaging, vol. 6, no. 8, p. 73, 2020. [5] M. Yasen and S. Jusoh, “A systematic review on hand gesture recognition techniques, challenges and applications,” PeerJ Comput. Sci., vol. 5, p. e218, 2019. [6] “DHG - 14/28,” Telecom-lille.fr. [Online]. Available: http://wwwrech.telecomlille. fr/DHGdataset/. [Accessed: 23-Nov-2021]. [7] Lu, Qiang Xiao, Mingjie Lu, Yiyang Yuan, Xiaohui Yu, Ye. (2019). Attention- Based Dense Point Cloud Reconstruction From a Single Image. IEEE Access. PP. 1-1. 10.1109/ACCESS.2019.2943235. [8] Kingma, Diederik P., and Jimmy Ba. ”Adam: A method for stochastic optimization.” arXiv preprint arXiv:1412.6980 (2014). [9] M¨uller, Rafael, Simon Kornblith, and Geoffrey E. Hinton. ”When does label smoothing help?.” Advances in neural information processing systems 32 (2019). 10. Nakamura, Kensuke, et al. ”Learning-Rate Annealing Methods for Deep Neural Networks.” Electronics 10.16 (2021): 2029. [10] Loshchilov, Ilya, and Frank Hutter. ”Sgdr: Stochastic gradient descent with warm restarts.” arXiv preprint arXiv:1608.03983 (2016). [11] Gandhi, A. (2021, May 20). Data augmentation: How to use deep learning when you have limited data. AI amp; Machine Learning Blog. Retrieved 45 April 24, 2022, from https://nanonets.com/blog/data-augmentation-how-to-usedeep- learning-when-you-have-limited-data-part-2/ [12] Mahmud, H., Hasan, M., Kabir, M., Mottalib, M.A.: Recognition of symbolic gestures using depth information. Adv. Hum. Comput. Interact. (2018). [13] Mahmud, Hasan Morshed, Mashrur Hasan, Md Kamrul. (2021). “A deep-learning– based multimodal depth-aware dynamic hand gesture recognition system”. [14] Islam, Robiul Mahmud, Hasan Hasan, Md Kamrul Rubaiyeat, Husne. (2016). Alphabet Recognition in Air Writing Using Depth Information. [15] Amma, Christoph, Dirk Gehrig, and Tanja Schultz. ”Airwriting recognition using wearable motion sensors.” Proceedings of the 1st Augmented Human international Conference. 2010. [16] Xie, Lei Wang, Chuyu Bu, Yanling Sun, Jianqiang Cai, Qingliang Wu, Jie Lu, Sanglu. (2018). TaggedAR: An RFID-based Approach for Recognition of Multiple Tagged Objects in Augmented Reality Systems. IEEE Transactions on Mobile Computing. PP. 1-1. 10.1109/TMC.2018.2857812. [17] Zhou, B., Wan, J., Liang, Y., Guo, G. (2021). Adaptive cross-fusion learning for multi-modal gesture recognition. Virtual Reality Intelligent Hardware, 3(3), 235- 247. [18] ˇSpakov, Oleg, Howell Istance, Kari-Jouko R¨aih¨a, Tiia Viitanen, and Harri Siirtola. ”Eye gaze and head gaze in collaborative games.” In Proceedings of the 11th ACM Symposium on Eye Tracking Research Applications, pp. 1-9. 2019. [19] Mahmud, Hasan Islam, Robiul Hasan, Md Kamrul. (2022). On-air English Capital Alphabet (ECA) recognition using depth information. The Visual Computer. 38. 10.1007/s00371-021-02065-x. [20] Alam, Md, Ki-Chul Kwon, Mohammed Y. Abbass, Shariar Md Imtiaz, and Nam Kim. ”Trajectory-based air-writing recognition using deep neural network and depth sensor.” Sensors 20, no. 2 (2020): 376. 46 [21] Qi, Charles Ruizhongtai, Li Yi, Hao Su, and Leonidas J. Guibas. ”Pointnet++: Deep hierarchical feature learning on point sets in a metric space.” Advances in neural information processing systems 30 (2017).	en_US
dc.identifier.uri	http://hdl.handle.net/123456789/1856
dc.description	Supervised by Dr. Hasan Mahmud, Asst. Professor, Department of Computer Science and Engineering(CSE), Islamic University of Technology (IUT) Board Bazar, Gazipur-1704, Bangladesh. This thesis is submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Science and Engineering, 2022.	en_US
dc.description.abstract	Hand gestures represent spatiotemporal body language conveyed by various aspects of the hand, such as the palm, shape of the hand, and finger position, with the aim of conveying a particular message to the recipient. Computer Vision has different modalities of input, such as depth image, skeletal joint points or RGB images. Raw depth images are found to have poor contrast in the region of interest, which makes it difficult for the model to learn important information. Recently, in deep learning-based dynamic hand gesture recognition, researchers have attempted to combine different input modality to improve recognition accuracy. In this paper, we use depth quantized image features and point clouds to recognize dynamic hand gestures (DHG). We look at the impact of fusing depth-quantized features in Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) with point clouds in lstm-based multi-modal fusion networks.	en_US
dc.language.iso	en	en_US
dc.publisher	Department of Computer Science and Engineering(CSE), Islamic University of Technology(IUT), Board Bazar, Gazipur, Bangladesh	en_US
dc.subject	Dynamic-Hand-Gestures, Multimodal, Point Clouds, Depth Images, depth-quantized	en_US
dc.title	PointLSTM and Depth-CRNN based Hand Gesture Recognition	en_US
dc.type	Thesis	en_US