Bangla Sign Language Dataset Generation using Depth Information

Abstract

Sign Language Recognition (SLR) targets on interpreting the sign language into text or speech, so as to facilitate the communication between deaf-mute people and ordinary people. This task has broad social impact, but is still very challenging due to the complexity and large variations in hand actions. Existing dataset for SLR in our country is based on RGB images (converted to grayscale) and CNN is used for classification. However, it is difficult to design model to adapt to the large variations of hand gestures in the dataset, also the computational expense is high. Modern researches on other sign languages have shown that using depth information in Sign Language Recognition (SLR) gives better accuracy, which hasn't been introduced yet in our country. In this paper, we intend to build a complete Bangla Sign Language (BSL) dataset using depth information from depth images. In order to do, we’ll be collecting our depth information from our captured image samples using MediaPipe which is a cross-platform framework for. building multimodal applied machine learning pipeline. It is quite a new and advanced technology in hand tracking and gesture recognition. As opposed to the existing image dataset, we intend to build a feature-based depth dataset, that, if accurately modeled, is more likely to give better results than the existing one in sign language recognition (SLR).