Bangla Sign Language Recognition Using Concatenated BdSL Network

Abedin, Thasin; Prottoy, Khondokar S. S.; Moshruba, Ayana

IUT Repository Home
→
Electrical and Electronics Engineering (EEE)
→
Thesis
→
Undergraduate
→
2021
→
View Item

dc.contributor.author	Abedin, Thasin
dc.contributor.author	Prottoy, Khondokar S. S.
dc.contributor.author	Moshruba, Ayana
dc.date.accessioned	2022-05-04T15:44:12Z
dc.date.available	2022-05-04T15:44:12Z
dc.date.issued	2021-03-30
dc.identifier.citation	[1] Rawan A Al Rashid Agha, Muhammed N Sefer, and Polla Fattah. “A comprehensive study on sign languages recognition systems using (SVM, KNN, CNN and ANN)”. In: Proceedings of the First International Conference on Data Science, E-learning and Information Systems. 2018, pp. 1–6. [2] Fahmid Nasif Arko et al. “Bangla sign language interpretation using image processing”. PhD thesis. BRAC University, 2017. [3] Sebastiano Battiato et al. Image Analysis and Processing-ICIAP 2017: 19th International Conference, Catania, Italy, September 11-15, 2017, Proceedings, Part I. Vol. 10484. Springer, 2017. [4] Manas Kamal Bhuyan, Mithun Kumar Kar, and Debanga Raj Neog. “Hand pose identification from monocular image for sign language recognition”. In: 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA). IEEE. 2011, pp. 378–383. [5] Zhe Cao et al. “OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 43 (2021), pp. 172–186. [6] George Caridakis, Stylianos Asteriadis, and Kostas Karpouzis. “Nonmanual cues in automatic sign language recognition”. In: Personal and ubiquitous computing 18.1 (2014), pp. 37–46. [7] James Charles et al. “Automatic and efficient human pose estimation for sign language videos”. In: International Journal of Computer Vision 110.1 (2014), pp. 70–90. [8] Swagatam Das et al. Proceedings of the 4th International Conference on Frontiers in Intelligent Computing: Theory and Applications (FICTA) 2015. Vol. 404. Springer, 2015. [9] Muttaki Hasan, Tanvir Hossain Sajib, and Mrinmoy Dey. “A machine learning based approach for the detection and recognition of Bangla sign language”. In: 2016 International Conference on Medical Engineering, Health Informatics and Technology (MediTec). IEEE. 2016, pp. 1–5. [10] Oishee Bintey Hoque et al. “Real time bangladeshi sign language detection using faster r-cnn”. In: 2018 International Conference on Innovation in Engineering and Technology (ICIET). IEEE. 2018, pp. 1–6. [11] Kazuyuki Imagawa, Shan Lu, and Seiji Igi. “Color-based hands tracking system for sign language recognition”. In: Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition. IEEE. 1998, pp. 462–467. 32 Bibliography [12] Jason Isaacs and Simon Foo. “Hand pose estimation for american sign language recognition”. In: Thirty-Sixth Southeastern Symposium on System Theory, 2004. Proceedings of the. IEEE. 2004, pp. 132–136. [13] Md Islam et al. “Recognition Bangla Sign Language using Convolutional Neural Network”. In: Sept. 2019, pp. 1–6. DOI: 10.1109/3ICT. 2019.8910301. [14] Sanzidul Islam et al. “A potent model to recognize bangla sign language digits using convolutional neural network”. In: Procedia computer science 143 (2018), pp. 611–618. [15] K-Nearest Neighbor(KNN) Algorithm for Machine Learning. https://www. javatpoint . com / k - nearest - neighbor - algorithm - for - machine -learning? fbclid=IwAR0aGwUDx_5Kj2N2-Ucv-9cf5ck-jLnIr-TFTga5u7yHjl6jG2LM83GAQr0. [16] Nahua Kang. “Multi-Layer Neural Networks with Sigmoid Function—Deep Learning for Rookies (2)”. In: Towards Data Science (2017). [17] Oscar Koller et al. “Deep sign: hybrid CNN-HMM for continuous sign language recognition”. In: Proceedings of the British Machine Vision Conference 2016. 2016. [18] Oscar Koller et al. “Weakly supervised learning with multi-stream CNNLSTM-HMMs to discover sequential parallelism in sign language videos”. In: IEEE transactions on pattern analysis and machine intelligence (2019). [19] Yun Li et al. “A sign-component-based framework for Chinese sign language recognition using accelerometer and sEMG data”. In: IEEE transactions on biomedical engineering 59.10 (2012), pp. 2695–2704. [20] Marcelo Ortega. Training a Hand Detector like the OpenPose one in Tensorflow. https://medium.com/@apofeniaco/training-a-hand-detectorlike-the-openpose-one-in-tensorflow-45c5177d6679 Access Date: Feb 27, 2021 [21] Maria Parelli et al. “Exploiting 3D hand pose estimation in deep learningbased sign language recognition from RGB videos”. In: European Conference on Computer Vision. Springer. 2020, pp. 249–263. [22] Pose Estimation. https://www.tensorflow.org/lite/examples/pose_ estimation/overview [23] Abdul Muntakim Rafi et al. “Image-based Bengali Sign Language Alphabet Recognition for Deaf and Dumb Community”. In: 2019 IEEE Global Humanitarian Technology Conference (GHTC). IEEE. 2019, pp. 1–7. [24] Muhammad Rahaman et al. “Bangla Language Modeling Algorithm For Automatic Recognition of Hand-Sign-Spelled Bangla Sign Language”. In: Frontiers of Computer Science (electronic) 14 (Aug. 2018). DOI: 10 . 1007/s11704-018-7253-3. [25] Muhammad Aminur Rahaman. “Computer vision based Bangla sign language recognition”. PhD thesis. University of Dhaka, 2018. Bibliography 33 [26] Muhammad Aminur Rahaman et al. “Real-time computer vision-based Bengali sign language recognition”. In: 2014 17th International Conference on Computer and Information Technology (ICCIT). IEEE. 2014, pp. 192– 197. [27] P Subha Rajam and G Balakrishnan. “Real time Indian sign language recognition system to aid deaf-dumb people”. In: 2011 IEEE 13th international conference on communication technology. IEEE. 2011, pp. 737–742. [28] Sumit Saha. A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way. https://towardsdatascience.com/a-comprehensiveguide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53? fbclid=IwAR2krVrepGRcAstJc_eHBcrI2VP3mnCxZbrFRnb_4ZnbB1EmCbTLSPqGsso. Feb 27, 2021. [29] Lee Schlenker. Artificial Neural Networks: Man vs Machine. https : / / groupfuturista.com/blog/artificial-neural-networks-man-vsmachine/? fbclid=IwAR1kDq1ZA6ZA1J5YvMdLzbohs8smApCnJ3hW9fyGeO8cF092XZjJBha0QmQ. Access Date: Feb 27, 2021 [30] Shirin Sultana Shanta, Saif Taifur Anwar, and Md Rayhanul Kabir. “Bangla sign language detection using sift and cnn”. In: 2018 9th International Conference on Computing, Communication and Networking Technologies (ICCCNT). IEEE. 2018, pp. 1–6. [31] Andrzej Sieminski et al. Modern approaches for intelligent information and database systems. Vol. 769. Springer, 2018. [32] Joyeeta Singha and Karen Das. “Indian sign language recognition using eigen value weighted Euclidean distance based classification technique”. In: arXiv preprint arXiv:1303.0634 (2013). [33] Tomáš Sixta et al. “Fairface challenge at ECCV 2020: analyzing bias in face recognition”. In: European Conference on Computer Vision. Springer. 2020, pp. 463–481. [34] Thad Starner and Alex Pentland. “Real-time american sign language recognition from video using hidden markov models”. In: Motion-based recognition. Springer, 1997, pp. 227–243. [35] Thad Starner, Joshua Weaver, and Alex Pentland. “Real-time american sign language recognition using desk and wearable computer based video”. In: IEEE Transactions on pattern analysis and machine intelligence 20.12 (1998), pp. 1371–1375. [36] SUE Academics. https://academics.su.edu.krd/#1. [37] Support Vector Machine Algorithm. https : / / www . javatpoint . com / machine- learning- support- vector- machine- algorithm?fbclid= IwAR2LtSPbU4hRDZMFBGcuZD20fyWM_4GXmm915UDLv6m2fI4kAjMO15sQgxQ. [38] Nobuhiko Tanibata, Nobutaka Shimada, and Yoshiaki Shirai. “Extraction of hand features for recognition of sign language words”. In: International conference on vision interface. 2002, pp. 391–398. 34 Bibliography [39] Christian Vogler and Dimitris Metaxas. “Parallel hidden markov models for american sign language recognition”. In: Proceedings of the Seventh IEEE International Conference on Computer Vision. Vol. 1. IEEE. 1999, pp. 116–122. [40] Chunli Wang, Wen Gao, and Shiguang Shan. “An approach based on phonemes to large vocabulary Chinese sign language recognition”. In: Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition. IEEE. 2002, pp. 411–416. [41] Shengjing Wei et al. “A component-based vocabulary-extensible sign language gesture recognition framework”. In: Sensors 16.4 (2016), p. 556. [42] Su Yang and Qing Zhu. “Continuous Chinese sign language recognition with CNN-LSTM”. In: Ninth International Conference on Digital Image Processing (ICDIP 2017). Vol. 10420. International Society for Optics and Photonics. 2017, 104200F. [43] Farhad Yasir et al. “Sift based approach on bangla sign language recognition”. In: 2015 IEEE 8th International Workshop on Computational Intelligence and Applications (IWCIA). IEEE. 2015, pp. 35–39. [44] Zahoor Zafrulla et al. “American sign language recognition with the kinect”. In: Proceedings of the 13th international conference on multimodal interfaces. 2011, pp. 279–286. [45] R. K, e¯nin, š. “Land Cover Classification using Very High Spatial Resolution Remote Sensing Data and Deep Learning”. In: Latvian Journal of Physics and T	en_US
dc.identifier.uri	http://hdl.handle.net/123456789/1475
dc.description	Supervised by Mr.Safayat Bin Hakim, Assistant Professor Department of Electrical and Electronic Engineering Islamic University of Technology (IUT) Boardbazar, Gazipur-1704.	en_US
dc.description.abstract	Communication has always been a challenge for the deaf-mute community. So sign language is the only way of interaction for them. But the problem is that sign language is way too complex for the general mass. Keeping this in mind we propose an effective alternative tool to recognise Bangla Sign Language (BdSL) using computer vision for the people in Bangladesh. In our research we propose a novel architecture, namely "Concatenated BdSL Network" combining Convolutional Neural Network (CNN) as an "Image Network" for visual feature extraction and a pretrained "Pose Estimation Network" for extraction of the hand keypoints from hand gestures. This research will hold promising future aspects for real-time sign language interpretation.	en_US
dc.language.iso	en	en_US
dc.publisher	Department of Electrical and Electronic Engineering, Islamic University of Technology (IUT) The Organization of Islamic Cooperation (OIC) Board Bazar, Gazipur-1704, Bangladesh	en_US
dc.title	Bangla Sign Language Recognition Using Concatenated BdSL Network	en_US
dc.type	Thesis	en_US