Human Facial Emotion Recognition Using Multi-head Cross Attention Network and Attention Consistency

Rahman, Md. Shakib Ur; Rafin, Dilir Daiyan; Hasan, Kazi Sajid

dc.contributor.author	Rahman, Md. Shakib Ur
dc.contributor.author	Rafin, Dilir Daiyan
dc.contributor.author	Hasan, Kazi Sajid
dc.date.accessioned	2024-09-04T10:21:42Z
dc.date.available	2024-09-04T10:21:42Z
dc.date.issued	2023-05-30
dc.identifier.citation	[1] Devansh Arpit, Stanis law Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. In In ternational conference on machine learning, pages 233–242. PMLR, 2017. 7 [2] Jie Cai, Zibo Meng, Ahmed Shehab Khan, Zhiyuan Li, James O’Reilly, and Yan Tong. Island loss for learning discriminative features in facial expression recogni tion. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pages 302–309. IEEE, 2018. 10 [3] Maurizio Corbetta and Gordon L Shulman. Control of goal-directed and stimulus driven attention in the brain. Nature reviews neuroscience, 3(3):201–215, 2002. 9 [4] Emre Dandıl and Rıdvan Ozdemir. Real-time facial emotion classification using ¨ deep learning. Data Science and Applications, 2(1):13–17, 2019. 8 [5] Abhinav Dhall, Roland Goecke, Simon Lucey, and Tom Gedeon. Collecting large, richly annotated facial-expression databases from movies. IEEE multimedia, 19(03):34–41, 2012. 21 [6] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 9, 26 [7] Amir Hossein Farzaneh and Xiaojun Qi. Discriminant distribution-agnostic loss for facial expression recognition in the wild. In Proceedings of the IEEE/CVF 28 Conference on Computer Vision and Pattern Recognition Workshops, pages 406– 407, 2020. 9, 25 [8] Amir Hossein Farzaneh and Xiaojun Qi. Facial expression recognition in the wild via deep attentive center loss. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 2402–2411, 2021. 10 [9] Beat Fasel and Juergen Luettin. Automatic facial expression analysis: a survey. Pattern recognition, 36(1):259–275, 2003. 5 [10] Hao Guo, Kang Zheng, Xiaochuan Fan, Hongkai Yu, and Song Wang. Visual at tention consistency under image transforms for multi-label image classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogni tion, pages 729–739, 2019. 11, 12 [11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 21, 24, 25, 26 [12] Akriti Jaiswal, A Krishnama Raju, and Suman Deb. Facial emotion detection using deep learning. In 2020 International Conference for Emerging Technology (INCET), pages 1–5. IEEE, 2020. 8 [13] Hanting Li, Mingzhe Sui, Feng Zhao, Zhengjun Zha, and Feng Wu. Mvt: mask vision transformer for facial expression recognition in the wild. arXiv preprint arXiv:2106.04520, 2021. 24, 25 [14] Shan Li, Weihong Deng, and JunPing Du. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. In Proceed ings of the IEEE conference on computer vision and pattern recognition, pages 2852–2861, 2017. 20, 22 [15] Richard Meyes, Melanie Lu, Constantin Waubert de Puiseau, and Tobias Meisen. Ablation studies in artificial neural networks. arXiv preprint arXiv:1901.08644, 2019. 22 [16] Dhara Mungra, Anjali Agrawal, Priyanka Sharma, Sudeep Tanwar, and Mo hammad S Obaidat. Pratit: a cnn-based emotion recognition system using his 29 togram equalization and data augmentation. Multimedia Tools and Applications, 79(3):2285–2307, 2020. 8 [17] Catherine Newmark. Charles darwin: the expression of the emotions in man and animals. In Schl¨usselwerke der Emotionssoziologie, pages 111–115. Springer, 2022. 5 [18] Ronald A Rensink. The dynamic representation of scenes. Visual cognition, 7(1- 3):17–42, 2000. 9 [19] Erika L Rosenberg and Paul Ekman. What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, 2020. 5 [20] Yahia Said and Mohammad Barr. Human emotion recognition based on facial expressions via deep learning on high-resolution images. Multimedia Tools and Applications, 80(16):25241–25253, 2021. 8 [21] Jiahui She, Yibo Hu, Hailin Shi, Jun Wang, Qiu Shen, and Tao Mei. Dive into ambiguity: Latent distribution mining and pairwise uncertainty estimation for facial expression recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6248–6257, 2021. 10 [22] Gurvinder Singh Shergill, Abdolhossein Sarrafzadeh, Olaf Diegel, and Aruna Shekar. Computerized sales assistants: the application of computer technology to measure consumer interest-a conceptual framework. 2008. 5 [23] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818– 2826, 2016. 26 [24] Sunil Thulasidasan, Tanmoy Bhattacharya, Jeff Bilmes, Gopinath Chennupati, and Jamal Mohd-Yusof. Combating label noise in deep learning using abstention. arXiv preprint arXiv:1905.10964, 2019. 7 30 [25] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 9 [26] Thanh-Hung Vo, Guee-Sang Lee, Hyung-Jeong Yang, and Soo-Hyung Kim. Pyra mid with super resolution for in-the-wild facial expression recognition. IEEE Ac cess, 8:131988–132001, 2020. 24, 25 [27] Kai Wang, Xiaojiang Peng, Jianfei Yang, Shijian Lu, and Yu Qiao. Suppressing uncertainties for large-scale facial expression recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6897– 6906, 2020. 7, 9, 10, 24, 25 [28] Zhengyao Wen, Wenzhong Lin, Tao Wang, and Ge Xu. Distract your attention: Multi-head cross attention network for facial expression recognition. arXiv preprint arXiv:2109.07270, 2021. 3, 5, 6, 7, 10, 12, 13, 14, 20, 22, 23, 24, 25 [29] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. Cbam: Con volutional block attention module. In Proceedings of the European conference on computer vision (ECCV), pages 3–19, 2018. 9 [30] Huan Yan, Yu Gu, Xiang Zhang, Yantong Wang, Yusheng Ji, and Fuji Ren. Mit igating label-noise for facial expression recognition in the wild. In 2022 IEEE In ternational Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2022. 7 [31] Jiabei Zeng, Shiguang Shan, and Xilin Chen. Facial expression recognition with inconsistently annotated datasets. In Proceedings of the European conference on computer vision (ECCV), pages 222–237, 2018. 10 [32] Yuhang Zhang, Chengrui Wang, and Weihong Deng. Relative uncertainty learn ing for facial expression recognition. Advances in Neural Information Processing Systems, 34:17616–17627, 2021. 9, 10, 24, 25 [33] Yuhang Zhang, Chengrui Wang, Xu Ling, and Weihong Deng. Learn from all: Erasing attention consistency for noisy label facial expression recognition. In Eu 31 ropean Conference on Computer Vision, pages 418–434. Springer, 2022. 5, 6, 10, 18, 24, 25 [34] Zhilu Zhang and Mert Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. Advances in neural information processing systems, 31, 2018. 7 [35] Zengqun Zhao, Qingshan Liu, and Feng Zhou. Robust lightweight facial expression recognition network with label distribution training. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 3510–3519, 2021. 25	en_US
dc.identifier.uri	http://hdl.handle.net/123456789/2155
dc.description	Supervised by Dr. Md. Hasanul Kabir Professor, Co-supervised by Mr. Shahriar Ivan Lecturer, Department of Computer Science and Engineering(CSE), Islamic University of Technology(IUT), Board Bazar, Gazipur-1704, Bangladesh	en_US
dc.description.abstract	In this report we discuss some methodologies related to facial expression recog nition(FER). Facial expressions can be recognized with the help of collective representation of multiple facial regions and proper decoding of the high-order interactions between the local features is very necessary to recognize a particu lar expression efficiently. However, if noise or inconsistency is present, the FER task becomes error-prone. Because of the noise involvement in the samples, the performance of a model degrades. So, it becomes compulsory to cope with the inconsistency at first. Thus we present a model which will try to recognize facial expressions with the ability to focus on multiple regions and tackle the noise in volvement or annotation ambiguity by suppressing it’s effect during the training	en_US
dc.language.iso	en	en_US
dc.publisher	Department of Computer Science and Engineering(CSE), Islamic University of Technology(IUT), Board Bazar, Gazipur-1704, Bangladesh	en_US
dc.title	Human Facial Emotion Recognition Using Multi-head Cross Attention Network and Attention Consistency	en_US
dc.type	Thesis	en_US