Bangla NER using Few-Shot Learning

Ferdous, Zannatul; Khandker, Nabliha; Chowdhury, Farzana

dc.contributor.author	Ferdous, Zannatul
dc.contributor.author	Khandker, Nabliha
dc.contributor.author	Chowdhury, Farzana
dc.date.accessioned	2025-03-06T08:35:05Z
dc.date.available	2025-03-06T08:35:05Z
dc.date.issued	2024-07-04
dc.identifier.citation	[1] J. Chen, Y. Lu, H. Lin, et al., Learning in-context learning for named entity recog nition, 2023. arXiv: 2305.11038 [cs.CL]. [Online]. Available: https://arxiv. org/abs/2305.11038. [2] X. Chen, K. Li, T. Song, and J. Guo, “Few-shot name entity recognition on stack overflow,” arXiv, vol. cs.CL, cs.AI, p. 5, Apr. 2024, arXiv:2404.09405 [pdf, other]. [3] S. S. S. Das, A. Katiyar, R. J. Passonneau, and R. Zhang, Container: Few-shot named entity recognition via contrastive learning, 2022. arXiv: 2109.07589 [cs.CL]. [Online]. Available: https://arxiv.org/abs/2109.07589. [4] N. Ding, G. Xu, Y. Chen, et al., “Few-NERD: A few-shot named entity recogni tion dataset,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Nat ural Language Processing (Volume 1: Long Papers), C. Zong, F. Xia, W. Li, and R. Navigli, Eds., Online: Association for Computational Linguistics, Aug. 2021, pp. 3198–3213. doi: 10.18653/v1/2021.acl-long.248. [Online]. Available: https://aclanthology.org/2021.acl-long.248. [5] E. V. Epure and R. Hennequin, Probing pre-trained auto-regressive language models for named entity typing and recognition, 2022. arXiv: 2108.11857 [cs.CL]. [Online]. Available: https://arxiv.org/abs/2108.11857. [6] Y. Ge, S. Das, K. O’Connor, M. A. Al-Garadi, G. Gonzalez-Hernandez, and A. Sarker, “Reddit-impacts: A named entity recognition dataset for analyzing clini cal and social effects of substance use derived from social media,” arXiv, vol. cs.CL, cs.AI, cs.LG, p. 7, May 2024, arXiv:2405.06145 [pdf, other]. [7] M. Z. Haque, S. Zaman, J. R. Saurav, S. Haque, M. S. Islam, and M. R. Amin, “B-ner: A novel bangla named entity recognition dataset with largest entities and its baseline evaluation,” IEEE Access, vol. 11, pp. 45 194–45 205, 2023. doi: 10.1109/ACCESS.2023.3267746. [8] B. Jehangir, S. Radhakrishnan, and R. Agarwal, “A survey on named entity recognition —datasets, tools, and methodologies,” Natural Language Process ing Journal, vol. 3, p. 100 017, 2023, issn: 2949-7191. doi: https : / / doi . 40 org / 10 . 1016 / j . nlp . 2023 . 100017. [Online]. Available: https : / / www . sciencedirect.com/science/article/pii/S2949719123000146. [9] H. Kim, J. Yoo, S. Yoon, J. Lee, and J. Kang, Simple questions generate named en tity recognition datasets, 2022. arXiv: 2112.08808 [cs.CL]. [Online]. Available: https://arxiv.org/abs/2112.08808. [10] P. Lai, F. Ye, L. Zhang, et al., “PCBERT: Parent and child BERT for Chinese few-shot NER,” in Proceedings of the 29th International Conference on Computa tional Linguistics, N. Calzolari, C.-R. Huang, H. Kim, et al., Eds., Gyeongju, Re public of Korea: International Committee on Computational Linguistics, Oct. 2022, pp. 2199–2209. [Online]. Available: https://aclanthology.org/2022. coling-1.192. [11] J. Li, A. Sun, J. Han, and C. Li, “A survey on deep learning for named entity recognition,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 1, pp. 50–70, Jan. 2022,issn: 2326-3865. doi: 10.1109/tkde.2020.2981314. [Online]. Available: http://dx.doi.org/10.1109/TKDE.2020.2981314. [12] Y. Li, Y. Yu, and T. Qian, Type-aware decomposed framework for few-shot named entity recognition, 2023. arXiv: 2302.06397 [cs.CL]. [Online]. Available: https: //arxiv.org/abs/2302.06397. [13] A. T. Liu, W. Xiao, H. Zhu, D. Zhang, S.-W. Li, and A. Arnold, Qaner: Prompting question answering models for few-shot named entity recognition, 2022. arXiv: 2203.01543 [cs.CL]. [Online]. Available: https://arxiv.org/abs/2203. 01543. [14] P. Liu, G. Wang, Y. Tong, J. Liang, Z. Ding, and H. Zhu, “Hybrid multi-stage de coding for few-shot ner with entity-aware contrastive learning,” arXiv, vol. cs.CL, arXiv:2404.06970 [pdf, other], Apr. 2024. [15] J. Ma, M. Ballesteros, S. Doss, et al., Label semantics for few shot named entity recognition, 2022. arXiv: 2203.08985 [cs.CL]. [Online]. Available: https:// arxiv.org/abs/2203.08985. [16] R. Ma, X. Zhou, T. Gui, et al., “Template-free prompt tuning for few-shot NER,” in Proceedings of the 2022 Conference of the North American Chapter of the Asso ciation for Computational Linguistics: Human Language Technologies, M. Carpuat, M.-C. de Marneffe, and I. V. Meza Ruiz, Eds., Seattle, United States: Association for Computational Linguistics, Jul. 2022, pp. 5721–5732. doi: 10.18653/v1/ 2022.naacl- main.420. [Online]. Available: https://aclanthology.org/ 2022.naacl-main.420. 41 [17] T. Ma, H. Jiang, Q. Wu, T. Zhao, and C.-Y. Lin, Decomposed meta-learning for few-shot named entity recognition, 2022. arXiv: 2204.05751 [cs.CL]. [Online]. Available: https://arxiv.org/abs/2204.05751. [18] M. Monajatipoor, J. Yang, J. Stremmel, et al., “Llms in biomedicine: A study on clinical named entity recognition,” arXiv, vol. cs.CL, arXiv:2404.07376 [pdf, other], Apr. 2024. [19] K. Pakhale,Comprehensive overview of named entity recognition: Models, domain specific applications and challenges, 2023. arXiv: 2309 . 14084 [cs.CL]. [On line]. Available: https://arxiv.org/abs/2309.14084. [20] A. Roy, Recent trends in named entity recognition (ner), 2021. arXiv: 2101.11420 [cs.CL]. [Online]. Available: https://arxiv.org/abs/2101.11420. [21] M. Tong, S. Wang, B. Xu, et al., Learning from miscellaneous other-class words for few-shot named entity recognition, 2021. arXiv: 2106.15167 [cs.CL]. [Online]. Available: https://arxiv.org/abs/2106.15167. [22] G. Wang, Y. Liu, and J. Hearne, “Few-shot learning for Sumerian named en tity recognition,” in Proceedings of the Third Workshop on Deep Learning for Low-Resource Natural Language Processing, C. Cherry, A. Fan, G. Foster, et al., Eds., Hybrid: Association for Computational Linguistics, Jul. 2022, pp. 136– 145. doi: 10 . 18653 / v1 / 2022 . deeplo - 1 . 15. [Online]. Available: https : //aclanthology.org/2022.deeplo-1.15. [23] L. Wang, R. Li, Y. Yan, et al., Instructionner: A multi-task instruction-based gen erative framework for few-shot ner, 2022. arXiv: 2203.03903 [cs.CL]. [Online]. Available: https://arxiv.org/abs/2203.03903. [24] P. Wang, R. Xu, T. Liu, et al., An enhanced span-based decomposition method for few-shot sequence labeling, 2022. arXiv: 2109.13023 [cs.CL]. [Online]. Avail able: https://arxiv.org/abs/2109.13023. [25] Z. Wang, K. Zhao, Z. Wang, and J. Shang, Formulating few-shot fine-tuning to wards language model pre-training: A pilot study on named entity recognition, 2022. arXiv: 2205.11799 [cs.CL]. [Online]. Available: https://arxiv.org/ abs/2205.11799. [26] H. Yan, B. Deng, X. Li, and X. Qiu, “TENER: adapting transformer encoder for named entity recognition,” CoRR, vol. abs/1911.04474, 2019. arXiv: 1911. 04474. [Online]. Available: http://arxiv.org/abs/1911.04474. [27] Y. Yang and A. Katiyar, Simple and effective few-shot named entity recognition with structured nearest neighbor learning, 2020. arXiv: 2010 . 02405 [cs.CL]. [Online]. Available: https://arxiv.org/abs/2010.02405. 42 [28] H. Zhang and Y. Zhuang, “A unified label-aware contrastive learning frame work for few-shot named entity recognition,” arXiv, vol. cs.CL, arXiv:2404.17178 [pdf, other], May 2024. [29] Y. Zhang and J. Yang, “Chinese NER using lattice LSTM,”CoRR, vol. abs/1805.02023, 2018. arXiv: 1805.02023. [Online]. Available: http://arxiv.org/abs/1805. 02023. [30] D. Zhou, S. Li, Q. Chen, and H. Yao, “Improving few-shot named entity recogni tion via semantics induced optimal transport,” Neurocomputing, vol. 597, p. 127 938, 2024, issn: 0925-2312. doi: https://doi.org/10.1016/j.neucom.2024. 127938. [Online]. Available: https://www.sciencedirect.com/science/ article/pii/S0925231224007094.	en_US
dc.identifier.uri	http://hdl.handle.net/123456789/2365
dc.description	Supervised by Dr. Hasan Mahmud, Associate Professor, Department of Computer Science and Engineering (CSE) Islamic University of Technology (IUT) Board Bazar, Gazipur, Bangladesh This thesis is submitted in partial fulfillment of the requirement for the degree of Bachelor of Science in Computer Science and Engineering, 2024	en_US
dc.description.abstract	Named Entity Recognition (NER) is a critical task in Natural Language Pro- cessing (NLP) for low-resource languages like Bengali. This paper explores few-shot learning for Bengali NER using ProtoBERT, NN-SHOT, and Struct- SHOT models with mBERT, XLM-RoBERTa, and BanglaBERT embeddings. We addressed class imbalance in the BNER dataset through oversampling tech- niques, significantly enhancing model performance. In our 5-way 5-shot ex- periment, we observed that XLM-RoBERTa generally yielded the highest F1 scores: 0.4045 for ProtoBERT, 0.378 for NN-SHOT, and 0.3912 for StructSHOT. This study underscores the importance of balanced datasets and suggests future research on optimizing sampling strategies and advanced model architectures.	en_US
dc.language.iso	en	en_US
dc.publisher	Department of Computer Science and Engineering(CSE), Islamic University of Technology(IUT), Board Bazar, Gazipur-1704, Bangladesh	en_US
dc.subject	NER, Few-Shot Learning, Low- Resource language, Bangla named entity, limited labeled data	en_US
dc.title	Bangla NER using Few-Shot Learning	en_US
dc.type	Thesis	en_US