Analyzing and Advancing Unpaired PET to CT Image Translation: Improvements through Enhanced CycleGAN and CUT

Saikat, Tanvir Hossain; Drubo, Arafat Bin Amin; Arko, Najmus Sakib

dc.contributor.author	Saikat, Tanvir Hossain
dc.contributor.author	Drubo, Arafat Bin Amin
dc.contributor.author	Arko, Najmus Sakib
dc.date.accessioned	2025-03-06T08:18:44Z
dc.date.available	2025-03-06T08:18:44Z
dc.date.issued	2024-07-05
dc.identifier.citation	[1] T. Ahmed, A. Munir, S. Ahmed, M. B. Hasan, M. T. Reza, and M. H. Kabir, “Structure-enhanced translation from pet to ct modality with paired gans”, in Proceedings of the 2023 6th International Conference on Machine Vision and Applications, 2023, pp. 142–146. [2] H. B. Barua, G. Krishnasamy, K. Wong, A. Dhall, and K. Stefanov, “Histohdr net: Histogram equalization for single ldr to hdr image translation”, arXiv preprint arXiv:2402.06692, 2024. [3] R. Chen, W. Huang, B. Huang, F. Sun, and B. Fang, “Reusing discriminators for encoding: Towards unsupervised image-to-image translation”, in Proceed ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2020. [4] A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., “An image is worth 16x16 words: Transformers for image recognition at scale”, arXiv preprint arXiv:2010.11929, 2020. [5] V. Dumoulin, I. Belghazi, B. Poole, et al., “Adversarially learned inference”, arXiv preprint arXiv:1606.00704, 2016. [6] I. Goodfellow, J. Pouget-Abadie, M. Mirza, et al., “Generative adversarial net works”, Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020. [7] M.-H. Guo, T.-X. Xu, J.-J. Liu, et al., “Attention mechanisms in computer vi sion: A survey”, Computational visual media, vol. 8, no. 3, pp. 331–368, 2022. [8] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recog nition”, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778. [9] L. He, Q. Xu, H. Hu, and J. Zhang, “Fast and accurate sea-land segmentation based on improved senet and coastline database for large-scale image”, in 2018 Fifth International Workshop on Earth Observation and Remote Sensing Applications (EORSA), IEEE, 2018, pp. 1–5. 86 [10] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks”, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1125–1134. [11] J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution”, in Computer Vision–ECCV 2016: 14th Euro pean Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceed ings, Part II 14, Springer, 2016, pp. 694–711. [12] V. Kapoor, B. M. McCook, and F. S. Torok, “An introduction to pet-ct imag ing”, Radiographics, vol. 24, no. 2, pp. 523–543, 2004. [13] T. Karras, T. Aila, S. Laine, and J. Lehtinen, “Progressive growing of gans for improved quality, stability, and variation”, arXiv preprint arXiv:1710.10196, 2017. [14] C. Ledig, L. Theis, F. Huszár, et al., “Photo-realistic single image super-resolution using a generative adversarial network”, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4681–4690. [15] H.-Y. Lee, H.-Y. Tseng, J.-B. Huang, M. Singh, and M.-H. Yang, “Diverse image to-image translation via disentangled representations”, in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 35–51. [16] M.-Y. Liu, T. Breuel, and J. Kautz, “Unsupervised image-to-image translation networks”, Advances in neural information processing systems, vol. 30, 2017. [17] S. Liu, “Scam-gan: Generating brain mr images from ct scan data based on cyclegan combined with attention module”, in Journal of Physics: Conference Series, IOP Publishing, vol. 2646, 2023, p. 012 018. [18] Q. Luo, H. Li, Z. Chen, and J. Li, “Add-unet: An adjacent dual-decoder unet for sar-to-optical translation”, Remote Sensing, vol. 15, no. 12, p. 3125, 2023. [19] T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida, “Spectral normalization for generative adversarial networks”, arXiv preprint arXiv:1802.05957, 2018. [20] M. Özbey, O. Dalmaz, S. U. Dar, et al., Unsupervised medical image translation with adversarial diffusion models, 2023. arXiv: 2207.08208 [eess.IV]. [21] T. Park, A. A. Efros, R. Zhang, and J.-Y. Zhu, “Contrastive learning for un paired image-to-image translation”, in Computer Vision–ECCV 2020: 16th Eu ropean Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IX 16, Springer, 2020, pp. 319–345. 87 [22] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation”, in Medical image computing and computer assisted intervention–MICCAI 2015: 18th international conference, Munich, Ger many, October 5-9, 2015, proceedings, part III 18, Springer, 2015, pp. 234–241. [23] G. Santini, C. Fourcade, N. Moreau, et al., “Unpaired pet/ct image synthesis of liver region using cyclegan”, in Proceedings of SPIE - The International Society for Optical Engineering, vol. 115830T, 2020. [Online]. Available: https : / / doi.org/10.1117/12.2576095. [24] A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need”,Advances in neural information processing systems, vol. 30, 2017. [25] O. Vinyals, A. Toshev, S. Bengio, and D. Erhan, “Show and tell: A neural image caption generator”, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3156–3164. [26] T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, A. Tao, J. Kautz, and B. Catanzaro, “High resolution image synthesis and semantic manipulation with conditional gans”, in Proceedings of the IEEE conference on computer vision and pattern recogni tion, 2018, pp. 8798–8807. [27] X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks”, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7794–7803. [28] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “Cbam: Convolutional block atten tion module”, in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 3–19. [29] S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated residual transfor mations for deep neural networks”, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1492–1500. [30] T. Xu, P. Zhang, Q. Huang, et al., “Attngan: Fine-grained text to image gener ation with attentional generative adversarial networks”, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1316– 1324. [31] J. Yu, Z. Lin, J. Yang, X. Shen, X. Lu, and T. S. Huang, “Generative image inpainting with contextual attention”, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 5505–5514. [32] F. Zhang, H. Gao, and Y. Lai, “Detail-preserving cyclegan-adain framework for image-to-ink painting translation”,IEEE Access, vol. 8, pp. 132 002–132 011, 2020. 88 [33] H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena, “Self-attention genera tive adversarial networks”, in International conference on machine learning, PMLR, 2019, pp. 7354–7363. [34] H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for image restora tion with neural networks”, IEEE Transactions on computational imaging, vol. 3, no. 1, pp. 47–57, 2016. [35] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network”, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2881–2890. [36] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image transla tion using cycle-consistent adversarial networks”, in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2223–2232. [37] J.-Y. Zhu, R. Zhang, D. Pathak, et al., “Toward multimodal image-to-image translation”, Advances in neural information processing systems, vol. 30, 2017. [38] X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable detr: Deformable transformers for end-to-end object detection”, arXiv preprint arXiv:2010.04159, 2020	en_US
dc.identifier.uri	http://hdl.handle.net/123456789/2364
dc.description	Supervised by Mr. Tareque Mohmud Chowdhury, Assistant Professor, Ms.Sabrina Islam, Lecturer, Farzana Tabassum, Lecturer, Department of Computer Science and Engineering (CSE) Islamic University of Technology (IUT) Board Bazar, Gazipur, Bangladesh This thesis is submitted in partial fulfillment of the requirement for the degree of Bachelor of Science in Computer Science and Engineering, 2024	en_US
dc.description.abstract	Medical imaging plays a crucial role in healthcare, with PET and CT scans providing complementary information about metabolic activity and detailed anatomical structures, respectively. However, the reliance on paired PET-CT data poses challenges in terms of data availability, cost, and clinical variability. This research addresses the unpaired translation of PET images to CT images. Motivated by the potential for improved decision-making and the cost-effective utilization of unpaired data, this work aims to overcome the limitations of current medical imaging approaches & to address the effectiveness of this task. The study explores the use of unpaired image to image translation models for the given task including - CycleGAN, CUT, ,UNIT, MUNIT, SCAM-GAN etc & proposed improvements with the current state of the art result for this specific task. The literature review highlights the effectiveness of these models in various domains, emphasizing their loss functions, including adversarial, identity, cycle consistency & perceptual losses, adding attention mechanism and so on. To enhance the model’s capabilities, the research incorporates attention mechanisms aims to capture dependencies in images. The study evaluates the proposed methodology’s accuracy and applicability by comparing results with existing models. The research team aims to demonstrate the feasibility of solving this task by evaluating different state of the art unpaired models & proposes incorporations which gives better accuracy than the current models.	en_US
dc.language.iso	en	en_US
dc.publisher	Department of Computer Science and Engineering(CSE), Islamic University of Technology(IUT), Board Bazar, Gazipur-1704, Bangladesh	en_US
dc.subject	unpaired image to image translation problem, attention mechanism, cycleGAN, spatial attention, medical image translation	en_US
dc.title	Analyzing and Advancing Unpaired PET to CT Image Translation: Improvements through Enhanced CycleGAN and CUT	en_US
dc.type	Thesis	en_US