Hostname: page-component-cd9895bd7-jn8rn Total loading time: 0 Render date: 2024-12-23T01:05:57.289Z Has data issue: false hasContentIssue false
  • English
  • Français

End-to-End Deep Learning-Based Cells Detection in Microscopic Leucorrhea Images

Published online by Cambridge University Press:  02 March 2022

Ruqian Hao Open the ORCID record for Ruqian Hao [Opens in a new window] ,
Xiangzhou Wang ,
Xiaohui Du Open the ORCID record for Xiaohui Du [Opens in a new window] ,
Jing Zhang ,
Juanxiu Liu  and
Lin Liu
Ruqian Hao
Affiliation:
School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, ChengDu, Sichuan 611731, China
Xiangzhou Wang
Affiliation:
School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, ChengDu, Sichuan 611731, China
Xiaohui Du
Affiliation:
School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, ChengDu, Sichuan 611731, China
Jing Zhang
Affiliation:
School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, ChengDu, Sichuan 611731, China
Juanxiu Liu
Affiliation:
School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, ChengDu, Sichuan 611731, China
Lin Liu*
Affiliation:
School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, ChengDu, Sichuan 611731, China
*
*Corresponding author: Lin Liu, E-mail: [email protected]
Get access

Abstract

Vaginitis is a prevalent gynecologic disease that threatens millions of women’s health. Although microscopic examination of vaginal discharge is an effective method to identify vaginal infections, manual analysis of microscopic leucorrhea images is extremely time-consuming and labor-intensive. To automate the detection and identification of visible components in microscopic leucorrhea images for early-stage diagnosis of vaginitis, we propose a novel end-to-end deep learning-based cells detection framework using attention-based detection with transformers (DETR) architecture. The transfer learning was applied to speed up the network convergence while maintaining the lowest annotation cost. To address the issue of detection performance degradation caused by class imbalance, the weighted sampler with on-the-fly data augmentation module was integrated into the detection pipeline. Additionally, the multi-head attention mechanism and the bipartite matching loss system of the DETR model perform well in identifying partially overlapping cells in real-time. With our proposed method, the pipeline achieved a mean average precision (mAP) of 86.00% and the average precision (AP) of epithelium, leukocyte, pyocyte, mildew, and erythrocyte was 96.76, 83.50, 74.20, 89.66, and 88.80%, respectively. The average test time for a microscopic leucorrhea image is approximately 72.3 ms. Currently, this cell detection method represents state-of-the-art performance.

Keywords

attention mechanismautomatic detectiondata augmentationdeep learningmicroscopic leucorrhea imagestransfer learningtransformer
Type
Biological Applications
Information
Microscopy and Microanalysis , Volume 28 , Issue 3 , June 2022 , pp. 732 - 743
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press on behalf of the Microscopy Society of America

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Carion, N, Massa, F, Synnaeve, G, Usunier, N, Kirillov, A & Zagoruyko, S (2020). End-to-End Object Detection with Transformers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12346 LNCS, pp. 213–229.CrossRefGoogle Scholar
Chawla, N, Bowyer, K, Hall, L & Kegelmeyer, WP (2002). SMOTE: Synthetic minority over-sampling technique. J Artif Intell Res 16, 321–357.CrossRefGoogle Scholar
Chen, S, Zhao, M, Wu, G, Yao, C & Zhang, J (2012). Recent advances in morphological cell image analysis. Comput Math Methods Med 2012, 101536.CrossRefGoogle ScholarPubMed
Díaz, E, Ayala, G, Díaz, ME, Gong, LW & Toomre, D (2010). Automatic detection of large dense-core vesicles in secretory cells and statistical analysis of their intracellular distribution. IEEE/ACM Trans Comput Biol Bioinform 7, 2–11.CrossRefGoogle Scholar
Du, X, Wang, X, Xu, F, Zhang, J, Huo, Y, Ni, G, Hao, R, Liu, J & Liu, L (2021). Morphological components detection for super-depth-of-field bio-micrograph based on deep learning. Microscopy 71, 50–59.CrossRefGoogle Scholar
Du, Y, Budman, HM & Duever, TA (2017). Segmentation and quantitative analysis of apoptosis of Chinese hamster ovary cells from fluorescence microscopy images. Microsc Microanal 23, 569–583.CrossRefGoogle ScholarPubMed
Dyk, DA & Meng, XL (2001). The art of data augmentation. J Comput Graph Stat 10, 1–50.Google Scholar
Egan, ME & Lipsky, MS (2000). Diagnosis of vaginitis. Am Fam Physician 62, 1095–1104.Google ScholarPubMed
Gonzalez, RC, Woods, RE & Masters, BR (2009). Digital image processing, third edition. J Biomed Opt 14, 029901.CrossRefGoogle Scholar
Haggerty, CL, Hillier, SL, Bass, DC & Ness, RB (2004). Bacterial vaginosis and anaerobic bacteria are associated with endometritis. Clin Infect Dis 39, 990–995.CrossRefGoogle ScholarPubMed
He, K, Gkioxari, G, Dollár, P & Girshick, R (2017). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969.CrossRefGoogle Scholar
He, K, Zhang, X, Ren, S & Sun, J (2016). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778.CrossRefGoogle Scholar
Hillier, SL, Nugent, RP, Eschenbach, DA, Krohn, MA, Gibbs, R, Martin, DH, Cotch, MF & Edelman, R (1996). Association between bacterial vaginosis and preterm delivery of a low- birth-weight Infant. Stud Fam Plann 27, 57.CrossRefGoogle Scholar
Irshad, H (2013). Automated mitosis detection in histopathology using morphological and multi-channel statistics features. J Pathol Inform 4, 10.CrossRefGoogle ScholarPubMed
Kuhn, HW (1955). The Hungarian method for the assignment problem. Nav Res Logist Q 2, 83–97.CrossRefGoogle Scholar
Li, H, Xiong, P, An, J & Wang, L (2018). Pyramid attention network for semantic segmentation. Preprint arXiv:180510180.Google Scholar
Li, TG, Wang, S & Zhao, N (2009). Gray-scale edge detection for gastric tumor pathologic cell images by morphological analysis. Comput Biol Med 39, 947–952. doi:10.1016/j.compbiomed.2009.05.010CrossRefGoogle ScholarPubMed
Lin, TY, Maire, M, Belongie, S, Hays, J, Perona, P, Ramanan, D, Dollár, P & Zitnick, CL (2014). Microsoft COCO: Common objects in context. European Conference on Computer Vision, pp. 740–755. Springer.CrossRefGoogle Scholar
Liu, W, Anguelov, D, Erhan, D, Szegedy, C, Reed, S, Fu, CY & Berg, AC (2016). SSD: Single Shot Multibox Detector. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9905 LNCS, pp. 21–37.CrossRefGoogle Scholar
Loshchilov, I & Hutter, F (2017). Decoupled weight decay regularization. Preprint arXiv:171105101.Google Scholar
Mahmoud, MAEFA, Aminou, HAK & Hashem, HA (2016). Are the fatty acids responsible for the higher effect of oil and alcoholic extract of Nigella sativa over its aqueous extract on Trichomonas vaginalis trophozoites? J Parasit Dis 40, 22–31. doi:10.1007/s12639-014-0479-6CrossRefGoogle ScholarPubMed
Mylonas, I & Bergauer, F (2011). Diagnosis of vaginal discharge by wet mount microscopy: A simple and underrated method. Obstet Gynecol Surv 66, 359–368.CrossRefGoogle ScholarPubMed
Nair, V & Hinton, GE (2010). Rectified linear units improve restricted Boltzmann machines. ICML.Google Scholar
Paczkowska, M, Chanaj-Kaczmarek, J, Romaniuk-Drapała, A, Rubiś, B, Szymanowska, D, Kobus-Cisowska, J, Szymańska, E, Winnicka, K & Cielecka-Piontek, J (2020). Mucoadhesive chitosan delivery system with Chelidonii herba lyophilized extract as a promising strategy for vaginitis treatment. J Clin Med 9, 1208.CrossRefGoogle ScholarPubMed
Paul, A, Dey, A, Mukherjee, DP, Sivaswamy, J & Tourani, V (2015). Regenerative random forest with automatic feature selection to detect mitosis in histopathological breast cancer images. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015. Lecture Notes in Computer Science, Navab, N, Hornegger, J, Wells, W & Frangi, A (Eds.), vol. 9350. Cham: Springer. https://doi.org/10.1007/978-3-319-24571-3_12.CrossRefGoogle Scholar
Prangemeier, T, Reich, C & Koeppl, H (2020). Attention-based transformers for instance segmentation of cells in microstructures. Proceedings - 2020 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2020, pp. 700–707.CrossRefGoogle Scholar
Redmon, J, Divvala, S, Girshick, R & Farhadi, A (2016). You only look once: Unified, real-time object detection. IEEE.CrossRefGoogle Scholar
Ren, S, He, K, Girshick, R & Sun, J (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39, 1137–1149.CrossRefGoogle ScholarPubMed
Sandfort, V, Yan, K, Pickhardt, PJ & Summers, RM (2019). Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks. Sci Rep 9, 1–9. doi:10.1038/s41598-019-52737-xCrossRefGoogle ScholarPubMed
Schmitz, C, Eastwood, BS, Tappan, SJ, Glaser, JR, Peterson, DA & Hof, PR (2014). Current automated 3D cell detection methods are not a suitable replacement for manual stereologic cell counting. Front Neuroanat 8, 1–13.CrossRefGoogle Scholar
Shen, CC, Yang, HH & Chang, YC (2012). A decision tree-based approach for Cervical smears. Int J Innov Comput Inf Control 8, 3251–3263.Google Scholar
Sommer, C, Fiaschi, L, Hamprecht, FA & Gerlich, DW (2012). Learning-based mitotic cell detection in histopathological images. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 2306–2309.Google Scholar
Soviany, P & Ionescu, RT (2019). Frustratingly Easy Trade-Off Optimization between Single-stage and Two-stage Deep Object Detectors. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11132 LNCS, pp. 366–378.CrossRefGoogle Scholar
Taha, TE, Hoover, DR, Dallabetta, GA, Kumwenda, NI, Mtimavalye, LA, Yang, LP, Liomba, GN, Broadhead, RL, Chiphangwi, JD & Miotti, PG (1998). Bacterial vaginosis and disturbances of vaginal flora: Association with increased acquisition of HIV. AIDS 12, 1699–1706.CrossRefGoogle ScholarPubMed
Tashk, A, Helfroush, MS, Danyali, H & Akbarzadeh, M (2013). An automatic mitosis detection method for breast cancer histopathology slide images based on objective and pixel-wise textural features classification. In The 5th Conference on Information and Knowledge Technology, pp. 406–410. doi: 10.1109/IKT.2013.6620101.CrossRefGoogle Scholar
Vaswani, A, Shazeer, N, Parmar, N, Uszkoreit, J, Jones, L, Gomez, AN, Kaiser, Ł. & Polosukhin, I (2017). Attention is all you need. Advances in Neural Information Processing Systems, December 2017, pp. 5999–6009.Google Scholar
Wang, Q, Bi, S, Sun, M, Wang, Y, Wang, D & Yang, S (2018). Deep learning approach to peripheral leukocyte recognition. PLoS ONE 14, 1–18.Google Scholar
Wang, Z, Zhang, L, Zhao, M, Wang, Y, Bai, H, Wang, Y, Rui, C, Fan, C, Li, J, Li, N, Liu, X, Wang, Z, Si, Y, Feng, A, Li, M, Zhang, Q, Yang, Z, Wang, M, Wu, W, Cao, Y, Qi, L, Zeng, X, Geng, L, An, R, Li, P, Liu, Z, Qiao, Q, Zhu, W, Mo, W, Liao, Q & Xu, W (2021). Deep neural networks offer morphologic classification and diagnosis of bacterial vaginosis. J Clin Microbiol 59, e02236-20.CrossRefGoogle ScholarPubMed
Yang, F, Mackey, MA, Ianzini, F, Gallardo, GM & Sonka, M (2004). Segmentation and quantitative analysis of the living tumor cells using large-scale digital cell analysis system. Medical Imaging 2004: Image Processing, San Diego, CA, United States, Vol. 5370, p. 1755.Google Scholar
Yin, Y & Liu, P (2010). Segmentation of urine sediment image based on improved Canny operator. Comput Eng Appl 46, 196–198.Google Scholar
Zhang, J, Wang, X, Ni, G, Liu, J, Hao, R, Liu, L, Liu, Y, Du, X & Xu, F (2021). Fast and accurate automated recognition of the dominant cells from fecal images based on Faster R-CNN. Sci Rep 11, 1–8. doi:10.1038/s41598-021-89863-4Google ScholarPubMed
Zhang, S, Chi, C, Yao, Y, Lei, Z & Li, SZ (2020). Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 9756–9765.CrossRefGoogle Scholar
Zhao, X, Zhou, P, Xu, K & Xiao, L (2021). An improved character recognition framework for containers based on DETR algorithm. Sensors 21, 4612.CrossRefGoogle ScholarPubMed