SEENS: Nuclei segmentation in Pap smear images with selective edge enhancement

Abstract

Accurate nuclei segmentation, as an indispensable basis and core link for multi-cell cervical image analysis, plays an important role in automatic pre-cancer detection. However, poor image quality due to the uneven staining, complex backgrounds and overlapped cell clusters poses a great challenge in nuclei segmentation. In this paper, we propose a new Selective-Edge-Enhancement-based Nuclei Segmentation method (SEENS). In the proposed method, selective search is integrated with mathematical operators to segment whole slide cervical images into small regions of interest (ROI) while automatically avoiding repeated segmentation as well as eliminating non-nuclei regions. In addition, an edge enhancement method based on the canny operator and mathematical morphology is presented to extract edge information as a weight to enhance the nucleus edge selectively. As a result, the enhanced ROI is then segmented by the Chan–Vese model with a higher accuracy. We evaluate our method with 18 whole slide images for a total of 395 cell nuclei. Experimental results demonstrate that SEENS achieves higher accuracy in cervical nuclei segmentation. Notably our method performs particularly better in low-contrast scenarios than baselines